Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Why does `inputs` skip the first line of the input file?

Tags:

json

csv

jq

Whenever I provide a file to jq and use the inputs command, I only get back all lines but the first one and I'm wondering why is that

I'm currently using jq 1.6 and I'm trying to use the inputs command to convert a TSV (Tab Separated Values) into JSON by grabbing the first line of the file as the headers and placing each header as the key of the corresponding value in the next lines

If I execute the following

echo -n 'line1'$'\n''line2' | jq -R 'inputs'

The result is

line2

and not

line1
line2

as I'd expect

As a workaround I'm currently prepending a new line to the input i'm giving to jq as in

echo -n $'\n''line1'$'\n''line2' | jq -R 'inputs'

but my expectation is to be able to use inputs and have it process the first line

like image 598
Miguel Sánchez Villafán Avatar asked May 05 '19 20:05

Miguel Sánchez Villafán


1 Answers

jq itself is reading the first line, then inputs (having received that line as its input) will read the rest. Typically, you want to use the -n option to prevent jq from doing any reading itself, letting any input or inputs filters do the actual reading.

$ echo -n $'line1\nline2\n' | jq -nR 'inputs'
"line1"
"line2"

In your case, letting jq read the header and inputs read the rest of the data is reasonable, but you have to do something with the header. This is probably more complicated than it needs to be, but does the job:

$ cat tmp.tsv
foo bar baz
1   2   3
4   5   6
$ jq -R 'split("\t") as $h | [inputs | split("\t") | [{key: $h[0], value: .[0]}, {key: $h[1], value: .[1]}, {key: $h[2], value: .[2]}] | from_entries]' tmp.tsv
[
  {
    "foo": "1",
    "bar": "2",
    "baz": "3"
  },
  {
    "foo": "4",
    "bar": "5",
    "baz": "6"
  }
]

jq reads the first line and splits it into the array h, then feeds that line to a filter which ignores it but uses inputs to read the rest of the lines, splitting each one up and creating a dict using (repeatedly) the value of $h.

like image 169
chepner Avatar answered Oct 17 '22 17:10

chepner