Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Parsing Json data columnwise in shell

Tags:

bash

shell

awk

When I run a command I get a response like this

{

    "status": "available",
    "managed": true,
    "name":vdisk7,
    "support":{
    "status": "supported"
    },
    "storage_pool": "pfm9253_pfm9254_new",
    "id": "ff10abad"-2bf-4ef3-9038-9ae7f18ea77c",
    "size":100
},

and hundreds of this type of lists or dictionaries I want a command that does such sort of a thing

if name = "something", 
    get the id

Any links that would help me in learning such sort of commands would be highly appreciated

I have tried

awk '{if ($2 == "something") print $0;}'

But I think the response is in Json so the colum wise awk formatting is not working.

Also it's just a single command that I need to run so I would prefer not to use any external library.

like image 292
The_Lost_Avatar Avatar asked Oct 22 '13 07:10

The_Lost_Avatar


People also ask

What is JQ in shell script?

So, to implement bash shell scripts using REST APIs, you must know how to parse the response of REST API (that is, parse JSON data). Jq is a program developed to filter JSON data. You can consider Jq like sed, awk, or grep program but designed specifically for filtering JSON data.

Can we parse JSON in Python?

If you have a JSON string, you can parse it by using the json. loads() method. The result will be a Python dictionary.

What does JQ do in bash?

jq command is used not only for reading JSON data but also to display data by removing the particular key. The following command will print all key values of Students. json file by excluding batch key. map and del function are used in jq command to do the task.

What is JSON explain the concept of parsing JSON Python code?

JSON (JavaScript Object Notation) is a popular data format used for representing structured data. It's common to transmit and receive data between a server and web application in JSON format. In Python, JSON exists as a string. For example: p = '{"name": "Bob", "languages": ["Python", "Java"]}'


1 Answers

JSON parser is better for this task

awk and sed are utilities to parse line-oriented text, but not json. What if your json formatting will change ? (some lines will go on one line ?).

You should use any standard json parser out there. Or use some powerful scripting language, such as PHP, Python, Ruby, etc.

I can provide you with example on how to do it with python.

What if I can't use powerful scripting language ?

If you totally unable to use python, then there is utility jq out there: link

If you have some recent distro, jq maybe already in repositories (example: Ubuntu 13.10 has it in repos).

I can use python!

I would do that using simple python inline script.

For example we have some some_command that returns json as a result.

We have to get value of data["name"].

Here we go:

some_command | python -c "import json, sys; print json.load(sys.stdin)['name']"

It will output vdisk7 in your case

For this to work you need to be sure, json is fully valid.

If you have a list of json objects:

[
  {
    ...
    "name": "vdisk17"
    ...
  },
  {
    ...
    "name": "vdisk18"
    ...
  },
  {
    ...
    "name": "vdisk19"
    ...
  },
...
]

You could use some list comprehensions:

some_command | python -c "import json, sys; [sys.stdout.write(x['name'] + '\n') for x in json.load(sys.stdin)]"

It will output:

vdisk17
vdisk18
vdisk19
like image 155
Waterlink Avatar answered Sep 22 '22 04:09

Waterlink