tools to work with json and curl from the console

Question

I'm giving my first steps with ElasticSearch and to test things I'm currently working with curl from the command line. However it's kind of annoying because I have to type everything on a text editor and paste in the console (no tab formatting in the console), and the json output you get is not formatted.

So I was wondering if there's some equivalente of the many sql consoles for working with json data, to make it easier to test and play around with json rest web services.

What tools do you use for this kind of tasks?

Answer 1

First of all, if the json output you get isn't formatted, append ?pretty=1 to any url. The result will be much nicer. Another option is to pipe the output to python -m json.tool if you have Python 2.6+. I like the latter option because I have it saved to a shell function and can simply type | j after any command that outputs json. Much shorter and generic.

For issuing curl commands there are a couple of options. The most universal solution is to create a shell script with the whole command. For example:

curl -s localhost:9200/foo/_search -d'
  {"query":
     {"query_string":
        {"query":"foo*"}}}' | python -m json.tool

Rather than pasting that at a shell prompt, save it in a file. Then just tweak the source and run it as often as you need.

% sh query

A variation on this is to use Emacs. I use it to run curl on regions of buffers, manipulate the result, etc. shell-command-on-region or M-| is a good choice there. So for the above script, I can issue M-| sh RET instead of switching to a shell. Vim allows you to do something similar.

I haven't checked this one out, but a lot of people seem to like elasticsearch-head. It's a UI that will issue queries and give you structured results. http://mobz.github.com/elasticsearch-head/

Update: I recently noticed that Perl's JSON module ships with a handy command-line utility called json_pp that function's similarly to Python's json.tool. If you install JSON::XS there's also a (faster) json_xs bin.