How to desing RESTful advanced search/filter

Question

First of all, I have read RESTful URL design for search and How to design RESTful search/filtering? questions. I am trying to design more advanced options for searching in a simple and RESTful way. The answers to those questions have given me some insight and clues on how to design my previous application url pattern for search/filter functionality.

First, I came up with quite nice and simple solution for basic filtering options, using pattern:

Equality search: key = val

IN search: key = val1 & key = val2

But as application has grown, so were the search requirements. And I ended up with some rather unpleasant and complex url pattern for advanced searching options which included:

Negation search: key-N = val

Like search: key-L = val

OR search: key1-O = val1 & key2 = val2

Range search: key1-RS = val1 & key1-RE = val2

Whats more, beside filters, query has to get information about pagination and order by, so the filter parameter has F- suffix, order by fields has O- suffix, and pagination has P- suffix.

I hope that at this point I do not have to add that parsing such request is rather malicious task, with the possibility of ambiguity if key will contain '-'. I have created some regexp to parse it, and it works quite well as for now, but...

Now I am starting to write a new web app and I have the chance to redesign this piece from scratch.

I am wondering about creating object in a browser containing all information in structured and self-explanatory way and send it to server as as JSON string, like:

filter = {{'type':'like','field':key,'value':val1,'operator':'and','negation':false},..}

But I get strange feeling that this is not good idea - I really don't know why.

So, this would be the definition of my context. Now the question:

I am searching for simpler and safer pattern for implementing advanced search including options I mentioned above as RESTful GET parameters - can you share some ideas? Or maybe some insights on not doing this in a RESTful way? Also, if you see some pitfalls in JSON way, please share them.

EDIT:

I know what makes sending json as get parameter, not so good idea. Encoding it - it makes it ugly and hard to read.

Info provided by links sended by thierry templier, gave me something to think about and I managed to design more consistient and safe filter handling in GET parameters. Below is definition of syntax.

For filters - multiple F parameters (one for each search criterium):

F = OPERATOR:NEGATION:TYPE:FIELD:VAL[:VAL1,:VAL2...]

allowed values:

[AND|OR]:[T|F]:[EQ|LI|IN|RA]:FIELD_NAME:VALUE1:VALUE2...

For order by - multiple O parameters (one for each ordered field):

O = ODINAL_NO:DIRECTION:FIELD

allowed values:

[0-9]+:[ASC|DESC]:FIELD_NAME

Pagination - single P parameter:

P = ITEMS_PER_PAGE:FROM_PAGE:TO_PAGE

I think this will be good solution - it meets all my requirements, it is easy to parse and write, it is readable and I do not see how that syntax can become ambiguous.

I wloud appreciate any thoughts on that idea - do you see any pitfalls?

Answer 1

There are several options here. But it's clear that if your queries tend to be complex with operators, and so on... you can't use a set of query parameters. I see two approaches:

• Provide the query as JSON content to a method POST
• Provide the query in a query parameter with a specific format / grammar to a method GET

I think that you could have a look at what ElasticSearch for their queries. They are able to describe very complex queries with JSON contents (using several levels). Here is a link to their query DSL: http://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html.

You can also have a look at what OData does for queries. They choose another approach with a single query parameter $filter. Here are some links that can give you some examples: https://msdn.microsoft.com/en-us/library/hh169248(v=nav.70) and http://www.odata.org/documentation/odata-version-3-0/url-conventions/. This option requires to have a grammar on the server side to parse your query.

In general, this link could also give you some hints at this level in its section "Filtering data": https://templth.wordpress.com/2014/12/15/designing-a-web-api/.

Hope it gives you some helpful hints to design your queries within your RESTful services ;-)

Thierry