Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I verify in Postgresql that JSON is valid?

I've got a big database with analytics data written in JSON.

I want to filter out rows with incorrect data:

  • invalid json (some rows has something like that: '{"hello": "world'
  • some attributes is not array so it would take '{"products": [1,2,3]}' and will leave out the '{"products": 1}'

I want to do something like that:

select * 
from analytics 
where (is_correct_json(json::json)) 
and (is_array(json::json->>'products'))

How can I achieve that?

like image 457
Jakub Troszok Avatar asked May 12 '15 10:05

Jakub Troszok


People also ask

How do I check if a JSON file is valid?

The simplest way to check if JSON is valid is to load the JSON into a JObject or JArray and then use the IsValid(JToken, JSchema) method with the JSON Schema. To get validation error messages use the IsValid(JToken, JSchema, IList<String> ) or Validate(JToken, JSchema, SchemaValidationEventHandler) overloads.

Does Postgres validate JSON?

postgres-json-schema allows validation of JSON schemas in PostgreSQL. It is implemented as a PL/pgSQL function and you can use it as a check constraint to validate the format of your JSON columns. postgres-json-schema supports the entire JSON schema draft v4 spec, except for remote (http) references.

How do I view JSON data in PostgreSQL?

2) Querying PostgreSQL JSON Data You can use the native PostgreSQL operators to query the data in PostgreSQL. The operator -> returns a JSON object field by key. The operator ->> returns a JSON object field by text.

How do I check if a JSON key exists in Postgres?

In Postgres, if you select a key that does not exist it will return null. so u can check the existence of a key by checking the null value of that key.


1 Answers

This is another good example why choosing the appropriate data type right from the start helps later ;)

There is no built-in function to check if a given text is valid JSON. You can however write your own:

create or replace function is_valid_json(p_json text)
  returns boolean
as
$$
begin
  return (p_json::json is not null);
exception 
  when others then
     return false;  
end;
$$
language plpgsql
immutable;

Caution: due to the exception handling this is not going to be fast. If you call that on many invalid values this is going to slow down your select massively.

However both '{"products": 1}' and '{"products": [1,2,3]}' are valid JSON documents. The fact that the former is invalid is based on your application logic, not on the JSON syntax.

To verify that you would need a similar function, that traps errors when calling json_array_length()

create or replace function is_valid_json_array(p_json text, p_element text)
  returns boolean
as
$$
begin
  return json_array_length( p_json::json -> p_element) >= 0;
exception 
  when others then
     return false;  
end;
$$
language plpgsql
immutable;
like image 110
a_horse_with_no_name Avatar answered Nov 15 '22 20:11

a_horse_with_no_name