Perl: JSON fails if a thread is started

Question

Can someone please tell my why JSON is not working if some thread is started?

use strict;
use warnings;
use JSON;
use threads;
use threads::shared;

sub th {   }

threads->create(\&th)->join() if $ARGV[0];

my $json = to_json({ val => "123"});           # WTF?!?
print "$json\n";

Works fine and prints the JSON-string. But pass 1 as an argument to the script to create the thread and to_json will fail with

hash- or arrayref expected (not a simple scalar, use allow_nonref to allow this)

Same effect if I use encode_json insead. On the manpage of JSON the word thread is not present and I see no reason why a thread should harm an outside string-conversion.

???

Answer 1

JSON(.pm) is just a front end for JSON::PP, JSON::XS or Cpanel::JSON::XS.

You have found a bug in JSON::XS. About this, JSON::XS's documentation says:

(I-)THREADS

This module is not guaranteed to be ithread (or MULTIPLICITY-) safe and there are no plans to change this. Note that perl's builtin so-called theeads/ithreads are officially deprecated and should not be used.

[Note that the last part is incorrect. The official position is actually: Threads are hard, so you should use something else instead. It's highly questionable since the alternatives are arguably just as hard.]

Workaround: Use one of the other backends (directly or via JSON(.pm)).

$ PERL_JSON_BACKEND=JSON::XS 46793885 0
{"val":"123"}

$ PERL_JSON_BACKEND=JSON::XS 46793885 1
hash- or arrayref expected (not a simple scalar, use allow_nonref to allow this) at /home/ikegami/usr/perlbrew/perls/5.26.0t/lib/site_perl/5.26.0/JSON.pm line 170.

$ PERL_JSON_BACKEND=Cpanel::JSON::XS 46793885 1
{"val":"123"}

$ PERL_JSON_BACKEND=JSON::PP 46793885 1
{"val":"123"}

You can control this within the script by adding the following before loading JSON:

BEGIN { $ENV{PERL_JSON_BACKEND} = 'Cpanel::JSON::XS' }

Answer 2

I ran into this as well (trying to use JSON with multi-threaded perl). Without launching a background thread, my code worked fine, but got the same error you are getting when there was a thread launched.

Like you, I didn't find any help online specific to threading with regards to this error text. However, following the allow_nonref error text, I found the following in JSON::XS's documentation:

"OLD" VS. "NEW" JSON (RFC 4627 VS. RFC 7159)

TL;DR: Due to security concerns, JSON::XS will not allow scalar data in JSON >texts by default - you need to create your own JSON::XS object and enable allow_nonref:

my $json = JSON::XS->new->allow_nonref;

$text = $json->encode ($data);
$data = $json->decode ($text);

In your case, you are trying to call to_json, which internally creates a JSON object and calls encode on it. Unfortunately it doesn't specify allow_nonref in its constructor. So to make your code work, you can do something like this:

use JSON::XS;

my $json_obj = JSON::XS->new->allow_nonref;
my $json = $json_obj->encode({ val => "123"});
print "$json\n";

I came up with this solution before reading the other responses here, so those may be better solutions, but this should at least get you past the issue with minimal changes.

Answer 3

This is definitely to do with JSON and global state. If you require and import JSON, after the thread invocation, it 'works'.

The warning in the module for JSON::XS includes:

(I-)THREADS ^

This module is not guaranteed to be ithread (or MULTIPLICITY-) safe and there are no plans to change this

The 'workaround' for a not-thread-safe module is to not load it via use (which happens at 'compile' time) and instead require and import (at runtime) after the parallel instances of the program (threads) have been started.

E.g.:

use strict;
use warnings;

use threads;
use threads::shared;

sub th { }

my $th = threads->create( \&th )->join;

## Just inside main thread
##can do this within individual threads instead if desired
require JSON;
JSON->import;


my $json = to_json({ val => "123" });    # WTF?!?
print "\n$json\n";