Removing ": " from JSON-like data

Question

I have a 1,00,000-line JSON text file. Manual extraction is not fair. I have written a Perl program to read each line of the file, which meets my needs.

Here's a sample text file

Sample.txt

  "key": "Programming",
  "doc_count": 1
  
  "key": "Base",
  "doc_count": 1,
  
  "key": "Experience",
  "doc_count": 1

  "key": "Electrophoresis",
  "doc_count": 1

I would like to take the key value alone delimited by double brackets, say Programming, Base, Experience and Electrophoresis.

Here's the Perl code that I tried:

ExtractKeyValue.pl

use strict;
use warnings;

my $file = $ARGV[0];
open my $info, $file or die "Could not open $file: $!";

while ( my $line = <$info> ) {
    if ( $line =~ /"key(.*)",/ ) {
        print $1;
        print "\n";
    }
}

close $info;

By using this, I am getting this output

": "Programming
": "Base
": "Experience
": "Electrophoresis

I don't want the leading colon and space.

I have tried $line =~ /"key: "(.*)",/. But it is not working. The command executes but no output and no error symptoms.

G:\ExtractKeyValue_Regex>perl ExtractKeyValue.pl Sample.txt > Output_Sample.txt

G:\ExtractKeyValue_Regex>

The output should be like,

Expected Output:

Programming
Base
Experience
Electrophoresis

I could not figure out why the colon : and space and double quotes " are not tracked by the pattern.

Answer 1

With the lines you show all you need is

my $key_assoc = $line =~ /: "([^"]+)/;

print "$key_assoc\n" if $key_assoc;

Or you can throw in the "key" string and ", for extra assurance and a format check

if ($line =~ /"key": "([^"]+)",/) {
    # ...
}

Note that + makes it not capture anything if there are empty quotes ("key": "",) while .* would get you the empty string in that case. A detail which may not matter, but they are different.

---

Note that the use of a single literal space   in a regex is prone to being overlooked (or doubled) and can also get swallowed if /x is added later, introducing a bug if not tested.

Better ways to specify a single simple space? We can use \x20, or Unicode \N{SPACE} via the charnames pragma. Thanks to Borodin for bringing this up and specifying these two.

A nice workaround, from Sinan Ünür, is to use a character class with space only, /a [ ] b/x. This both emphasizes that a literal space is intended (without /x), and allows it under /x.

Another fix is to explicitly escape it, /\ /, which draws attention to it and protects it from /x.

Answer 2

You should use this:

"key": "(.*?)"

Do it in the regex check:

if($line =~ /"key": "(.*?)",/){
   ...

This way, the first capture group ($1) will contain exactly what you want.