Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to Remove \n and \r from JSON string using Bash?

Tags:

regex

bash

I've been trying to remove the new line and carriage return indicators from my JSON output using this answer.

I've also tried running this based on another answer I've seen on Stack Overflow, but it still does not appear to be working:

 sed -E ':a;N;$!ba;s/\r{0,1}\n/\\n/g' text.json > text_clean.json

My current .sh script looks like this:

getPage() {
curl --fail -X GET 'https://api.test.com/v1/marx?page=1&pageSize=1000&sort=desc' \
  -H 'Authorization: Bearer xxxxxxx' \
  -H 'cache-control: no-cache'  
}

getPage \
| jq -c '.data[]' \
  > text.json

What am I missing here?

If it helps, an example of the string containing it takes on many different forms in the output, but here's a good one:

Ok! Thank you so much.\r\nBest,\r\nClaire\r\n\r\n

like image 225
gooponyagrinch Avatar asked Feb 21 '19 02:02

gooponyagrinch


People also ask

How do you escape a new line in JSON?

JSON strings do not allow real newlines in its data; it can only have escaped newlines. Snowflake allows escaping the newline character by the use of an additional backslash character.

How does JSON handle new line?

In JSON object make sure that you are having a sentence where you need to print in different lines. Now in-order to print the statements in different lines we need to use '\\n' (backward slash). As we now know the technique to print in newlines, now just add '\\n' wherever you want.

What is carriage return in shell script?

The carriage return, also referred to as Ctrl+M, character would show up as an octal 15 if you were looking at the file with an od octal dump) command. The characters CRLF are often used to represent the carriage return and linefeed sequence that ends lines on Windows text files.


2 Answers

If you want to modify JSON, do it in jq.

In this case, that might mean changing:

getPage \
| jq -c '.data[]' \
> text.json

...to...

getPage \
| jq -c '.data[] | sub("\r\n"; " ") | sub("\n"; " ")' \
> text.json
like image 122
Charles Duffy Avatar answered Sep 20 '22 05:09

Charles Duffy


I would use tr for this.

In you example, try the following:

 echo "Ok! Thank you so much.\r\nBest,\r\nClaire\r\n\r\n"| tr -d '\012\015'
like image 36
LincolnP Avatar answered Sep 21 '22 05:09

LincolnP