Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How can I output the whole document in jq while replacing an item based on a field's value?

I am attempting to use jq to parse a AWS CloudFront configuration JSON file and change a few values so that I can then issue an update statement with that configuration.

The Document

The document format snippet that matters, with redacted values, is:

{
    "ETag": "REDACTED",
    "DistributionConfig": {
        "Origins": {
            "Quantity": 2,
            "Items": [
                {
                    "Id": "redacted-1",
                    "DomainName": "redacted1.us-east-1.amazonaws.com",
                    "OriginPath": "/redacted"
                },
                {
                    "Id": "redacted-2",
                    "DomainName": "redacted2.s3.amazonaws.com",
                    "OriginPath": ""
                }
            ]
        }
    }    
}

What I'm trying to do

I want to output the entire document, but:

  • I want to set the ETag value to ""
  • On the 2nd item in the items array, I want to set the OriginPath to a value of my choosing

(this is to support our CI/CD process being able to point the CloudFront distribution to a new folder of just-deployed code within an S3 bucket. I want to modify the existing configuration in those specific ways but leave the rest in-tact.)

Something that works...sort of

A jq filter of . | (.DistributionConfig.Origins.Items[1].OriginPath = "Hello") | .ETag = "" does what I need it to do, resulting in:

{
  "ETag": "", // correctly updated
  "DistributionConfig": {
    "Origins": {
      "Quantity": 2, // correctly retained
      "Items": [
        {
          "Id": "redacted-1",
          "DomainName": "redacted1.us-east-1.amazonaws.com",
          "OriginPath": "/redacted"
        },
        {
          "Id": "redacted-2",
          "DomainName": "redacted2.s3.amazonaws.com",
          "OriginPath": "Hello" // correctly updated
        }
      ]
    }
  }
}

Where I'm having trouble

The solution above works...as long as I'm referring to the 2nd item in the array. But I'm not always sure it will be the second item in the array.

So instead, I'd like to match based on the Id property.

Any solutions I've been able to find that do this seem to filter the document to that section of JSON, rather than matching and updating the value as part of outputting the document.

Question

How can I filter an array for a field that has a certain value, and still output the whole document?

Or put another way -- given the document above, how can I change:

. | (.DistributionConfig.Origins.Items[1].OriginPath = "Hello") | .ETag = ""

To something that lets me refer to .Id="redaacted-2" rather than Items[1]?

A link to a demo

Demo of the script at https://jqplay.org/s/ZQ7XcM5-BY in case anyone wants to try an answer.

like image 287
SeanKilleen Avatar asked Dec 23 '22 15:12

SeanKilleen


2 Answers

The basic way to accomplish edits such as these is using the syntax:

PATHSPEC = VALUE

or if the value depends on PATHSPEC in a certain way:

PATHSPEC |= VALUE

where PATHSPEC is a jq path specification.

Using this principle, in your case, you could write:

.ETag = ""
| .DistributionConfig.Origins.Items[1].OriginPath = "myvalue"

or, if you want to base the second update on .Id:

.ETag = ""
| .DistributionConfig.Origins.Items[] |=
   if .Id == "redacted-2" then .OriginPath = "myvalue" else . end

The example immediately above can be seen in action at https://jqplay.org/s/u5xbhbSs4l

Variations

There are of course numerous variations. For example, you could use .Quantity as the index of the item to be updated:

.ETag = ""
| .DistributionConfig.Origins.Quantity as $ix
| .DistributionConfig.Origins.Items[$ix - 1].OriginPath = "myvalue"

or more DRYly:

.ETag = ""
| .DistributionConfig.Origins |=
    (.Items[.Quantity  - 1].OriginPath = "myvalue")

like image 164
peak Avatar answered Dec 28 '22 13:12

peak


Just a minor variation on @peak's answer, using select to choose which item(s) to update:

.ETag = ""
| (.DistributionConfig.Origins.Items[] | 
      select(.Id == "redacted-2")
  ).OriginPath = "foo"

The second assignment receives the entire original input as its input, so produces it as its output. The parenthesized filter, though, selects only the appropriate element(s) of Items whose OriginPath key you want to update.

like image 23
chepner Avatar answered Dec 28 '22 12:12

chepner