Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

yq (GO/Mike Farah) uniquify all arrays recursively

This is probably related to another question I posted: yq (GO/Mike Farah) sort all arrays recursively?

Mike Farah's yq provides documentation for making arrays unique but I'm having trouble figuring how to apply that to lists that are nested deeper

Input

classes:
  driver:
    fields:
      - height
      - age
      - age
  vehicle:
    fields:
      - model
      - model
      - color
      - year

Desired output

classes:
  driver:
    fields:
      - age
      - height
  vehicle:
    fields:
      - color
      - model
      - year

Naively trying to globally uniqify

cat to_sort.yaml | yq 'unique'                     

Error: Only arrays are supported for unique

And if it takes arguments, I don't know what to provide. I don't want to just sort one explicit path, but I did try this:

 cat to_sort.yaml | yq 'unique(.classes.driver.fields)'

Error: Bad expression, please check expression syntax

I have seen some yq examples where one has to do a select operation first, but I don't know what to try in this case.

like image 245
Mark Miller Avatar asked Oct 21 '25 14:10

Mark Miller


2 Answers

 yq e '(... | select(type == "!!seq")) |= unique' input

Will recursively loop over all the items, and select() those of type !!seq

Then update (|=) those with unique:

Result from provided input:

classes:
  driver:
    fields:
      - height
      - age
  vehicle:
    fields:
      - model
      - color
      - year

  • ...: Recursive Descent
like image 147
0stone0 Avatar answered Oct 23 '25 05:10

0stone0


You have to first traverse there, then update the array (here, using the update |= operator).

Either one after another:

yq '
  .classes.driver.fields |= unique
  | .classes.vehicle.fields |= unique
' to_sort.yaml

Or both at once:

yq '
  (.classes.driver.fields, .classes.vehicle.fields) |= unique
' to_sort.yaml

Both output

classes:
  driver:
    fields:
      - height
      - age
  vehicle:
    fields:
      - model
      - color
      - year
like image 43
pmf Avatar answered Oct 23 '25 05:10

pmf