Accessing nested JSON Fields in Common Lisp

Question

I have been playing with the spotify API in an effort to get comfortable with common Lisp. The language is a large departure from what I am used to and I am afraid I may be missing some understanding of how to parse objects and lists.

When I poll the spotify API looking for a song by title. It returns a lot of json which I access using cl-json decode method. However in order to access any of the fields I find myself doing long nested car cdr combinations. The cl-json returns not a listp or lista but a giant list of nested cons. It seems like a nightmare to parse. I am attaching the code below as an example. So what I want to ask is, What am I missing? This code looks way to cumbersome and messy to be the way its done in LISP, and I assume there is a nice way to map all the JSON to some type of object or hierarchical tree? Which would make querying easier.

(ql:quickload '(:cl-json :drakma))
(defvar *auth-token*)   
(defvar *results*)
(setq *url* "https://api.spotify.com/v1/search")
(push (cons "application" "json") drakma:*text-content-types*)
(setf drakma:*header-stream* *standard-output*)

(defun search-spotify (&optional (search-param "blue moon")  (search-type "track") (limit 5))
  "Perform a search on the API for track, artist, or album information"
  (defvar complete-url "")
  (defvar url nil)
  (defvar params nil)
  (setq url "https://api.spotify.com/v1/search")
  (setq params (format nil "?q=~a&type=~a&limit=~a" (replace-all search-param " " "+") (replace-all search-type " " "+") limit))
  (setq complete-url (format nil "~a~a" url params))
  (print complete-url)
  (setq *results* (cl-json:decode-json-from-string
                   (drakma:http-request complete-url
                                        :method :get
                                        :additional-headers `(("Authorization" . ,(format nil "Bearer ~a" (cdar *auth-token*))))
                                        ))))

(defun print-spotify-artist ()
  (loop for result in (cdr (caddar *results*))
        do (format t "Artist Name: ~a Spotify ID: ~a ~%" (cdadr (cddadr (caddar result))) (cdar (cdaadr (caddar result))))))

Running the above code and calling the print-spotify-artist function will print a list as follows:

Artist Name: Eve 6 Spotify ID: https://open.spotify.com/artist/4Eqd24yS5YcxI8b6Xfuwr8 Artist Name: Bill Murray Spotify ID: https://open.spotify.com/artist/3wkZ8WTrs7WcfE13voUCK1 Artist Name: Various Artists Spotify ID: https://open.spotify.com/artist/0LyfQWJT6nXafLPZqxe9Of Artist Name: Pat Martino Spotify ID: https://open.spotify.com/artist/4DlMMgnldzX6OkCskmeGKz Artist Name: Jess & Zeb Spotify ID: https://open.spotify.com/artist/1oAndP8vmGtTlB6mbpieJs

Example JSON return, you get this returned for each song, for this example data I set the maximum results to 1. Notice that it is heavily nested. The below is what is returned from decode json function of cl-json.

    ((:TRACKS
  (:HREF
   . "https://api.spotify.com/v1/search?query=open%2Broad%2Bsong&type=track&offset=0&limit=1")
  (:ITEMS
   ((:ALBUM (:ALBUM--TYPE . "album")
     (:ARTISTS
      ((:EXTERNAL--URLS
        (:SPOTIFY . "https://open.spotify.com/artist/4Eqd24yS5YcxI8b6Xfuwr8"))
       (:HREF . "https://api.spotify.com/v1/artists/4Eqd24yS5YcxI8b6Xfuwr8")
       (:ID . "4Eqd24yS5YcxI8b6Xfuwr8") (:NAME . "Eve 6") (:TYPE . "artist")
       (:URI . "spotify:artist:4Eqd24yS5YcxI8b6Xfuwr8")))
     (:AVAILABLE--MARKETS "AD" "AR" "AT" "AU" "BE" "BG" "BO" "BR" "CA" "CH"
      "CL" "CO" "CR" "CY" "CZ" "DE" "DK" "DO" "EC" "EE" "ES" "FI" "FR" "GB"
      "GR" "GT" "HK" "HN" "HU" "ID" "IE" "IL" "IS" "IT" "JP" "LI" "LT" "LU"
      "LV" "MC" "MT" "MY" "NI" "NL" "NO" "NZ" "PA" "PE" "PH" "PL" "PT" "PY"
      "RO" "SE" "SG" "SK" "SV" "TH" "TR" "TW" "US" "UY" "VN" "ZA")
     (:EXTERNAL--URLS
      (:SPOTIFY . "https://open.spotify.com/album/1qJOmC60ez9RNWPg4ELMBW"))
     (:HREF . "https://api.spotify.com/v1/albums/1qJOmC60ez9RNWPg4ELMBW")
     (:ID . "1qJOmC60ez9RNWPg4ELMBW")
     (:IMAGES
      ((:HEIGHT . 639)
       (:URL
        . "https://i.scdn.co/image/3f7891ffa993954dd72ed50245280b15f5db5844")
       (:WIDTH . 621))
      ((:HEIGHT . 300)
       (:URL
        . "https://i.scdn.co/image/16f1a6277b827fd97cb450c5dc246d29c2ed1d52")
       (:WIDTH . 291))
      ((:HEIGHT . 64)
       (:URL
        . "https://i.scdn.co/image/935aca4d861213415fc64f780b0e9e5a0e8d865c")
       (:WIDTH . 62)))
     (:NAME . "Eve 6") (:RELEASE--DATE . "1998")
     (:RELEASE--DATE--PRECISION . "year") (:TYPE . "album")
     (:URI . "spotify:album:1qJOmC60ez9RNWPg4ELMBW"))
    (:ARTISTS
     ((:EXTERNAL--URLS
       (:SPOTIFY . "https://open.spotify.com/artist/4Eqd24yS5YcxI8b6Xfuwr8"))
      (:HREF . "https://api.spotify.com/v1/artists/4Eqd24yS5YcxI8b6Xfuwr8")
      (:ID . "4Eqd24yS5YcxI8b6Xfuwr8") (:NAME . "Eve 6") (:TYPE . "artist")
      (:URI . "spotify:artist:4Eqd24yS5YcxI8b6Xfuwr8")))
    (:AVAILABLE--MARKETS "AD" "AR" "AT" "AU" "BE" "BG" "BO" "BR" "CA" "CH" "CL"
     "CO" "CR" "CY" "CZ" "DE" "DK" "DO" "EC" "EE" "ES" "FI" "FR" "GB" "GR" "GT"
     "HK" "HN" "HU" "ID" "IE" "IL" "IS" "IT" "JP" "LI" "LT" "LU" "LV" "MC" "MT"
     "MY" "NI" "NL" "NO" "NZ" "PA" "PE" "PH" "PL" "PT" "PY" "RO" "SE" "SG" "SK"
     "SV" "TH" "TR" "TW" "US" "UY" "VN" "ZA")
    (:DISC--NUMBER . 1) (:DURATION--MS . 194866) (:EXPLICIT)
    (:EXTERNAL--IDS (:ISRC . "USRC19806851"))
    (:EXTERNAL--URLS
     (:SPOTIFY . "https://open.spotify.com/track/7kAKO1EYHt2MVlombUuoLN"))
    (:HREF . "https://api.spotify.com/v1/tracks/7kAKO1EYHt2MVlombUuoLN")
    (:ID . "7kAKO1EYHt2MVlombUuoLN") (:IS--LOCAL) (:NAME . "Open Road Song")
    (:POPULARITY . 44)
    (:PREVIEW--URL
     . "https://p.scdn.co/mp3-preview/b793285aeeb4a1176b93dc739ffb361c6aabf4e5?cid=b067abcbc67f4ceba0d61e414926c9f5")
    (:TRACK--NUMBER . 5) (:TYPE . "track")
    (:URI . "spotify:track:7kAKO1EYHt2MVlombUuoLN")))
  (:LIMIT . 1)
  (:NEXT
   . "https://api.spotify.com/v1/search?query=open%2Broad%2Bsong&type=track&offset=1&limit=1")
  (:OFFSET . 0) (:PREVIOUS) (:TOTAL . 43))) 

Answer 1

The Access library might help, it allows to access nested data structures.

(defparameter my-plist (list :foo "foo" :bar "bar"))

;; bar is a plist
(defclass obj-test ()
  ((foo :accessor foo :initarg :foo :initform :foo)
   (bar :accessor bar :initarg :bar :initform (copy-list MY-PLIST))))

(defvar my-obj ((make-instance 'obj-test))

(accesses MY-OBJ 'bar 'foo) ;; => "foo"

There is a way to use dotted paths, if that is to your liking.

Access is battle tested, it is the heart of the Djula templating engine, one of the most downloaded libraries on Quicklisp.

Access also allows consistent access across data structures.

Deeper overview: https://lisp-journey.gitlab.io/blog/generice-consistent-access-of-data-structures-dotted-path/