How to use the GitHub API from Bash?

Question

I have a repo that I'm writing an 'installation script' in Bash for. My plan is to have all of the scripts I want to install in a scripts/ directory under the repo root, and an script named install at the very top.

My plan is to have the user run something like this:

curl -sSL http://github.com/me/repo/raw/master/install | bash

and the install script will download all of the scripts in the directory on GitHub and place them in the appropriate place.

My question, how exactly would I do this from Bash? GitHub returns its API responses in JSON for example, so how would I parse the response contents in my script? Is there any way to change the response format so it would be Bash-parseable? (Bonus points if you provide a clear example for my specific use case.)

Thanks.

Answer 1

You shouldn't have to use the Github API for this. You can instruct the user to get the raw file:

curl -sSL https://raw.githubusercontent.com/me/repo/master/install | bash

This is a common idiom, apparently.

---

As an aside, I prefer using process substitution:

bash <(curl -sSL https://raw.githubusercontent.com/me/repo/master/install)

This leaves the standard input unharmed, if you wish to use it.

---

If I misunderstood, and you intend to use the API for getting the list of scripts, it will be multi-step process:

1. Get the branch's tree from https://api.github.com/repos/user/repo/branches/master from the url attribute.
2. The look up the directory in the entries of that tree (https://api.github.com/repos/user/repo/git/trees/hash) - look for entities of type tree, path being scripts.
3. Then look for entries of type blob in that tree (https://api.github.com/repos/user/repo/git/trees/hash)
4. Get each of those blobs, (https://api.github.com/repos/user/repo/git/blobs/hash) and then base64-decode the content attribute (the encoding is also specified in the encoding attribute, so there's that to be handled as well).

It would be far, far easier to just maintain a list of scripts in your scripts directory, fetch that using a curl -sSL https://raw.githubusercontent.com/me/repo/master/scripts/list-of-scripts and then curl them. You could then do, for example:

curl -sSL https://raw.githubusercontent.com/me/repo/master/scripts/list-of-scripts |
  xargs -L1 -i{} curl -sSL https://raw.github.com/me/repo/master/scripts/{}

---

If you really must use the API, the simplest way would be to use the jq tool. Download it in your script (change version, OS and architecture as needed):

curl -sSL https://github.com/stedolan/jq/releases/download/jq-1.5/jq-linux64 > jq
chmod +x jq

Now begins the long torturous journey:

tree_url=$(curl -sSL https://api.github.com/repos/user/repo/branches/master | 
  ./jq -r '.commit.commit.tree.url')
script_tree=$(curl -sSL "$tree_url" | 
  ./jq -r '.tree[] | select(.type == "tree" and .path == "scripts") | .url')
curl -sSL "$script_tree" | 
  ./jq -r '.tree[] | select(.type == "blob") | .url' | 
   xargs curl -sSLO

And now run base64 -d on each of the newly downloaded files (you could use a loop instead of xargs to make it easier).

Alternatively, for that last step, you could do:

curl -sSL "$script_tree" | 
  ./jq -r '.tree[] | select(.type == "blob") | .path' |
  xargs -L1 -i{} curl -sSLO https://raw.githubusercontent.com/me/repo/master/scripts/{}