git bitbucket private submodule ssh alias issues

Question

git version 1.7.1 ( I have no power over this, on this server )

I have a private repo with several submodules, some of which are also private. The submodules which are not private are coming in without issue. However the first private repo encountered returns this ssh: Could not resolve hostname bitbucket.org:username: Name or service not known, and the process stops here.

As per the docs, I have set up a deployment key for the staging server and have created an ~/.ssh/config alias to point bitbucket.org to that key. This is working as the private parent repo is pulling in just fine. The staging server has a few locked files in the environment which I have to work around, so I am initialising as follows:

git init
git remote add origin ssh://[email protected]/user/parent-repo.git 
git fetch
git checkout -t origin/master

So far so good but the submodules are empty:

# Update submodules 
git submodule update --init --recursive
# git submodule update --remote # Not available in 1.7.1
git submodule foreach git pull origin master 
## git fetch --recurse-submodules=yes # introduced 1.7.3 

Here is where things go wrong and ssh can't connect and the process halts on the first private repo. This isn't totally surprising as to access private repos, we need to associate another key, which cannot be the same as the one associated with the account. (scroll down to 'Private repos" heading for some interesting comments in this article by Luke Woodward)

So after making individual (I could have created just one) keys and creating ssh aliases for each of them (on the staging server), I'm stuck as to how to reference those key properly.

Just to be sure the aliases are working we can test them with:

ssh -Tv git@<alias>

Which when successful returns a lot of output ending with:

....
authenticated via a deploy key.
You can use git or hg to connect to Bitbucket. Shell access is disabled.
This deploy key has read access to the following repositories: XXXXX

Here is what I've tried so far. After each change, Im running:

git submodule update --init --recursive
git submodule foreach git pull origin master

EDIT: as suggested in comments re updating the links, at this point a git submodule sync will sync all submodules which have already completed a pull. None of the private repos will be listed in this operation. So sadly this operation does not seem to address issues with missing private repos.

1. change paths to ssh key aliases .gitmodules

Result: ssh: Could not resolve hostname bitbucket.org:username

It looks like an alias is resolving, but something is blocking the key?

2. change paths to ssh key aliases in .git/config.

Result: ssh: Could not resolve hostname <alias>:username

Obviously the alias isn't resolving at all.

---

As option 1 seemed to be at least making the ssh request, I manipulated the ssh alias that I thought was routing the request to the proper key. There must be some caching in effect, because changing the host on the alias to something impossible like bitbucketXXX.org. The result was the same error: ssh: Could not resolve hostname bitbucket.org:username. I would have thought the error would have read ssh: Could not resolve hostname *bitbucketXXX.org:username* --HUMMMM

---

Update: After a bit of thought

My approach so far has been to create an ssh key alias linking bitbucket.org to the parent key that I established for this server in my personal bitbucket settings.

When that didn't work for subs, I set up keys for each of them, and then on the server, aliases for each of them too, but keeping the parent repo key the same as before. Thus far i've been hunting for ways to reference these aliased-keys in .gitmodules and in .git/config. But this simply isn't working, the parent key is being recognised, but aliases for submodules keys are not.

My last approach has been to make one common key (different from my account key, they wont let you do that) and assign it to the parent repo, and each submodule. On the server, the alias for bitbucket.org points to this common key.

now when I test it with ssh -Tv [email protected] after a lot of output, we get:

authenticated via a deploy key.
You can use git or hg to connect to Bitbucket. Shell access is disabled.
This deploy key has read access to the following repositories:
*A list of all the repos associated with that key*

While this looks promising, the result is the same ssh: Could not resolve hostname . . .

I've even tried absorbing the user name into the alias like so

Host bitbucket.org:<user_name>
    HostName bitbucket.org          
    IdentityFile ~/.ssh/shared_bb_key

but this results in conq: invalid repository syntax.

---

At this point I can't see the forest, never mind trees. Any thoughts?

Answer 1

How to use public & private submodules within a repo

I needed to create a deployment strategy for wordpress that employed private submodules repos. There are some good source out in the interwebs on this already, but they don't directly address private submodules. So I'm documenting the steps I took to make this work with Bitbucket. Some of this may apply to other services as well, though I've only been tested it with Bitbucket. If you're using Github, you might be interested in this answer too.

Just to preface, I'm stuck using git 1.7.1 on my staging server, which as near as I can tell was released early 2011, so just a few years old now. Perhaps this complicated my situation, but if you see me use command x, instead of command Y, this might be why. I've tried to note what I understand to be more contemporary approaches, and will of course be making updates with any feedback. I've also tried to signpost hazards along the way, but I don't believe the issue or solution is unique to my particular host.

I'll assume that you're already familiar with, or have set up submodules in your project. If not these articles are a place to start: https://www.atlassian.com/git/tutorials/
http://blogs.atlassian.com/2013/03/git-submodules-workflows-tips/

For those already in the know

If you already have keys and aliases established, and are wondering why doesn't this just work, skip ahead. The answer lies in identifying git as a user in your ssh key aliases.

Ground work: Setting up SSH keys

First, you will need to be able to ssh into your server using a terminal, and once there you will need to establish a key (or keys) for the modules you will be pulling. Bitbucket allows you to share ssh keys between repos, so your use case may require more then one. For simplicity however, I'll illustrate a 'one key to rule them all' approach.

Note: This key is NOT the key established in your user account to allow you to push to your repos. Bitbucket won't allow you to use this key as deployment keys on repos for security reasons.

Also, because this process will be using the deployment key only, we will only be able to pull. Pushing back upstream is this possible for parent repo, if you own it, but requires a separate alias for that repo. My use case however is a staging server for testing a website, so pull only was good enough for me.

Atlassian provides full instructions for generating keys here: https://confluence.atlassian.com/x/YwV9E

In short, keys are normally kept in ~/.ssh/ and running ssh-keygen will prompt you through the steps to create the key –– just remember: don't specify a passphrase.

Note: I found that running the keygen from within my ~/.ssh directory caused permissions issues later. This might be unique to my server, but to be safe, just start the process from somewhere like your home directory.

Once the key has been created, you should be able to see it if you li -a ~/.ssh/. Assuming this is where you saved your key, cat ~/.ssh/your_bitbucket_submod.pub will return the public portion of the key, so copy this (being careful not to add any extra characters), and paste it into the deployment key section of each of your private repos.

Instructions here: https://confluence.atlassian.com/display/BITBUCKET/Use+deployment+keys

Creating SSH Alises

Once you have assigned the key to the repos on Bitbucket, you will need to create ssh alias(es) on your server. These alias will deliver the public key to bitbucket.org, whenever ssh is called to connect to those private repos.

As you may have surmised, we're going to be using ssh, and not https for all our private repo connections.

Normal git ssh paths should look like this: ssh://[email protected]:<USER-NAME>/repo-name, though Bitbucket will accept the path with or without the ssh://

SSH aliases are relatively straight forward, and are specified in ~/.ssh/config/.

If this config file exists, we'll be modifying it, if not we'll be creating it. Using your text editor of preference, add the following lines:

Host your_bitbucket_submod.pub
    HostName bitbucket.org
    IdentityFile ~/.ssh/your_bitbucket_submod
    User git

This is the meat of what we need, or at least one major piece of it.

Just a run down:

• Host is the alias keyword that ssh will looking for in paths sent to it. This can be anything, but I often name it the same as the key generated.

• HostName is the domain that the alias is pointing to, in this case bitbucket.org.

• IdentityFile points to the private portion of your key (note no .pub) and with the two pieces, ssh can validate the session.

• User specifies the ssh user http://unixhelp.ed.ac.uk/CGI/man-cgi?ssh+1 See ENVIRONMENT

User? Now this is a touch different then some of the other instruction you'll find about. It turns out that with ssh Git specifies itself as the user, and places the git account hosting the repo (ie your Bitbucket user name) after the semicolon. This one line cost me a few days of head scratching, until I re-read this article by Jochen Kupperschmidt and finally understood what he was getting at. This is the only article I've found that specifically references the User parameter in this context, so hat-tip to Jochen.

Testing your Aliases

It's worth checking to see if your alias is connecting, and if the key is assigned to all the repos you're expecting it to. In order to check, run the following:

ssh -Tv your_bitbucket_submod

With any luck you should see a lot of verbiage which ends something like this:

authenticated via a deploy key.
You can use git or hg to connect to Bitbucket. Shell access is disabled.
This deploy key has read access to the following repositories:
* A list of all the repos associated with this key . . . *

No joy? Here's some help: https://confluence.atlassian.com/display/BITBUCKET/Troubleshoot+SSH+Issues

Setting up the repo:

Now I'm not a whiz with git, so there may be differing opinions on how best to do this next part. My use case was to place a repo into a directory with pre-existing content which was not going to be tracked, but must remain. I suspect that this is one area that will see a bit of refinement as my betters offer feedback.

First we will cd into the directory where the repo will live. Once there, we will do the following:

git init
git remote add origin your_bitbucket_submod:<USER-NAME>/your-fav-repo.git
git fetch
git checkout -t origin/master

Note that we're using the alias here, as my parent repo is also private and I've the deployment key associated with it as well.

Here is where you might set up a second alias pointing to your primary account key instead. Though I've not tested this, this should enable pushing up stream to everything but submodules, even if they are under the same account as your profile's ssh master key.

So, if we got this far, our alias and key are working, and the parent repo is now established. You might be tempted at this point to pull in the submodules with something like the following:

# Update submodules without risk of recursion
git submodule update --init --recursive
## git submodule update --remote # Not available in 1.7.1
git submodule foreach git pull origin master 
# git fetch --recurse-submodules=yes # introduced 1.7.3 

If you did, you should see successful transfers of any public submodules you might have, but as soon as a private repo is encountered, the process grinds to a halt and you might see.

In case you have a submodules within your submodules, there is a risk of an infinite loop of cloning. This two step method of updating should prevent this.

ssh: Could not resolve hostname bitbucket.org:<USER-NAME>

Hu?!

Alternatively you could be prompted to input your password, in which case your submodules are https format, and will need to be converted to ssh paths.

Setting up .bitbucket/config:

When you first checked out your parent repo, if git found there were submodules, it will have made a number if hidden files and directories to track them (ls -a to reveal them). What we will need to do is edit (with vim, nano, etc) the .git/config file, and amend the paths listed here with our alias to the development key created earlier.

The config file will look something like:

[core]
        repositoryformatversion = 0  
        filemode = true  
        bare = false  
        logallrefupdates = true  
[remote "origin"]  
        url = your_bitbucket_submod:<USER-NAME>/parent-repo.git  
        fetch = +refs/heads/*:refs/remotes/origin/*  
[branch "master"]  
        remote = origin  
        merge = refs/heads/master  
[submodule "plugins/akismet"]  
        url = https://github.com/git-mirror/wordpress-akismet.git    
[submodule "plugins/your-famous-plugin"]
    url = ssh://[email protected]:<USER-NAME>/your-famous-plugin.git
    . . . 

Note that the url under [core] is already aliased, from when you specified remote add origin. Later on for the akismet plugin, we see its public repo on github connecting via https, and at last a private repo on Bitbucket. Here is where you will change all the references like:

ssh://[email protected]:<USER-NAME>/your-famous-plugin.git

to: your_bitbucket_submod:<USER-NAME>/your-famous-plugin.git

Once this is done we can re-update the submodules:

git submodule update --init --recursive
git submodule foreach git pull origin master 

And watch as they come rolling in . . .

Any failures at this point will be either some sort of mangled path from when you specified your submodules, or when adjusting the config file. Or perhaps the deployment key is not in fact assigned to the repo in bitbucket.

Hope this helps a few folks.

Further Reading:

• https://chrisjean.com/git-submodules-adding-using-removing-and-updating/
• https://medium.com/@porteneuve/mastering-git-submodules-34c65e940407