Pulling remote into subdirectory

Question

I have a Git repository hosted on bitbucket that has the following directory structure:

/Automation
/Website
/Website/   <-- [WebsiteFilesGoHere]
/Dependencies
/WebServices

We have a supplier that is building the /Website part of solution.

They have a repo that looks like this

/    <-- [WebsiteFilesGoHere]

What is the best way to pull their repo into our subfolder, and then be able to push back at a later date?

UPDATE: Its worth mentioning that both repos have files in the current `/Website' folder without knowing about each other.

UPDATE 2: This is also a Windows based Git repo, that is shared between a number of developers (so avoiding any local machine configuration would be great).

Answer 1

There is a way to bring in a remote repository without using submodules or subtrees, even if that repository is completely unrelated to yours and has clashing directories.

Firstly, add that repository as one of your remotes in the .git/config. For instance, let's say you want to bring in Zaldor's Gruff library from Github. These names are completely fictitious:

[remote "gruff-upstream"]
fetch = +refs/heads/*:refs/remotes/gruff/*
url = https://github.com/zaldor/gruff.git

Now you can do a git fetch gruff-upstream to get all the objects. Now you can see the available branches in the gruff projects.

$ git branch -r
gruff/experimental-hack
gruff/master
origin/master

The two gruff lines are gruff's branches. The origin/master is our own origin. The names clash: gruff has a master and we also have master. That doesn't matter: we can give gruff/master a different local branch name.

$ git branch -t gruff-master gruff/master
Branch gruff-master set up to track remote branch master from gruff-upstream.

Now, if we git checkout gruff-master, all of our git-tracked files will disappear, replaced by the gruff working copy:

$ git checkout gruff-master

Now we we can groom up this gruff-master branch a little bit. For instance, we can move all of its files to a subdirectory, remove unwanted files and such:

$ mkdir gruff
$ git mv ...files... gruff
$ git commit -a -m "moving gruff stuff to gruff/ subdir"

Next, we switch back to our own master. Gruff files disappear, and our files are back:

$ git checkout master

Now we can bring some files from gruff into our master:

$ git checkout gruff-master -- gruff/file1 gruff/file2 ...

These files materialize in our working copy and are added to our index:

$ git checkout
A gruff/file1
A gruff/file2
...

we can just commit these into our master now.

When the upstream gruff publishes new versions of the files, we can switch to our gruff-branch and pull. Resolve the new changes against our cleanup and commit. Back on our master, we can cherry-pick that new update from our gruff-master local tracking branch.

Yes, this is basically using Git as a glorified patch tool. The material from gruff-master picked into master do not have any ancestry; it just looks like added files. However, it is better than nothing and in many ways less cumbersome than subtrees or modules.

[May 2022 Appendix]

We can also use git merge to bring the groomed subdirectory into our master instead of checkout. The down side of that is that there will be a two-parent commit which has all the foreign repo's history behind the one parentage, and all of our history behind the other parentage. That history can be trimmed, though, if we limit the depth of the original fetch.

Answer 2

Suggest often best to use Subtrees instead of Submodules

A good run through of benefits here:

http://blogs.atlassian.com/2013/05/alternatives-to-git-submodule-git-subtree/

There are several reasons why you might find subtree better to use:

Management of a simple workflow is easy.

Older version of git are supported (even before v1.5.2).

The sub-project’s code is available right after the clone of the super project is done.

subtree does not require users of your repository to learn anything new, they can ignore the fact that you are using subtree to manage dependencies.

subtree does not add new metadata files like submodules doe (i.e. .gitmodule).

Contents of the module can be modified without having a separate repository copy of the dependency somewhere else.

In my opinion the drawbacks are acceptable:

You must learn about a new merge strategy (i.e. subtree).

Contributing code back upstream for the sub-projects is slightly more complicated.

The responsibility of not mixing super and sub-project code in commits lies with you.