How to add an existing nested repo (already checked out in a subdir) to a parent Git repo as a submodule?

Question

What happens if I'm creating the initial commit from my working (parent) directory, but there are subdirs with independently checked-out git repos?

I simply did git add . but that brought me to a strange situation when the subdirs with nested Git repos are not registered as submodules of the parent repo.

So: how to proceed after an initial "git add ." in a parent working dir where there have been subdirs with independetly checked-out nested git repos (in order to get correct submodules)?

An example:

[imz@z super-existing-sub]$ ls 
c  sub
[imz@z super-existing-sub]$ ls -a sub/
.  ..  a  .git
[imz@z super-existing-sub]$ 

So, there is already a pre-existing super-existing-sub/sub git repo inside super-existing-sub.

After I run in super-existing-sub:

$ git init
$ git add .

what can be done to correctly register the pre-existing Git repo as a submodule?

Now, Git somehow has tracked it:

$ git status 
On branch master

Initial commit

Changes to be committed:
  (use "git rm --cached <file>..." to unstage)

    new file:   c
    new file:   sub

$ 

but git submodule has some problems:

$ git submodule status 
No submodule mapping found in .gitmodules for path 'sub'
$ 

How to convert it to a correct submodule?

---

I tried to proceed the way that was suggested in the answer (composed by Victor and me), namely git submodule add URL subdir, but that breaks unfortunately:

$ git submodule status
No submodule mapping found in .gitmodules for path 'wp-content/themes/liquorice'
$ git submodule add [email protected]:/nudgeme/Liquorice.git ./wp-content/themes/liquorice
'wp-content/themes/liquorice' already exists in the index
/sshx:kosmoplus:/home/kosmoplus/kosmoplus.ru.old $ git submodule status
No submodule mapping found in .gitmodules for path 'wp-content/themes/liquorice'
$ 

Answer 1

Building on Victor Johnson's answer, here's a bash oneliner for adding a whole whack of submodules at once.

find . -name config | grep git/config | grep -v ProjectToExclude | grep -v AnotherProjectToExclude | xargs -n 1 -I % sh -c 'cat % | grep -m 1 url | sed "s/url = //g"; echo % | sed "s/.git\/config//g";' | xargs -n 2 echo git submodule add

Detailed explanation

Suppose you have a folder containing many git repos, and you want a parent repo to have them all as submodules:

path/to/one_repo
path/to/one_repo/.git/config
path/to/another_repo
path/to/another_repo/.git/config
(etc.)

And you want to run git submodule add ./path/to/submodule/ for each of these.

1. This oneliner prints that command for each possible submodule. Run it as a dry-run to see what folders have .git/config files that contain url =. Note that it assumes the first occurrence of url is the remote one - make sure you sanity check the results.

The oneliner again, explained:

find . -name config |                            # find all config files
    grep git/config |                            # include only those that have "git/config" in their path
    grep -v ProjectToExclude |                   # exclude config files with ProjectToExclude in their path
    grep -v AnotherProjectToExclude |            # ditto
    xargs -n 1 -I % sh -c '                      # one line at a time, execute the following commands, substituting the path for `%`
        cat % |                                  # print the config file contents
            grep -m 1 url |                      # take the first line that contains `url`
            sed "s/url = //g";                   # remove the "url = " part
        echo % |                                 # print the path to the config file
            sed "s/.git\/config//g";             # remove '.git/config' to keep only the project path
    ' |                                          # summary: print the remote url on every 2nd line; the path on every 2nd line 
    xargs -n 2 echo git submodule add            # take two lines at a time, print them with the command

Note that you can add any number of grep -v ProjectToExclude to the pipe to exclude those folders - run the oneliner again to see the new list of commands to execute, and notice that ProjectToExclude is not there anymore.

This will print out:

git submodule add https://github.com/username/one_repo.git path/to/one_repo
git submodule add [email protected]:another_repo.git path/to/another_repo
(etc.)

2. Then, execute it for real: Remove the echo at the end of the command, so the last part changes from

xargs -n 2 echo git submodule add

to

xargs -n 2 git submodule add

(Instead of echoing the command, we're actually going to execute it.)

3. add it to git. You can now run git status to see what you've just done.

Answer 2

(I've always used git submodule init to get git to recognize them and then git submodule update to actually clone the submodules into the working directory. But that's not the situation that is being asked about.)

To create a new git repo from a parent directory, you need to run git init on the parent directory, then git submodule add ./path/to/submodule/.

Note: the path to the submodule must be an absolute path, so you should prefix the path with ./

Or, if you want to have a good external URL as the URL for the submodule, you should first lookup the origin URL in ./path/to/submodule/.git/config, and then you should be able to

git submodule add URL ./path/to/submodule/

Haven't tried this out yet in practice, but the manpage for git-submodule says:

If <path> does exist and is already a valid Git repository, then this is added to the changeset without cloning. This second form is provided to ease creating a new submodule from scratch, and presumes the user will later push the submodule to the given URL.

Answer 3

See TL;DR at the end.

Let's read the man-page for git submodule <URL> <path> carefully, and notice the following remark:

If <path> does exist and is already a valid Git repository, then this is added to the changeset without cloning. This second form is provided to ease creating a new submodule from scratch, and presumes the user will later push the submodule to the given URL.

Let's try to use this in a situation like ours (after you added the parent directory recursively to the new parent repo whereas there was already an existing nested Git repo).

The idea is, first, to lookup the origin URL in ./path/to/submodule/.git/config in order to get a nice real external URL for the meta-information about the submodule, and second, invoke git submodule <URL> <path> (which hopefully does what we want, according to the man-page).

Let's try:

$ git submodule status
No submodule mapping found in .gitmodules for path 'wp-content/themes/liquorice'
$ git submodule add [email protected]:/nudgeme/Liquorice.git ./wp-content/themes/liquorice
'wp-content/themes/liquorice' already exists in the index
/sshx:kosmoplus:/home/kosmoplus/kosmoplus.ru.old $ git submodule status
No submodule mapping found in .gitmodules for path 'wp-content/themes/liquorice'
$ 

Unfortunately, it breaks.

Well, let's think how we can solve the issue that something "already exists in the index" and is not the the thing we want...

...we can simply remove it from the index! This works successfully:

$ git submodule status
No submodule mapping found in .gitmodules for path 'wp-content/themes/liquorice'
$ git rm --cached ./wp-content/themes/liquorice
rm 'wp-content/themes/liquorice'
$ git submodule add [email protected]:/nudgeme/Liquorice.git ./wp-content/themes/liquorice
Adding existing repo at 'wp-content/themes/liquorice' to the index
$ git submodule status
 9733c0ab3e4207352e3f51d612f2a1c9c4a0b63a wp-content/themes/liquorice (liquorice2.1-1-g9733c0a)
$ 

TL;DR

• lookup the origin URL in subdir;
• git rm --cached subdir
• git submodule add URL subdir

This could even be made a simple script to automate the task of adding an existing nested repo into the parent as a submodule, but I don't see the point in multiplying things. At this level, simply learn the basic Git commands, and use and combine them!