Android source code and repo - What exactly is happening when getting code

Question

I have come across plenty of short questions on minimal aspects of using repo to get Android source or very broad definitions of what repo does and as a result don't really understand what is happening when I use repo. I am following the instructions on the Android Source site. I do this all while in my repodir so all mention of files are relative to this.

Init and sync repo:

repo init -u https://android.googlesource.com/platform/manifest repo sync 

After this completes, my folder is full of android type folders (bionic, bootable, etc etc) and a hidden .repo folder. Notably, there is a folder called ics-mr1 which relates to a specific version and it contains mostly the same folders in my repodir inside itself.

Pull down a specific branch:

repo init -u https://android.googlesource.com/platform/manifest -b gingerbread repo sync 

This completes relatively quickly and the main change I see is there is now a folder called gingerbread (all the original folders seem to remain).

I now try this and it takes AGES:

repo init -u https://android.googlesource.com/platform/manifest -b android_4.2.2_r1 repo sync 

After, it still contains the same gingerbread and ics-mr1 and a few new random folders (like abi) but nothing that seems to be of that new version.

The question

From the results of this, I am seeing I really don't understand what exactly my actions are doing and how to achieve what I am trying to.

How can I obtain a local version of the entire master branch of the source, and then correctly change between branches as I see fit? The way I am at the moment seems wrong as the files left after branch changes seem illogical.

To further that, is there an efficient way to have local copies of numerous branches simultaneously such that I can compare the components of each (obviously without re-downloading them each time)?

I am asking these questions with big holes in understanding (ones which I have been unable to find clear consistent information for), so any extra information about the answer to assist in understanding of what is actually being done would really be of value.

Answer 1

repo init

Since you seem pretty savvy, you know repo is just a python script, right? You can read it to see exactly how it works. I haven't read through the entire thing, but, basically, it wraps git and provides support for working across multiple git repositories. The idea is there is a manifest file that specifies requirements for different versions of Android. When you do repo init with an argument it looks at which git repositories it needs to clone and which git repositories you have already synced, and which branches it needs to fetch once it has obtained the proper repositories. It then handles keeping all the repositories it is managing on the proper branches. Think of repo as adding another layer to the standard git workflow (which I'll assume you're familiar with).

The first time you use repo, to get the master branch, you need to use

repo init -u https://android.googlesource.com/platform/manifest 

And afterwards you need to sync all the files from the server:

repo sync 

This will update your current working directory to the exact state specified by the master version of the manifest you downloaded. To checkout a different version of the AOSP, you use:

repo init -b version_name 

This updates the manifest to the one that contains information about the Android version you want (also called a branch).

Don't forget to sync.

To switch to another manifest branch, repo init -b otherbranch may be used in an existing client. However, as this only updates the manifest, a subsequent repo sync (or repo sync -d) is necessary to update the working directory files.

The behavior you are experiencing may seem weird because you're probably not used to working with a system that overwrites your local state so easily. When using git, you're not supposed to init a bunch of times in the same directory. Another option is to make new directories for each project. The purpose of the .repo directory is to store all the information relevant to your current repo setup (also like the .git directory). In fact, running repo init does many things:

1. Downloads the manifest from the specified url.
2. Tries to open or create the .repo directory.
3. Makes sure your GPG key is set up.
4. Clones the specified branch (which defaults to REPO_REV if none is specified).
5. Verifies it.
6. Checks out the appropriate branch for each of the projects.

I would assume the clone operation writes over information in the .repo folder. The reason it takes ages after the first command you run:

repo init -u https://android.googlesource.com/platform/manifest repo sync 

Is because it has to download many gigabytes of information. Now, going from the master branch to gingerbread repo knows it simply needs to drop about 68 commits which can be done very quickly.

$ repo init -u https://android.googlesource.com/platform/manifest -b gingerbread $ repo sync ... .repo/manifests/: discarding 68 commits ... 

Ramping back up to android_4.2.2_r1 means repo must download any information needed by those commits again, and also update the current branches on all the referenced projects. This will take a long time; repo is trading disk usage for processing time.

really?

Now, this presents a problem: what if you want to compare two repo branches at once? This is difficult because when you repo init && repo sync you lose the old information you were looking at. The answer is to copy the relevant information and then repo init && repo sync again. This will get annoying really fast -- thankfully repo provides a way to speed up this process if you have the disk space.

One strategy to make things quicker is to create a local mirror in a workspace/master directory. Then, checkout your desired branch from the mirror in a new directory, e.g. workspace/gingerbread. Now you can switch between branches simply by changing to the appropriate directory.

Mirror the AOSP locally:

cd workspace mkdir master && cd master repo init --mirror 

will cause repo to mirror the remote server on your local machine. Then, when you want to switch to a new branch, you can:

mkdir ../gingerbread && cd ../gingerbread repo init -b version_name --reference=../master 

The result is a workspace with a folder containing the mirror and a folder containing the gingerbread branch referencing the mirror where possible.

The other option is to simply initialize and sync the branch you want, then copy the folder to another location and use the copy to initialize and sync again. Repo should only download what's missing.

Other repo uses:

Beyond init, repo provides support for passing commands, such as branch, to all the different git repositories in a given manifest so that you don't have to worry about that when working with the code. It also facilitates collaboration by making it easy to submit your local changes to the gerrit code review system. I'm unsure what the official reason for partitioning the AOSP into multiple git repositories is, but I would imagine it was done in order to manage the scaling issues of version control and in order to keep the project robust and fault-tolerant (if someone brakes one git repo it doesn't destroy the entire AOSP). It is also necessary to provide a way that vendors can contribute back to the source and letting them manage their own git repositories which can simply be registered with/into an overarching manifest makes sense. I didn't answer your questions line-item, but hopefully I provided some background.