Proper Git workflow for combined OS and Private code?

Question

I have a closed source project that is built on my open source framework. I want to know how I should structure my workflow. Below is my best guess using git with submodules.

1. I create a public framework repo on github with submodules that are separate git repos.
2. I purchase a "micro" account on github ($7) so I can have a private repo.
3. I create a private repo and clone the public framework repo.

From here I can make changes to:

1. My private code and push to my private repo on github
2. The public framework code and push to my private github repo and then send a pull request from the public framework..? Or how would this work?

How do I handle a repo that contains private and public code and submodules. Right now it seems like I just have to maintain two separate codebases to achieve this.

I'm looking for the best answer that can help someone fairly new to git streamline the process of working on a codebase that is half open source and half private. One good thing about it is that each folder is either private or public so there is no worry about having private and public files together somewhere - yet some of the private folders might be in public ones!

Another example I could give would be using zendframework to build your private company site while still being able to do pulls each day (and maybe patch pushes) to the zend repo. And also pulls and pushes of your private site inside the zendframework.

For example, imagine a directory structure like this:

/private_folder
/public
        /public_folder
        /public_folder2
        /private_folder

Perhaps I'm asking two much to handle them all in one joined repo directory. Maybe there is no easy way to do this and I should separate them and do all the public patches in one and then just pull into my private repo. Of course, this means that if I am in the middle of working on some private code - I'll have to leave that repo and go open up the public one and make the patched code change, then go back to the private one, merge, and then continue working on the private code.

Answer 1

I recommend not to use git submodules, but 2 different repositories that are not connected on github.

You could build the relationship between them using symlinks on the checked out copies, which is basic and simple. The symlinks only have to be created once per location (production, development, coworkers).

The advantage is that nobody has to do the extra effort to learn and maintain git submodules, and you avoid the risk and complexity it brings.

It could be done by keeping a working copy of the os and of the private git repo somewhere on your local machine:

/repos/myproject-os
/repos/myproject-priv

Then you could create create your directory structure where the project actually will live and be worked on somewhere else on this machine (not inside the /repos/ tree) and create symblinks for the subdirectories you use:

ln -s /repos/myproject-os/dir1 /wrk/myproject/base/dir1
ln -s /repos/myproject-os/dir2 /wrk/myproject/base/dir2
ln -s /repos/myproject-priv/dir1 /wrk/myproject/base/dir3
ln -s /repos/myproject-priv/dir2 /wrk/myproject/base/someother/dir4
mkdir /wrk/myproject/base/config
mkdir /wrk/myproject/base/tmp

That way you have the repository structure always clean and can mix and arrange the directories from both repositories the way you want them, and you have also a space for local configs or temp files that do not go into the repositories.

You would do the git commits and everything from the /repos/ tree and your project would run and you would edit the files from the /wrk/ tree. Please note that the .git diretory where the git data lives would not be available form the /wrk/ tree, because you only link to subdirectories (or possibly single files from the root directory).

Part2: You say you want to make sure that you do not accidently push private code into the public repository. You could set up an additional git repository between your working OS repository and the github repository, let's say you put it into /repos/gatekeeper, then your tree looks like this:

/repos/gatekeeper/myproject-os
/repos/myproject-os
/repos/myproject-priv

Every time you push from /repos/myproject-os it goes to /repos/gatekeeper/myproject-os. But from /repos/myproject-priv you push directly to your private github repo.

That way you have the same workflow in both /repos/myproject-os and /repos/myproject-priv and you don't need to worry so much. From time to time when you want to push your changes to the real OS codebase, you go to /repos/gatekeeper/myproject-os and push from there to github.

You could do additional code review before that and look at the diffs so you are sure that only that what you really want goes public.

If you want additional security the /repos/gatekeeper/myproject-os could also be on a different machine or even different location.

Answer 2

You can have a 'public' and 'private' branch in your local repository. When you push, each branch gets pushed to a separate remote repository (look up the 'git push' syntax). Then, you can freely merge from public to private.

I'm sure there's a way you could merge selected changes from private to public, too, though I'd have to look it up.