Keeping all Visual Studio projects in sync with a library

Question

Scenario

I have a Library that contains Projects A, B, and C.

I have two solutions. Solution 1 includes a copy of Project A, and Solution 2 includes a copy of Projects A and B.

---

When I build Solution 1, here's what should happen:

---

When I build Solution 2, here's what should happen:

---

How can I do this?

Is this something I could automate with a version control system or off-the-shelf file syncing software? Or do I need to roll my own solution?

If I do build my own solution, I have some thoughts on how it could work, but I'd appreciate any input you may have:

• Could be a simple console app with a command-line switch for specifying the "source solution", for example:

  c:\Program Files\Library Syncronizer\LibSync.exe /solution:Solution 1
  

• XML file used to register active solutions that contain library projects. Possible format:

  <solutions>
    <solution>
      <name>Solution1</name>
      <path>c:\...\Projects\Solution 1</path>
    </solution>
    <solution>
      <name>Solution2</name>
      <path>c:\...\Projects\Solution 2</path>
    </solution>
    <!-- more solutions -->
  </solutions>
  

• Program would do the following:

  • Read in source solution
  • Determine what library projects it has
  • Copy the folders for those projects to the library
  • Loop through each solution in the XML file (except the source)
  • Copy the folders for all library projects as necessary
  • Perhaps some backup operation could also occur, just in case the syncing process overwrites something important.

This sounds relatively simple in concept, but this may have serious unintended consequences I'm not thinking of. Hopefully someone will warn me if it does :)

---

Update - What is my motivation for copying project folders?

In a word - Version Control.

If I keep the library projects in a separate folder and only link to them in my various solutions (rather than physically locate the folders in my solution folders), my version control repository ends up not containing the source code to my library projects. So, if I update to "three version ago", and I need to make a minor change to one of my library methods, the code is not there.

My workaround for this has been to add tags to the revisions in my library's repository that say things like "Solution 1 - Version 2.5.3", but this is pretty clunky. And things get really awkward if I'm working on "three version ago of" of Solution 1 and the current version of Solution 2. Now, Solution 2 will be pointing to an old version of the library projects, which makes it potentially impossible to work with and test until I'm done working on the old version of Solution 1.

If I were working with copies instead, all solutions would contain the library source code in their repositories, and I could go back to it easily any time I need to.

I should note here that I've been using Tortoise HG (Mercurial) for version control.

Anyway, I'm open to any solution to this problem. It doesn't have to involve copying project folders around--that's just the only thing I could think of to ensure that all my version control repositories are complete, stand-alone packages.

---

Update 2

First of all, just a note. I'm using Mercurial (TortoiseHG) for version control, not SVN. I could change if absolutely necessary, but I really prefer Mercurial.

Based on responses so far, I've decided to do away with the "bi-directional copying" idea, and go back to referencing my library projects. Here's a new diagram:

I continue to have the same goals, however:

1. Latest version of each solution is using latest library code
2. One solution/application per repository
3. Each repository contains all source code, including library projects
4. Everything is as automated as possible to minimize the risk of mistakes

Goal #1 is taken care of automatically by referencing the library projects instead of using copies, and Goal #2 is just a matter of how my set up my repositories, but Goals #3 and #4 remain elusive.

With Mercurial, there is a subrepositories feature that seems like it would handle my situation, but as the documentation indicates, this is still considered experimental/risky.

For the moment, I'm thinking a good workaround might be to just store backup copies of the library projects in my Solution folders. When I say "backup copies", I mean that literally. I would still be referencing the library projects--the copies would be solely for the purpose of ensuring all source code ends up in my repository (Goal #3). To satisfy Goal #4, these backups could be automated using a post-build event in studio.

I welcome your thoughts on any of this.

Answer 1

Consider why your Applications need to work with different versions of the libraries. If you design your libraries properly, and make sure you don't break anything while upgrading them (use continuous integration and unit tests, as well as testing all dependent projects) then the best (simplest, cleanest) approach in most cases is simply to have all Applications run off the same version of the Libraries. I would go so far as to say that if your Applications can't run off the same Library version, that tells you that your Libraries are not designed/implemented/maintained properly, and you should reconsider your approach.

Another approach is to use a branch for each Application, so each has its own independent copy of the Libraries (use a relative reference between the projects and the libraries and then you can relocate the code in the branches). This allows the Applications to build off different versions of the libraries, but does have all the disadvantages of branching, though - two copies of the libraries so you have to be careful not to edit in the wrong branch; the potential for nasty merges, etc.

I'd recommend moving the Library projects into the Library. Your diagram shows the Application solutions building library code and pushing it up to the library, then sucking the built library files back down into the projects - a bi-directional copy, eek! Just place the library projects in the library and build the library, then reference that library from your projects - a unidirecitonal copy. You may think this would mean you have to build two solutions (Library and then Application), but you can actually add/remove library projects in your application Solution whenever you wish, so this layout does not necessitate a two-stage build process. But the logical distinction between the library and application is much cleaner.

Answer 2

The way we solve a similar problem is just what you describe in update 2, but where you suggest 'backup copies', we use clones and they are an integral part of our workflow.

My solution.

We use a combination of clones and named branches to allow us to control when the libraries for a particular application (solution in your example) are updated.

What we do is have a core repository for each of our libraries which contains the mainline branch of that library. When we create a new application we clone all of the relevant libraries (we use one mercurial repo per library, where a library may contain the library and it's test application) into directories beside the new application and reference the libraries in that application.

Any commits to the libraries for this application are committed to a branch named according to the application. When an application is complete, we review the changes to the library and then merge appropriate changes back into the mainline, creating a new mainline. This also has the advantage that in the future, you can easily identify which application caused a particular change to a library by looking at which branch that change was committed to.

Whenever we touch an application, we can chose whether to continue using the old libraries, or update to a more recent mainline version (or even another projects development version) of the library. Library changes aren't pushed out to every application which uses them (some might be legacy applications which haven't been touched for years and are working fine with the library they were suplied with), they are pulled in by applications which need the new facilities in a given version of the library.

Why do I do it this way?

The problem I'm trying to avoid here is one I constantly ran into at a previous company.

I would get my application working with a given library. Someone else would change the library for their application so that it did what they needed and the next time I went to edit my project, it wouldn't compile. I would then have to spend time fixing up my project to beghave in the way the library now wanted or spend time to revert inappropriate changes made to the library (which would then cascade back to the other developer).

With local project clones, every application gets to chose when it updates to a more recent library version and from a chance management perspective, we get to review changes to libraries due to application requirements before they get merged back into the mainline.

This is a workflow which would never have been possible with a traditional VCS like SourceSafe, but works beautifully with a DVCS like Hg.

Specific solutions to those elusive goals.

To try and answer your elusive goals

.3. Each repository contains all source code, including library projects

By cloning libraries into the directories close to your application, you get a set of repositories which can be managed together and possibly eventually will convert nicely into sub-repos of an 'application' super-repo.

.4. Everything is as automated as possible to minimize the risk of mistakes

I do this with a bunch of batch files which I store in my application directory.

For instance a given applications "_projects.bat" might contain:

@echo off
@shift
set APPDIR=%CD%
cd ..\Library1
%0 %1 %2 %3 %4 %5 %6 %7 %8 %9

cd ..\Library2
%0 %1 %2 %3 %4 %5 %6 %7 %8 %9

cd ..\Library3
%0 %1 %2 %3 %4 %5 %6 %7 %8 %9

cd %APPDIR%
%0 %1 %2 %3 %4 %5 %6 %7 %8 %9

pause

Then I have batch files to do useful synchronisation things like check what files would be pulled with "__incoming.bat:

@echo off
call _projects hg incoming

and check what would be pushed with "__outgoing.bat":

@echo off
call _projects hg outgoing

Or actually do those operations with "__push.bat":

@echo off
call _projects hg push

and "__pull.bat":

@echo off
call _projects hg pull

while "__update.bat" updates everything to the latest version (within the same branch, you still have to explicitly update outside the current branch):

@echo off
call _projects hg update

One of my favourites is "__ids.bat", which shows what changesets (and branched) are currently in use and critically, whether they have uncomitted changes outstanding:

@echo off
call _projects hg id

Whenever I come to start work on an application update, I check to see if there are updated libraries, pull down everything which is appropriate, update to the versions I want and then continue development.

Note the underscores in the filenames are just there to push them to the top of the directory lists, to make them easier to find. *8')

Future options.

Eventually I would like to fully automate this workflow by using sub-repos (so the state of the whole application and all of it's libraries are recorded in one application changeset), but while it is getting pretty stable in Mercurial, TortoiseHG integration doesn't seem to be a priority. I couldn't get the THG commit tool to do anything useful the last time I tried it (around 0.8.3). I really should retest with 0.9.3 but I can't remember seeing any notes about sub-repos in the changelogs, so it isn't worth it unless people tell me THG sub-repo support has been silently added.

As it is, the batch files work quite well. The next thing on my todo list is a "__clone.bat" which should make cloning the application and all of it's libraries a whole lot easier, but I haven't had the time to get that working.

If anyone fancies having a go though, I'd love to see a reply in this thread. *8')

Take care, hope this helps,

Mark..........