How to integrate build output from one git repo into another

Question

I am working on a 2 projects which are stored in 2 git repos Project1 and Project2

Project1 is a javascript project. It has its own repo (for example https://github.com/gandra/project1.git)
Project2 is a java maven project. It has its own repo (for example https://github.com/gandra/project2.git)

Here is a current workflow which I want to improve:

1. Commit changes in a Project1 and push it to origin
2. Run grunt in a Project1. This generates Project1/build directory
3. Manually copy Project1/build contents into Project2/libs/project1-lib directory
4. Commit changes(Project2/libs/project1-lib) in Project2
5. Runk jenkins build and deploy

I want to somehow to avoid step 3 (manual copy of Project1/build contents into Project2/libs/project1-lib directory) I thought about integrate Project1 into Project2 as a subtree but the problem with this approach is beacuse it gets all Project1 directory structure into Project2/libs/project1-lib and I want only to take subdirectory of a Project1 (Project1/build/*)

Some important note: Project1 changes occurs only in its own repo(https://github.com/gandra/project1.git) and this change should be propagated to Project2. So there is no update of Project1 from Project2/libs/project1-lib In other words:
- Commits in Project1 affect Project2
- commits in Project2 not affects Project1

Answer 1

If you control the Git server you're using, then you can accomplish what you're asking for by using a post-receive server-side hook.

However, I can't help feeling that there should be a better approach than pushing changes from Project1 to Project2. It's obvious that Project1 is Project2's dependency. I'm more used to handling dependencies the way they do it with Maven and Gradle:

• Have the build tool (e.g. Gradle) for Project1 publish the build artifacts into a build artifact repository (e.g. Artifactory).
• Have the build tool for Project2 pull the latest artifacts from the build artifact repository as needed.

The advantages that I see in this approach are:

• Project1 is independent from Project2. Since it's Project2 that needs Project1 (and not vice-versa), it shouldn't be Project1's responsibility to keep Project2 updated.
• Project2 is in control of how it consumes Project1. You are free, for example, to freeze the dependency at a certain Project1 release (e.g. because Project1 is currently broken and you want to keep developing Project2 without waiting for the fix).
• You're not polluting Project2's commits with changes that are unrelated to Project2's source. If every time Project1 is updated you get a new commit in Project2, soon enough it'll make your life difficult when it comes to merging, rebasing, bisecting or any number of other things you normally have to do.

The disadvantages lie mainly in the infrastructure requirements (suddenly you need yet another server) and in choosing and configuring your build tools. When it comes to Java, I would recommend Gradle hands down for the build tool and Artifactory for the build artifact repository. But you said your Project1 is Javascript and I haven't worked in that ecosystem, so I don't have any good recommendations there.

Edit: I just now realized that I mixed up Project1 and Project2. The question says Project2 consumes Project1 and my initial answer was written as if Project1 consumed Project2. I fixed that now, to avoid confusion.

Answer 2

Ok let's recap the most important facts here:

1. Project2 is maven project(with war packaging probably, since you will deploy it ).
2. Project2 depends on Project1 in a way that it expects his output build on a path Project2/libs/project1-lib

In my opinion builds(even partial buils) are not meant to be under any source code management, so I wouldn't reserve any path of Project2 like Project2/libs/project1-lib to be under the git. Maven war plugin has this nice feature maven-war-plugin-overlays. So if Project2 would see Project1 as maven dependency with war type of packaginig e.g.

  <dependencies>
    ...
    <dependency>
      <groupId>your.group.id</groupId>
      <artifactId>project1</artifactId>
      <version>1.0-SNAPSHOT</version>
      <type>war</type>
      <scope>runtime</scope>
    </dependency>
    ...
  </dependencies>

, you wouldn't have to do any coping into Project2/libs/project1-lib location, because that will maven do for you in phase of packaing. But to do this, you have to install war artifact of your Project1 into your nexus. There are couple of solutions here:

Solution 1)

1. Make Project1 to be fully maven project, and make sure output of build finished on a path ./libs/project1-lib in his war. Since Project1 now would be build with maven, you would have to integrate previous building tool of Project1(grunt, gulp or whichever) to be called by maven in some phase before packaging phase.

2. In Project2 add new dependency like this:

   <dependency>
     <groupId>your.group.id</groupId>
     <artifactId>project1</artifactId>
     <version>1.0-SNAPSHOT</version>
     <type>war</type>
     <scope>runtime</scope>
   </dependency>
   

Solution 2).

1. With your current builder of Project1, make sure you package your project into war with following structure ./libs/project1-lib. Using same builder install this as war artifact on your nexus (NOTE: you would have to have pom.xml installed on your nexus so other project can reference Project1) If you are using grunt as a js builder luckily, there is this npm component grunt-nexus-deployer which will do exactly what I described here.
2. same as for Solution 1)...

Explanation:

When you push your changes of Project1 your CI will trigger job which will install project1-1.0-SNAPSHOT.war on your nexus after this is done, CI job for Project2 will be triggered and it will have everything needed for project2 build.

NOTE: In both solutions, delete Project2/libs/project1-lib hierarchy, since it not used anymore...

EDIT:

For solution 2) maybe you should use grunt-maven-tasks instead of grunt-nexus-deployer because it has more options and it is more suitable.

Answer 3

What Vojislav said is correct. The building of the two projects (in the way you described it) should be decoupled. If the final product needs both components, then it is the build tool's (jenkins) job to compose the application. The two projects should be pretty agnostic of each other.

I haven't used Jenkins in a little while but in Bamboo I would have one task called BuildProduct which would have two tasks (Build Project 1) and (Build Project 2) where the building of Project 2 would use the result of the first build (the built JS).