Java project becoming too large

Question

We have a large Java 8 Spring Hibernate Maven project that is getting larger.

Problems :

• Build time is 10-12 minutes at best; 3 minutes without tests
• We already have a command-line switch to skip rarely modified modules, which is the symptom of the build process reaching practical limits
• Eclipse is struggling to manage the project (although IntelliJ is ok for now)
• Things are getting worse as the project grows, and as more scenarios from the test team get incorported as integration tests in the code base.

How we work now

• The project is configured in about 20 Maven modules, like so:

    Parent
    |--- Tier1
    |--- Tier2
    |--- WebTier
         |---- ModuleA
         |---- ModuleB
         |---- ModuleC
         |---- ...
         |---- Entities
         |---- Shared
         |---- Batch
         |---- IntegrationTests

• The application is built as a single WAR
• Developers deploy a single tier (typically WebTier) as an artefact from Eclipse or IntelliJ to their local Tomcat
• Although the project seems nicely split in modules, there are many undesired coupling points between them. Specially in Shared, where modules needing "cross-modules" access put their services
• All integration tests are in a dedicated module (no idea why)

Ideas to make it better

• Add a MessageBroker module to allow loose coupling where relevant. Maybe JMS, or simply a dumb in-memory component for synchronous communication
• Get rid of the Shared module
• Make sure modules have coarse-grained entry-points
• Remove undesired coupling between siblings and prefer the message broker when possible
• Might keep Entities. At least the core-business entities (Customer, CustomerFile, ...). But some entities obviously belong to a single module (a batch execution info would be in the Batch module)

That way, anyone making a change to ModuleA would most of the time only build and run tests in that module without fearing to break the application.

Questions

• Does that seem like a good plan? By good, I mean future-proof, with good chances to improve things, and not requiring an excessive amount of work given the situation
• Should we have 1 Eclipse/IJ project per tier, let the IDE build the artefact and deploy it to Tomcat, or should we have 1 project per module, and dependencies toward Nexus? Or maybe the latter option is overkill?
• Any other suggestions?

Some metrics

• Windows 7, Java 8, Maven 3.0.3, TestNG.
• SSD or 7200rpm HDD (limited impact)
• 6Gb RAM
• Heap 1Gb (maven)
• CI with Jenkins

Thanks a bunch!

Answer 1

CI would be real answer but it looks that your project is not modular as it should be. You don't build entire project from scratch every time. You build jars, test them in different projects and then use as single items. Each project should be small enough and cover just one area. Do you think that Java builds let's say security jars when they work on io package? Divide and conquer - that's the whole idea of OOP and encapsulation.