Building Eclipse IDE from scratch - how to specify the target OS?

Question

This post is about building the whole Eclipse IDE from the source code - and more in particular how to specify the target OS you want to build the executable for.

I found the following tutorial to guide me through the build procedure: http://www.vogella.com/tutorials/EclipsePlatformDevelopment/article.html

Especially chapter 13 of that Tutorial is helpful. I will describe the steps that need to be taken one by one here - so you don't need to read the vogella tutorial.

NEEDED SOFTWARE :

Apparently you need to have the following software installed on your (Windows) machine:

• Git
• Maven Version 3.3.1 (make sure that the bin folder where you extract Maven is added to the PATH. For installation instructions see https://maven.apache.org/install.html)
• Oracle 1.8 JDK or higher (don't forget to set the JAVA_HOME environment variable)

Now you're ready to start:

STEP 1 :

Create a new folder in the C:\ disk. Let's name it C:\myEclipse. This will be the folder in which we pull the Git repository of the latest Eclipse version.

STEP 2 :

Open the Windows cmd shell with administrator privileges (right click on cmd icon if you made one on your desktop, and select open with admin rights). Go to that folder through the command cd C:\myEclipse. Now type the following command: git config --system core.longpaths true. This notifies Git that long names can occur in the repository. Forget this command and it will result in errors later on.

STEP 3 :

Clone the Git repository:

git clone -b master --recursive git://git.eclipse.org/gitroot/platform/eclipse.platform.releng.aggregator.git

Now you should have some patience for all the sourcecode to get copied to your local folder.

STEP 4 :

Finally it is time to start the build. The build procedure will already take hours, so it is wise to skip the unit tests. Therefore do not forget to add the -DskipTests parameter. This is the command to start the build:

mvn clean verify -DskipTests

After some hours, the build should be finished.

STEP 5 :

The results of the build should now be in the following folder:

C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.releng.tychoeclipsebuilder\sdk\target\products

Notice that the Eclipse IDE is built for all possible target machines. 32-bit Windows, 64-bit Windows, Macintosh, Linux, ... No wonder it takes hours! I would like to build for just one single target - for example my 64-bit Windows machine. This will hopefully reduce the build time significantly and allow me to verify code changes quicker.

Does anyone know how to achieve this?

---

EDIT :

I have added the argument -Dnative=win32.win32.x86_64 to the build command. Unfortunately, I get an error after 15 to 20 mins. This time, I've added the argument -X as well, to get debug information:

                      ...

[INFO] equinox-sdk ........................................ SKIPPED
[INFO] org.eclipse.rcp.id ................................. SKIPPED
[INFO] org.eclipse.rcp.sdk.id ............................. SKIPPED
[INFO] org.eclipse.platform.ide ........................... SKIPPED
[INFO] org.eclipse.platform.sdk ........................... SKIPPED
[INFO] org.eclipse.sdk.ide ................................ SKIPPED
[INFO] eclipse-junit-tests ................................ SKIPPED
[INFO] eclipse.platform.repository ........................ SKIPPED
[INFO] platform-aggregator ................................ SKIPPED
[INFO] -----------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] -----------------------------------------------------------------------
[INFO] Total time: 23:54 min
[INFO] Finished at: 2016-06-19T18:12:14+02:00
[INFO] Final Memory: 668M/3263M
[INFO] -----------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.8: run (natives) on project org.eclipse.swt.win32.win32.x86_64: An Ant BuildException has occured: The following error occurred while executing this line:
[ERROR] C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.swt\bundles\org.eclipse.swt\buildSWT.xml:915: The following error occurred while executing this line:
[ERROR] C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.swt\bundles\org.eclipse.swt\buildSWT.xml:1012: exec returned: 1
[ERROR] around Ant part ...<ant antfile="build.xml" target="build_libraries"/>... @ 4:54 in C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.swt.binaries\bundles\org.eclipse.swt.win32.win32.x86_64\target\antrun\build-main.xml
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.8:
    run (natives) on project org.eclipse.swt.win32.win32.x86_64:
        An Ant BuildException has occured:
            The following error occurred while executing this line:
            C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.swt\bundles\org.eclipse.swt\buildSWT.xml:915: The following error occurred while executing this line:
            C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.swt\bundles\org.eclipse.swt\buildSWT.xml:1012: exec returned: 1 around Ant part ...<ant antfile="build.xml" target="build_libraries"/>... @ 4:54 in C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.swt.binaries\bundles\org.eclipse.swt.win32.win32.x86_64\target\antrun\build-main.xml
                at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
                at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) 
                at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) 
                at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
                at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
                at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
                at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
                at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
                at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
                at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
                at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
                at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
                at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.lang.reflect.Method.invoke(Method.java:498)
                at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
                at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
                at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
                at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
    Caused by:
        org.apache.maven.plugin.MojoExecutionException:
            An Ant BuildException has occured:

                The following error occurred while executing this line:
                C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.swt\bundles\org.eclipse.swt\buildSWT.xml:915:

                The following error occurred while executing this line:
                C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.swt\bundles\org.eclipse.swt\buildSWT.xml:1012: exec returned: 1 around Ant part ...<ant antfile="build.xml" target="build_libraries"/>... @ 4:54 in C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.swt.binaries\bundles\org.eclipse.swt.win32.win32.x86_64\target\antrun\build-main.xml
                    at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:342)
                    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(Default BuildPluginManager.java:134)
                    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
                    ... 20 more
                Caused by: C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.swt.binaries\bundles\org.eclipse.swt.win32.win32.x86_64\target\antrun\build-main.xml

                :4: The following error occurred while executing this line:
                C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.swt\bundles\org.eclipse.swt\buildSWT.xml

                :915:The following error occurred while executing this line:
                C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.swt\bundles\org.eclipse.swt\buildSWT.xml:1012: exec returned: 1
                    at org.apache.tools.ant.ProjectHelper.addLocationToBuildException(ProjectHelper.java:568)
                    at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:443)
                    at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
                    at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source)
                    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                    at java.lang.reflect.Method.invoke(Method.java:498)
                    at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
                    at org.apache.tools.ant.Task.perform(Task.java:348)
                    at org.apache.tools.ant.Target.execute(Target.java:435)
                    at org.apache.tools.ant.Target.performTasks(Target.java:456)
                    at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
                    at org.apache.tools.ant.Project.executeTarget(Project.java:1364)
                    at org.apache.maven.plugin.antrun.AntRunMojo.execute(AntRunMojo.java:313)
                    ... 22 more
                Caused by: C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.swt\bundles\org.eclipse.swt\buildSWT.xml

                :915: The following error occurred while executing this line:
                C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.swt\bundles\org.eclipse.swt\buildSWT.xml:1012: exec returned: 1
                    at org.apache.tools.ant.ProjectHelper.addLocationToBuildException(ProjectHelper.java:568)
                    at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:443)
                    at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
                    at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
                    at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source)
                    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                    at java.lang.reflect.Method.invoke(Method.java:498)
                    at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
                    at org.apache.tools.ant.Task.perform(Task.java:348)
                    at org.apache.tools.ant.Target.execute(Target.java:435)
                    at org.apache.tools.ant.Target.performTasks(Target.java:456)
                    at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
                    at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
                    at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
                    at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:441)
                    ... 33 more

                Caused by: C:\myEclipse\eclipse.platform.releng.aggregator\eclipse.platform.swt\bundles\org.eclipse.swt\buildSWT.xml:1012: exec returned: 1
                    at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:643)
                    at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:669)
                    at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:495)
                    at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
                    at sun.reflect.GeneratedMethodAccessor67.invoke(Unknown Source)
                    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                    at java.lang.reflect.Method.invoke(Method.java:498)
                    at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
                    at org.apache.tools.ant.Task.perform(Task.java:348)
                    at org.apache.tools.ant.Target.execute(Target.java:435)
                    at org.apache.tools.ant.Target.performTasks(Target.java:456)
                    at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
                    at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
                    at org.apache.tools.ant.Project.executeTargets(Project.java:1248)
                    at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:441)
                    ... 46 more
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn <goals> -rf :org.eclipse.swt.win32.win32.x86_64

---

I still have no answer, so I started a bounty. Please reproduce step 1 until step 5 on your machine (with the -Dnative=win32.win32.x86_64 argument added to the fourth step). Do you achieve a successful build of the Eclipse IDE for just one OS target (preferably Windows 64-bit)? Does the build go faster - compared to the general build process that targets all OS's?

PS: I've got some other questions related to the Eclipse "Build from scratch" topic. If you're interested:

• Building Eclipse IDE from scratch - how to choose CDT over JDT?
• Building Eclipse IDE from scratch - how to specify the target OS?
• Building Eclipse IDE from scratch - making bigger icons sometimes successful

Let us all unite to build the Eclipse IDE ourselves - and tweak it to our needs ;-)

---

EDIT :

Mr/Mss. Zapl wrote an interesting comment. He/She thinks that the -Dnative argument is not correct, after he/she read the following link: http://comments.gmane.org/gmane.comp.ide.eclipse.cbi.devel/896

Hopefully we'll get closer to the answer now :-)

---

FINAL EDIT :

Suddenly this question catched the attention of the community. I got a lot of suggestions/answers in a short time. I cannot verify every single one of them right now because the build procedure takes about 2 hours.

I have 19 hours left to award the 50 points bounty. Please let me know if you successfully reproduced the STEPS 1..5 from this question on your own machine, and succeeded to build the Eclipse IDE from scratch for one specific OS. I will award you the bonus :-)

Answer 1

There are multiple projects that are built here

1. SWT natives. (-Dnative=)
2. IDE. (platform independent)

SWT natives are built(by SWT team) every day at 19:00 EST/EDT everyday and pushed to eclipse.platform.swt.binaries.git repo. This build requires a quite a bit of dependencies like visual studio, xulrunner, WebKit, etc. For trying out a build I don't suggest you to attempt this .

CBI(Common Build Infrastructure) does the eclipse plugin builds and signing and packaging. This process is platform independent.

CBI build starts at 20:00 EST/EDT daily and the swt.binaries are picked up only during packaging. The packaging would probably takes about 5 minutes in the whole process. So I am not expecting much difference if you specify only 1 target to build.

Answer 2

This snippet is taken from a working pom.xml that exports to win32.win32.x86_64. Maybe some parts are not required for your project but it may be a good starting point to add this to your pom file.

..
..
<build>
    <plugins>
        <plugin>
            <groupId>org.eclipse.tycho</groupId>
            <artifactId>tycho-maven-plugin</artifactId>
            <version>0.20.0</version>
            <extensions>true</extensions>
        </plugin>
        <plugin>
            <groupId>org.eclipse.tycho</groupId>
            <artifactId>target-platform-configuration</artifactId>
            <version>0.20.0</version>
            <configuration>
                <resolver>p2</resolver>
                <environments>
                    <environment>
                        <os>win32</os>
                        <ws>win32</ws>
                        <arch>x86_64</arch>
                    </environment>
                </environments>
            </configuration>
        </plugin>
        <plugin>
            <groupId>org.eclipse.tycho</groupId>
            <artifactId>tycho-compiler-plugin</artifactId>
            <version>0.20.0</version>
            <configuration>
                <compilerArgument>-err:-forbidden</compilerArgument>
            </configuration>
        </plugin>
    </plugins>
</build>
..
..

EDIT

As mentioned in the comments I followed your steps and had some problems. I solved most of them now. If someone else also gets a lot of errors here are some solutions.

• In the file eclipse.pde.ui/pom.xml line 19: the version was 4.6.0-SNAPSHOT, but has to be 4.7.0-SNAPSHOT

• Hint: To speed up the build you can add e.g. "-T 4" to the mvn command to enable parallel builds. I finally ended with >mvn clean verify -DSkipTests -T 4 -Dnative=win32.win32.x86_64

After this fix I was able to run until I got the same errors mentioned in the initial question. I went to the project folder \eclipse.platform.swt.binaries\bundles\org.eclipse.swt.win32.win32.x86_64 and started an Ant build (Apache Ant version 1.9.7) and it finished without errors. But I got the same errors when it was executed by the Maven build. Thus, I checked lines 915 and 1012 of the file \eclipse.platform.swt\bundles\org.eclipse.swt\buildSWT.xml. After setting failonerror="false" (is set to "true" by default) in both lines I was able to continue with the build. I assess this as a very poor solution and it should not be the final solution for a problem to disable such tests.

Finally, I ended with errors in project org.eclipse.pde.build.tests and stopped following the problems. The whole project seems to be misconfigured in many parts and I could not find a better solution so far. People should remember the golden rule not to commit anything that breaks the build to avoid such problems. Maybe the problems will be solved after a few updates to the repo.