Compile errors finding symbols including Pipeline, PCollection, PipelineOptions, etc

Question

As of today, I'm getting a build break for existing code that used to compile correctly, due to error locating many key classes in the Dataflow SDK for Java. For example:

[ERROR] /tmp/first-dataflow/src/main/java/com/google/cloud/dataflow/examples/common/DataflowExampleUtils.java:[30,37] cannot find symbol
[ERROR] symbol:   class Pipeline
[ERROR] location: package com.google.cloud.dataflow.sdk

Have the APIs changed?

Answer 1

Existing Maven projects that use the previously recommended version range [1.0.0, 2.0.0) for the Google Cloud Dataflow SDK for Java may soon pick up the new 2.0.0-beta1 version of that SDK. This new version has APIs that are not compatible with the 1.x releases, so using this it with existing code will cause these kinds of breakages.

If you are impacted, update your Maven pom.xml to use a version range that precludes anything in the 2.x family, for example by using [1.0.0, 1.99), as follows:

<dependency>
  <groupId>com.google.cloud.dataflow</groupId>
  <artifactId>google-cloud-dataflow-java-sdk-all</artifactId>
  <version>[1.0.0,1.99)</version>
</dependency>

This should fix your compile issues and allow you to continue with the most recent release in the 1.x series (currently 1.9.0).

For more information and updates, you can follow this GitHub issue.

Separately, you can learn more about the 2.0.0-beta1 release, including what these incompatible API changes are, in its release notes. But be aware that it is an early preview and has corresponding caveats about API stability, support timelines, and documentation.