Menu
I am running Geotools inside of Apache Storm, and having a heck of a time with geotools dependency management. Every thing works when I run the storm cluster locally (windows 7 laptop), but when I deploy to the cluster I get this exception in my logs.
EDIT: Here is the Java I use that calls this functionality
GridCoverage2D image =
new GeoTiffReader(f).read(new GeneralParameterValue[]{policy, gridsize, useJaiRead});
/**
* reproject to WGS84
*/
CoordinateReferenceSystem targetCRS = CRS.decode("EPSG:4326");
GridCoverage2D reprojectedImage = (GridCoverage2D) Operations.DEFAULT.resample(image, targetCRS);
Here's my environment
geotools 11.1
java 7
POM is below
running on windows 7 when in local mode (works perfectly here)
this problem happens on Ubuntu 12.04.5 LTS (GNU/Linux 3.2.0-63-virtual x86_64)
here's the exception from the log
[ERROR] Exception in Bolt org.geotools.data.DataSourceException: GEOTIFF Module Error Report
No code "EPSG:32637" from authority "EPSG" found for object of type "EngineeringCRS".
ModelPixelScaleTag: [2.0,2.0,0.0]
ModelTiePointTag: (1 tie points)
TP #0: [0.0,0.0,0.0] -> [337668.0,3837288.0,0.0]
ModelTransformationTag: NOT AVAILABLE
GeoKey #1: Key = 2049, Value = GCS_WGS_1984
GeoKey #2: Key = 2054, Value = 9102
GeoKey #3: Key = 3072, Value = 32637
GeoKey #4: Key = 1024, Value = 1
GeoKey #5: Key = 1025, Value = 1
GeoKey #6: Key = 1026, Value = PCS Name = WGS_1984_UTM_zone_37N
GeoKey #7: Key = 3076, Value = 9001
org.opengis.referencing.NoSuchAuthorityCodeException: No code "EPSG:32637" from authority "EPSG" found for object of type "EngineeringCRS".
at org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.noSuchAuthorityException(CartesianAuthorityFactory.java:136)
at org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.createEngineeringCRS(CartesianAuthorityFactory.java:130)
at org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.createCoordinateReferenceSystem(CartesianAuthorityFactory.java:121)
at org.geotools.referencing.factory.AuthorityFactoryAdapter.createCoordinateReferenceSystem(AuthorityFactoryAdapter.java:802)
at org.geotools.coverage.grid.io.imageio.geotiff.GeoTiffMetadata2CRSAdapter.createProjectedCoordinateReferenceSystem(GeoTiffMetadata2CRSAdapter.java:284)
at org.geotools.coverage.grid.io.imageio.geotiff.GeoTiffMetadata2CRSAdapter.createCoordinateSystem(GeoTiffMetadata2CRSAdapter.java:205)
at org.geotools.gce.geotiff.GeoTiffReader.getHRInfo(GeoTiffReader.java:299)
at org.geotools.gce.geotiff.GeoTiffReader.<init>(GeoTiffReader.java:211)
at org.geotools.gce.geotiff.GeoTiffReader.<init>(GeoTiffReader.java:156)
at dgi.eii.utils.PixelExtractor.extract(PixelExtractor.java:80)
at dgi.eii.storm.bolts.RasterPixelExtractorBolt.execute(RasterPixelExtractorBolt.java:59)
at backtype.storm.daemon.executor$fn__5641$tuple_action_fn__5643.invoke(executor.clj:631)
at backtype.storm.daemon.executor$mk_task_receiver$fn__5564.invoke(executor.clj:399)
at backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58)
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
at backtype.storm.daemon.executor$fn__5641$fn__5653$fn__5700.invoke(executor.clj:746)
at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:745)
I also get the same error when encountering an image with EPSQ:4326
[ERROR] Exception in Bolt org.geotools.data.DataSourceException: GEOTIFF Module Error Report
No code "EPSG:4326" from authority "EPSG" found for object of type "EngineeringCRS".
ModelPixelScaleTag: [2.0230196490091333E-5,2.0230196490071028E-5,0.0]
ModelTiePointTag: (1 tie points)
TP #0: [0.0,0.0,0.0] -> [36.8167576323252,34.429979601192464,0.0]
ModelTransformationTag: NOT AVAILABLE
GeoKey #1: Key = 2048, Value = 4326
GeoKey #2: Key = 2049, Value = GCS_WGS_1984
GeoKey #3: Key = 2054, Value = 9102
GeoKey #4: Key = 1024, Value = 2
GeoKey #5: Key = 2057, Value = 6378137.0
GeoKey #6: Key = 1025, Value = 1
GeoKey #7: Key = 2059, Value = 298.257223563
org.opengis.referencing.NoSuchAuthorityCodeException: No code "EPSG:4326" from authority "EPSG" found for object of type "EngineeringCRS".
at org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.noSuchAuthorityException(CartesianAuthorityFactory.java:136)
at org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.createEngineeringCRS(CartesianAuthorityFactory.java:130)
at org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.createCoordinateReferenceSystem(CartesianAuthorityFactory.java:121)
at org.geotools.referencing.factory.AuthorityFactoryAdapter.createCoordinateReferenceSystem(AuthorityFactoryAdapter.java:802)
at org.geotools.coverage.grid.io.imageio.geotiff.GeoTiffMetadata2CRSAdapter.createGeographicCoordinateReferenceSystem(GeoTiffMetadata2CRSAdapter.java:389)
at org.geotools.coverage.grid.io.imageio.geotiff.GeoTiffMetadata2CRSAdapter.createCoordinateSystem(GeoTiffMetadata2CRSAdapter.java:208)
at org.geotools.gce.geotiff.GeoTiffReader.getHRInfo(GeoTiffReader.java:299)
at org.geotools.gce.geotiff.GeoTiffReader.<init>(GeoTiffReader.java:211)
at org.geotools.gce.geotiff.GeoTiffReader.<init>(GeoTiffReader.java:156)
at dgi.eii.utils.PixelExtractor.extract(PixelExtractor.java:80)
at dgi.eii.storm.bolts.RasterPixelExtractorBolt.execute(RasterPixelExtractorBolt.java:59)
at backtype.storm.daemon.executor$fn__5641$tuple_action_fn__5643.invoke(executor.clj:631)
at backtype.storm.daemon.executor$mk_task_receiver$fn__5564.invoke(executor.clj:399)
at backtype.storm.disruptor$clojure_handler$reify__745.onEvent(disruptor.clj:58)
at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:125)
at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:99)
at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
at backtype.storm.daemon.executor$fn__5641$fn__5653$fn__5700.invoke(executor.clj:746)
at backtype.storm.util$async_loop$fn__457.invoke(util.clj:431)
at clojure.lang.AFn.run(AFn.java:24)
at java.lang.Thread.run(Thread.java:745)
I have seen several answers that indicated I need to put the gt-epsg-hsql in my pom, and I have, still doesn't find what it needs.
Here are my geotools POM entries
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-swing</artifactId>
<version>11.1</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-opengis</artifactId>
<version>11.1</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-epsg-hsql</artifactId>
<version>11.1</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-geotiff</artifactId>
<version>11.1</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-image</artifactId>
<version>11.1</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-wms</artifactId>
<version>11.1</version>
</dependency>
here is how I am building the uber-jar with maven shade plugin
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
<configuration>
<archive>
<manifestFile>src/main/resources/META-INF/MANIFEST.MF</manifestFile>
</archive>
<!-- <minimizeJar>true</minimizeJar>-->
<shadedArtifactAttached>true</shadedArtifactAttached>
<shadedClassifierName>stand-alone</shadedClassifierName>
<artifactSet>
<excludes>
<exclude>org.slf4j:slf4j-api:jar:</exclude>
<exclude>org.slf4j:slf4j-simple:jar:1.6.4:jar:</exclude>
<exclude>org.slf4j:slf4j-log4j12:jar:</exclude>
<exclude>org.slf4j:jcl-over-slf4j:jar:</exclude>
<exclude>org.slf4j:slf4j-api:jar:1.7.5:jar:</exclude>
<!-- <exclude>org.slf4j*:</exclude>-->
<exclude>commons-logging:commons-logging:jar:</exclude>
<exclude>commons-logging:commons-logging-api:jar:</exclude>
</excludes>
</artifactSet>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<finalName>storm-topos</finalName>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>dgi.eii.storm.base.StormTopologyRunner</mainClass>
</transformer>
<!-- <transformer
implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.handlers</resource>
</transformer>
<transformer
implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.schemas</resource>
</transformer>-->
</transformers>
</configuration>
</plugin>
</plugins>
</build>
EDIT: So i figured out a few things and got a little further... now getting this error
[ERROR] Exception in Bolt org.geotools.data.DataSourceException: org.hsqldb.DatabaseURL.parseURL(Ljava/lang/String;ZZ)Lorg/hsqldb/persist/HsqlProperties;
This slight advancement that led to the above new error was due to using the right transformers, and using the maven jar plugin to inject the right entries into the MANIFEST.MF file. Here is the new build section of my POM, which might help someone at least overcome the consolidation of geotools' META-INF/services files and to inject the right entries to overcome the old vendorname cannot be null! error
<build>
<resources>
<resource>
<directory>${basedir}/src/main/resources</directory>
<filtering>false</filtering>
<includes>
<include>schema.xsd</include>
</includes>
</resource>
</resources>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
</execution>
</executions>
<configuration>
<shadedArtifactAttached>true</shadedArtifactAttached>
<shadedClassifierName>stand-alone</shadedClassifierName>
<artifactSet>
<excludes>
<exclude>org.slf4j:slf4j-api:jar:</exclude>
<exclude>org.slf4j:slf4j-log4j12:jar:</exclude>
<exclude>org.slf4j:jcl-over-slf4j:jar:</exclude>
<exclude>commons-logging:commons-logging:jar:</exclude>
<exclude>commons-logging:commons-logging-api:jar:</exclude>
</excludes>
</artifactSet>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<finalName>insightcloud-storm-topos</finalName>
<transformers>
<transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>dgi.eii.storm.base.StormTopologyRunner</mainClass>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
<transformer
implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.handlers</resource>
</transformer>
<transformer
implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>META-INF/spring.schemas</resource>
</transformer>
</transformers>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifestFile>src/main/resources/META-INF/MANIFEST.MF</manifestFile>
</archive>
</configuration>
</plugin>
</plugins>
</build>
693
Geotools uses Java's Service infrastructure to load the class responsible for EPSG lookups. The original gt-epsg-hsql.jar has entries in /META-INF/services/ which specify which interfaces are implemented by the jar-file, and which can be dynamically loaded at runtime.
When building a uber-jar, you combine multiple jar files into one. Most likely, another jar file implements the same interfaces as well (for example gt-referencing.jar) and has thus files with the same names in its /META-INF/services/. When putting everything into one jar file, those entries will very likely be overwritten (at least I couldn't find any reference that the maven-shade-plugin merges such services files).
You could verify that by looking at the services-directory in the created uber-jar, especially at the entry /META-INF/services/org.opengis.referencing.crs.CRSAuthorityFactory. Both gt-epsg-hsql.jar and gt-referencing.jar have such a file (and other jar-files from GeoTools probably as well), and most likely, only the content of one will be in your uber-jar, resulting in all the other classes not being found/loaded at runtime.
I'm not really familiar with the maven-shade-plugin, but other questions on SO (like [1]) suggest to use an additional transformer:
<transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer" />
EDIT: As this answer gets regular visits, and I'm familiar with the shade plugin by now, here is a more detailed guide to use the shade-plugin.
So instead of using the maven-assembly plugin, we can use the maven-shade plugin to create an all-in-one jar. To do so, configure the maven-shade plugin indoor pom.xml and bind it to the package phase (so, whenever you call mvn package, the shaded jar will be created:
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.1.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<transformers>
<!-- This bit sets the main class for the executable jar as you otherwise -->
<!-- would with the assembly plugin -->
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<manifestEntries>
<Main-Class>com.example.YourMainClass</Main-Class>
<Implementation-Vendor>Your Company Name</Implementation-Vendor>
<Implementation-Version>${project.version}</Implementation-Version>
</manifestEntries>
</transformer>
<!-- This bit merges the various GeoTools META-INF/services files -->
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
The Implementation-Vendor and Implementation-Version may not be needed in all cases, but I observed situations where some code (I think JAI - Java advanced imaging) complained and misbehaved when this information was missing, likely as the original JAR included such information and the shaded one did not by default, so it might be best to just include it.
When running mvn package, it will rename the original jar-file to something like original-myArtifact.jar and place the shaded (=all-in-one, fat-jar, uber-jar) at myArtifact.jar. If you don't like this behavior and want to keep the original jar file intact and have the shaded jar-file separat, add the following line inside the configuration block:
<shadedArtifactAttached>true</shadedArtifactAttached>
In this case, the build process will create a file myArtifact-shaded.jar (similar to myArtifact-jar-with-dependencies.jar created by the assembly plugin).
[1] Maven shade + resteasy Could find writer for content-type
78
I also ran into this problem. Instead of banging your head against the wall, there is a workaround you could use which is to create the CRS from WKT instead of using decode:
private static final String EPSG4326 = "GEOGCS[\"WGS 84\",DATUM[\"WGS_1984\",SPHEROID[\"WGS 84\",6378137,298.257223563,AUTHORITY[\"EPSG\",\"7030\"]],AUTHORITY[\"EPSG\",\"6326\"]],PRIMEM[\"Greenwich\",0,AUTHORITY[\"EPSG\",\"8901\"]],UNIT[\"degree\",0.01745329251994328,AUTHORITY[\"EPSG\",\"9122\"]],AUTHORITY[\"EPSG\",\"4326\"]]";
CoordinateReferenceSystem worldCRS = CRS.parseWKT(EPSG4326);
One tip is that if you use this method, the resulting CRS will store the longitude first in the WKT followed by the latitude. Using the ServicesResourceTransformer gives WKT that has latitude followed by longitude. More on this issue at this link http://docs.geotools.org/latest/userguide/library/referencing/order.html
8
GeoTools uses a factory based plugin system to provide multiple referencing databases, but you have to choose one of them. gt-referencing provides the interfaces and the factory. The actual authorities are in the EPSG plugins (choose only one to prevent conflict):
6
For other problems with hsql db, you can enable debug logging for geotools.
Geotools uses default java logging. Add following entries to logging.properties config file.
org.geotools.level = FINEST
org.geotools.handlers = java.util.logging.ConsoleHandler
This file is usually located in %JAVA_HOME% or explicitly set by
-Djava.util.logging.config.file=logging.properties
runtime argument
1
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
