Logo Questions Linux Laravel Mysql Ubuntu Git Menu
HTML CSS JAVASCRIPT SQL PYTHON PHP BOOTSTRAP JAVA JQUERY R React Kotlin
 

Unresolved dependency: com.hadoop.gplcompression#hadoop-lzo;0.4.16 when "sbt update" in scalding

Tags:

scala

sbt

scalding

After getting code from git using clone https://github.com/twitter/scalding.git and doing ./sbt update I get:

::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: com.hadoop.gplcompression#hadoop-lzo;0.4.16: not found

And later:

sbt.ResolveException: unresolved dependency: com.hadoop.gplcompression#hadoop-lzo;0.4.16: not found
    at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:213)
    at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:122)
    at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:121)
    at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116)

I have removed ~/.sbt directory, installed latest Scala and cloned repository anew. The problem remains.

My configuration:

[dk@localhost scalding]$ scala -version
Scala code runner version 2.10.3 -- Copyright 2002-2013, LAMP/EPFL

[dk@localhost scalding]$ uname -a
Linux localhost.localdomain 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

CentOS 6.5 (Final)

What can be possibly wrong with ./sbt update? Should I install the dependency hadoop-lzo separately and set CLASSPATH to install Scalding? Any other solutions?

Here is the complete installation log:

[dk@localhost wks]$ cd scalding
[dk@localhost scalding]$ ./sbt update
Downloading sbt launcher for 0.13.0:
  From  http://typesafe.artifactoryonline.com/typesafe/ivy-releases/org.scala-sbt/sbt-launch/0.13.0/sbt-launch.jar
    To  /home/dk/.sbt/launchers/0.13.0/sbt-launch.jar
Getting org.scala-sbt sbt 0.13.0 ...
:: retrieving :: org.scala-sbt#boot-app
    confs: [default]
    43 artifacts copied, 0 already retrieved (12440kB/229ms)
Getting Scala 2.10.2 (for sbt)...
:: retrieving :: org.scala-sbt#boot-scala
    confs: [default]
    5 artifacts copied, 0 already retrieved (24390kB/150ms)
[info] Loading project definition from /home/dk/wks/scalding/project
[info] Updating {file:/home/dk/wks/scalding/project/}scalding-build...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Compiling 3 Scala sources to /home/dk/wks/scalding/project/target/scala-2.10/sbt-0.13/classes...
[warn] there were 10 deprecation warning(s); re-run with -deprecation for details
[warn] there were 2 feature warning(s); re-run with -feature for details
[warn] two warnings found
[info] Set current project to scalding (in build file:/home/dk/wks/scalding/)
[info] Updating {file:/home/dk/wks/scalding/}scalding-args...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Updating {file:/home/dk/wks/scalding/}scalding-date...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Updating {file:/home/dk/wks/scalding/}maple...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Updating {file:/home/dk/wks/scalding/}scalding...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Updating {file:/home/dk/wks/scalding/}scalding-core...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Updating {file:/home/dk/wks/scalding/}scalding-json...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Updating {file:/home/dk/wks/scalding/}scalding-parquet...
[info] Resolving com.hadoop.gplcompression#hadoop-lzo;0.4.16 ...
[warn]  module not found: com.hadoop.gplcompression#hadoop-lzo;0.4.16
[warn] ==== local: tried
[warn]   /home/dk/.ivy2/local/com.hadoop.gplcompression/hadoop-lzo/0.4.16/ivys/ivy.xml
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom
[warn] ==== snapshots: tried
[warn]   http://oss.sonatype.org/content/repositories/snapshots/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom
[warn] ==== releases: tried
[warn]   http://oss.sonatype.org/content/repositories/releases/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom
[warn] ==== Concurrent Maven Repo: tried
[warn]   http://conjars.org/repo/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom
[warn] ==== Clojars Repository: tried
[warn]   http://clojars.org/repo/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom
[warn] ==== Twitter Maven: tried
[warn]   http://maven.twttr.com/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: com.hadoop.gplcompression#hadoop-lzo;0.4.16: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[info] Updating {file:/home/dk/wks/scalding/}scalding-jdbc...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Updating {file:/home/dk/wks/scalding/}scalding-avro...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[info] Updating {file:/home/dk/wks/scalding/}scalding-commons...
[info] Resolving com.hadoop.gplcompression#hadoop-lzo;0.4.16 ...
[warn]  module not found: com.hadoop.gplcompression#hadoop-lzo;0.4.16
[warn] ==== local: tried
[warn]   /home/dk/.ivy2/local/com.hadoop.gplcompression/hadoop-lzo/0.4.16/ivys/ivy.xml
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom
[warn] ==== snapshots: tried
[warn]   http://oss.sonatype.org/content/repositories/snapshots/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom
[warn] ==== releases: tried
[warn]   http://oss.sonatype.org/content/repositories/releases/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom
[warn] ==== Concurrent Maven Repo: tried
[warn]   http://conjars.org/repo/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom
[warn] ==== Clojars Repository: tried
[warn]   http://clojars.org/repo/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom
[warn] ==== Twitter Maven: tried
[warn]   http://maven.twttr.com/com/hadoop/gplcompression/hadoop-lzo/0.4.16/hadoop-lzo-0.4.16.pom
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: com.hadoop.gplcompression#hadoop-lzo;0.4.16: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[info] Updating {file:/home/dk/wks/scalding/}scalding-repl...
[info] Resolving org.codehaus.jackson#jackson-core-asl;1.8.8 ...
[info] Done updating.
sbt.ResolveException: unresolved dependency: com.hadoop.gplcompression#hadoop-lzo;0.4.16: not found
    at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:213)
    at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:122)
    at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:121)
    at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116)
    at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116)
    at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:104)
    at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:51)
    at sbt.IvySbt$$anon$3.call(Ivy.scala:60)
    at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:98)
    at 
....
.... many lines of exception stack-trace are skipped ...
....

[error] (scalding-commons/*:update) sbt.ResolveException: unresolved dependency: com.hadoop.gplcompression#hadoop-lzo;0.4.16: not found
[error] (scalding-parquet/*:update) sbt.ResolveException: unresolved dependency: com.hadoop.gplcompression#hadoop-lzo;0.4.16: not found
[error] Total time: 53 s, completed Feb 14, 2014 4:51:51 PM
[dk@localhost scalding]$ ./sbt update^C
[dk@localhost scalding]$ uname -a
Linux localhost.localdomain 2.6.32-431.3.1.el6.x86_64 #1 SMP Fri Jan 3 21:39:27 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
[dk@localhost scalding]$ scala -version
Scala code runner version 2.10.3 -- Copyright
like image 956
Anton Ashanin Avatar asked Feb 14 '14 13:02

Anton Ashanin

2 Answers

SBT is unable to fetch com.hadoop.gplcompression#hadoop-lzo;0.4.16 from resolvers defined in your build.sbt

Adding maven.twttr.com: "Twitter Maven Repo" at "http://maven.twttr.com" should work.

like image 154
Hanginbat Avatar answered Sep 25 '22 05:09

Hanginbat

I don't think that would work anymore, use: "https://maven.twttr.com" instead. (Notice the https) They changed it in Jan 2018

Hope it helps

like image 38
Nir Hedvat Avatar answered Sep 25 '22 05:09

Nir Hedvat
Sign in to Comment

Related questions

Convert scala list to DataFrame or DataSet
Can't find spark submit when typing spark-shell
Convert Row to map in spark scala
: _* notation in scala [duplicate]
Error when Spark 2.2.0 standalone mode write Dataframe to local single-node Kafka
Unknown data type "JSONB" when running tests in play slick with H2 Database
How to create a random instance of a case class?
Why does the definition of Array.map in Scala is "throw new Error()"
Why can I assign null to a Unit value and why does it get converted to ()?
Why can't I create an array of generic type?
Is there a standard Scala function for running a block with a timeout?
How to convert a Some(" ") to None in one-line?
How to initialize empty variables from your own type in Scala?
Scala syntax to access property of an option inline and chain "OrElse"?
What is the syntax for creating a Map in Scala that uses an enum as a key?
Can anyone explain how the symbol "=>" is used in Scala
Getting started with Scala, Scalatest, and Maven
Futures for blocking calls in Scala
Parallel version of Files.walkFileTree (java or scala)
def or val for defining Function in Scala

Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!