Sqoop Hive exited with status 1

Question

When I execute:

sqoop import --connect jdbc:mysql://localhost/testdb --table test --hive-table test --hive-import -m 1 

I get following error message:

13/04/21 16:42:50 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Hive exited with status 1
    at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:364)
    at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:314)
    at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:226)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:415)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:238)

I tried to google for it but found no solution. I have Hadoop set up locally in pseudo-distributed fashion. Hive is running fine ... I used the embedded metastore. Any ideas how to fix this? Thanks, Diddy

Answer 1

Based on the log it seems that you're hitting following exception:

13/04/22 18:34:44 INFO hive.HiveImport: Exception in thread "main" java.lang.NoSuchMethodError: org.apache.thrift.EncodingUtils.setBit(BIZ)B

I've seen this issue before when users were using HBase and Hive in "incompatible" versions. The incompatibility can be generally on multiple levels, but this particular one is when HBase and Hive are using different thrift versions. As Sqoop is adding both HBase and Hive jars to the classpath, only one thrift version can be active and thus the "second" tool (usually hive) is not working properly.

Do you by any chance installed both HBase and Hive on the box where you're executing Sqoop? If so, can you check the thrift version that each project is using? Just search for "*thrift*.jar". If answer to both questions is positive, then you could potentially set HBASE_HOME to something non-existing to force Sqoop not load HBase's version of thrift.

Answer 2

I agree with Jarek Jarcec Cecho's answer.

Another workaround is to copy libthrift*.jar file from $HIVE_HOME/lib to $SQOOP_HOME/lib.