Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Apache Oozie failed loading ShareLib

Tags:

hadoop

oozie

i got the following oozie.log :

org.apache.oozie.service.ServiceException: E0104: Could not fully initialize service [org.apache.oozie.service.ShareLibService], Not able to cache sharelib. An Admin needs to install the sharelib with oozie-setup.sh and issue the 'oozie admin' CLI command to update the sharelib

i run the following command:

oozie-setup.sh sharelib create -fs hdfs://localhost:54310
oozied.sh start

hdfs dfs -ls /user/hduser/share/lib
15/02/24 18:05:03 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 3 items
drwxr-xr-x   - hduser supergroup          0 2015-02-24 17:19 /user/hduser/share/lib/lib_20150224171855
drwxr-xr-x   - hduser supergroup          0 2015-02-24 17:19 /user/hduser/share/lib/lib_20150224171908
drwxr-xr-x   - hduser supergroup          0 2015-02-24 17:29 /user/hduser/share/lib/lib_20150224172857 

but :

oozie admin -shareliblist -oozie http://localhost:11000/oozie
[Available ShareLib]

oozie admin -sharelibupdate -oozie http://localhost:11000/oozie
null

my oozie-site.xml contains:

<property>
    <name>oozie.service.WorkflowAppService.system.libpath</name>
    <value>/user/${user.name}/share/lib/</value>
</property>

<property>
    <name>oozie.service.HadoopAccessorService.hadoop.configurations</name>
    <value>*=hadoop-conf</value>
</property>

Do you have any idea of my mistake ?

like image 647
seb835 Avatar asked Dec 05 '22 22:12

seb835


2 Answers

I struggled to fix the same error for couple of days and I finally fix it.

It was related to the sharelib service that was looking for sharelib folder on my local file system instead of my hdfs.

So to fix it:

stop oozie

edit conf/oozie-site.xml

<property>        
      <name>oozie.service.HadoopAccessorService.hadoop.configurations</name> 
      <value>*=/usr/local/hadoop/etc/hadoop/</value>
</property>

restart oozie.

And Voila!

The value of the property was set to *=hadoop-conf by default. I still don't know yet what hadoop-conf should point to but in my case it wasn't the configuration folder hadoop so I changed it for *=/usr/local/hadoop/etc/hadoop/.

like image 56
Breach Avatar answered Jan 19 '23 00:01

Breach


In my oozie-4.2.0, the default setting *hadoop-conf points to the subfolder in

path/to/oozie/conf/hadoop-conf

In my case, there is a file 'core-site.xml', which I think should be the config file. I add below properties in this file.

<property>
    <name>fs.defaultFS</name>
    <value>hdfs://localhost:9000</value>
</property>

Restart oozie and the issue is resolved.

like image 38
Juniver Hazoic Avatar answered Jan 19 '23 00:01

Juniver Hazoic