Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Simple oozie example of hive query?

Tags:

hadoop

hive

I'm trying to convert a simple work flow to oozie. I have tried looking through the oozie examples but they are a bit over-whelming. Effectively I want to run a query and output the result to a text file.

hive -e 'select * from tables' > output.txt

How to I got about translating that into oozie to have it run every hour?

like image 321
nickponline Avatar asked May 08 '12 13:05

nickponline


1 Answers

Your workflow might look something like this...
workflow.xml

<workflow-app xmlns="uri:oozie:workflow:0.2" name="hive-wf">
    <start to="hive-node"/>
    <action name="hive-node">
        <hive xmlns="uri:oozie:hive-action:0.2">
           <job-tracker>localhost:50001</job-tracker>
            <name-node>hdfs://localhost:50000</name-node>
            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>default</value>
                </property>
                <property>
                    <name>oozie.hive.defaults</name>
                    <value>/user/user1/oozie/hive-site.xml</value>
                </property>
            </configuration>
            <script>script.q</script>
            <param>INPUT_TABLE=SampleTable</param>
            <param>OUTPUT=/user/user1/output-data/hive</param>
        </hive>
        <ok to="end"/>
        <error to="fail"/>
    </action>
    <kill name="fail">
        <message>Hive failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

So here hive-site.xml is the site xml present in $HIVE_HOME/conf folder.
script.q file contains the actual hive query. select * from ${INPUT_TABLE} .


how and where can we use the OUTPUT param?

like image 51
WR10 Avatar answered Sep 19 '22 09:09

WR10