黑猴子的家：Oozie 执行多个Job调度

使用Oozie执行多个Job调度

1、解压官方案例模板

[[email protected] oozie-4.0.0-cdh5.3.6]$ tar -xzvf oozie-examples.tar.gz
           

2、编写脚本

[[email protected] oozie-4.0.0-cdh5.3.6]$ vim oozie-apps/shell/p2.sh
##内容如下
#!/bin/bash
/bin/date > /tmp/p2.log
           

3、修改job.properties和workflow.xml文件

job.properties

[[email protected] oozie-4.0.0-cdh5.3.6]$ vim oozie-apps/shell/job.properties
nameNode=hdfs://hadoop102:9000
jobTracker=hadoop103:8032
queueName=default
examplesRoot=oozie-apps
oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/shell
EXEC1=p1.sh
EXEC2=p2.sh
           

workflow.xml

[[email protected] oozie-4.0.0-cdh5.3.6]$ vim oozie-apps/shell/workflow.xml
<workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
    <start to="p1-shell-node" />
    <action name="p1-shell-node">
        <shell xmlns="uri:oozie:shell-action:0.2">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
            <exec>${EXEC1}</exec>
            <file>/user/victor/oozie-apps/shell/${EXEC1}#${EXEC1}</file>
            <!-- <argument>my_output=Hello Oozie</argument> -->
            <capture-output />
        </shell>
        <ok to="p2-shell-node" />
        <error to="fail" />
    </action>

    <action name="p2-shell-node">
        <shell xmlns="uri:oozie:shell-action:0.2">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
            <exec>${EXEC2}</exec>
            <file>/user/victor/oozie-apps/shell/${EXEC2}#${EXEC2}</file>
            <!-- <argument>my_output=Hello Oozie</argument> -->
            <capture-output />
        </shell>
        <ok to="end" />
        <error to="fail" />
    </action>
    <decision name="check-output">
        <switch>
            <case to="end">
                ${wf:actionData('shell-node')['my_output'] eq 'Hello Oozie'}
            </case>
            <default to="fail-output" />
        </switch>
    </decision>
    <kill name="fail">
        <message>Shell action failed, error
            message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <kill name="fail-output">
        <message>Incorrect output, expected [Hello Oozie] but was
            [${wf:actionData('shell-node')['my_output']}]</message>
    </kill>
    <end name="end" />
</workflow-app>
           

4、上传任务配置

删除

[[email protected] oozie-4.0.0-cdh5.3.6]$ hdfs dfs -rm -R /user/victor/oozie-apps/
           

上传

[[email protected] oozie-4.0.0-cdh5.3.6]$ hdfs dfs -put oozie-apps/ /user/victor
           

5、执行任务

[[email protected] oozie-4.0.0-cdh5.3.6]$ bin/oozie job \
-oozie http://hadoop102:11000/oozie \
-config oozie-apps/shell/job.properties \
-run