黑猴子的家：Oozie 排程 Shell 腳本

1、解壓官方案例模闆

[[email protected] oozie-4.0.0-cdh5.3.6]$ tar -xzvf oozie-examples.tar.gz -C ./
           

2、建立工作目錄

[[email protected] oozie-4.0.0-cdh5.3.6]$ mkdir oozie-apps/
           

3、拷貝任務模闆到oozie-apps/目錄

[[email protected] oozie-4.0.0-cdh5.3.6]$ cp -r examples/apps/shell/ oozie-apps/
           

4、随意編寫一個腳本p1.sh

[[email protected] oozie-4.0.0-cdh5.3.6]$ where ifconfig
[[email protected] oozie-4.0.0-cdh5.3.6]$ vim oozie-apps/shell/p1.sh
##内容如下
#!/bin/bash
/sbin/ifconfig > /tmp/p1.log

[[email protected] oozie-4.0.0-cdh5.3.6]$ chmod 755 oozie-apps/shell/p1.sh
           

5、修改job.properties和workflow.xml檔案

job.properties

[[email protected] oozie-4.0.0-cdh5.3.6]$ vim oozie-apps/shell/job.properties
#HDFS位址，端口8020或者9000，主要看自己的設定
nameNode=hdfs://hadoop102:9000

#ResourceManager位址， 8032預設端口不用改
jobTracker=hadoop103:8032

#隊列名稱
queueName=default

examplesRoot=oozie-apps  
##hdfs://hadoop102:9000/user/victor/oozie-apps/shell
oozie.wf.application.path=${nameNode}/user/${user.name}/${examplesRoot}/shell

EXEC=p1.sh
           

workflow.xml

[[email protected] oozie-4.0.0-cdh5.3.6]$ vim oozie-apps/shell/workflow.xml
<workflow-app xmlns="uri:oozie:workflow:0.4" name="shell-wf">
    <start to="shell-node" />
    <action name="shell-node">
        <shell xmlns="uri:oozie:shell-action:0.2">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>mapred.job.queue.name</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
            <exec>${EXEC}</exec>
            <!-- <argument>my_output=Hello Oozie</argument> -->
            <file>/user/victor/oozie-apps/shell/${EXEC}#${EXEC}</file>
            <capture-output />
        </shell>
        <ok to="end" />
        <error to="fail" />
    </action>
    <decision name="check-output">
        <switch>
            <case to="end">
                ${wf:actionData('shell-node')['my_output'] eq 'Hello Oozie'}
            </case>
            <default to="fail-output" />
        </switch>
    </decision>
    <kill name="fail">
        <message>Shell action failed, error
            message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <kill name="fail-output">
        <message>Incorrect output, expected [Hello Oozie] but was
            [${wf:actionData('shell-node')['my_output']}]</message>
    </kill>
    <end name="end" />
</workflow-app>
           

6、上傳任務配置

[[email protected] oozie-4.0.0-cdh5.3.6]$ hdfs dfs -put oozie-apps/ /user/victor
           

7、執行任務

[[email protected] oozie-4.0.0-cdh5.3.6]$ bin/oozie job \
-oozie http://hadoop102:11000/oozie \
-config oozie-apps/shell/job.properties \
-run
           

8、殺死某個任務

[[email protected] oozie-4.0.0-cdh5.3.6]$ bin/oozie job \
-oozie http://hadoop102:11000/oozie \
-kill 0000004-170425105153692-oozie-z-W
           

oozie 有時候因為err，你給它kill 掉， oozie 的web還顯示，這是一個oozie 的bug,不要緊，放着就行了， 強迫症的怎麼辦呢？？ 去mysql oozie庫 相應表把相應資料删除了就行了

9、注意事項

編輯檔案時，使用vim編輯器，注意執行權限