azkaban搭建文檔、使用者管理以及維護文檔

準備檔案

這四個檔案，然後。

Azkaban 2.5安裝

• 系統時區設定

• 安裝和配置mysql

• Azkaban Web Server 安裝

• Azkaban Executor Server安裝

• 啟動驗證

• Azkaban web server插件安裝

• Azkaban Executor Server 插件安裝

一、系統時區配置

1.檢視時區

[[email protected] azkaban-web-2.5.0]$ more /etc/sysconfig/clock
ZONE="Asia/Shanghai"
UTC=true
ARC=false      

[[email protected] ~]$ more /etc/sysconfig/clock
# The time zone of the system is defined by the contents of /etc/localtime.
# This file is only for evaluation by system-config-date, do not rely on its
# contents elsewhere.
ZONE="Asia/Shanghai"
[[email protected] ~]$ su root
Password: 
[[email protected] hadoop]# vim /etc/sysconfig/clock 
[[email protected] hadoop]# more /etc/sysconfig/clock
# The time zone of the system is defined by the contents of /etc/localtime.
# This file is only for evaluation by system-config-date, do not rely on its
# contents elsewhere.
ZONE="Asia/Shanghai"
UTC=true
ARC=false
[[email protected] hadoop]#
           

2.修改時區

時區檔案都在/usr/share/zoneinfo，複制相應時區覆寫/etc/localtime即可

[[email protected] ~]$ ll /usr/share/zoneinfo
cp /usr/share/zoneinfo/Asia/Shanghai  /etc/localtime      

　修改web server 和excutor server 都需要配置時區，必須和系統時區一緻（注意，這一步可以到最後才來修改）

default.timezone.id=Asia/Shanghai      

安裝mysql

就不贅述了。

二、配置mysql

　　下載下傳mysql資料腳本

wget https://s3.amazonaws.com/azkaban2/azkaban2/2.5.0/azkaban-sql-script-2.5.0.tar.gz
      

tar -zxvf azkaban-sql-script-2.5.0.tar.gz –C ~/app/azkaban/      

[[email protected] azkaban]$ pwd
/home/hadoop/app/azkaban
[[email protected] azkaban]$ ll
total 0
[[email protected] azkaban]$ rz

[[email protected] azkaban]$ ll
total 4
-rw-r--r-- 1 hadoop hadoop 1928 May  1 11:43 azkaban-sql-script-2.5.0.tar.gz
[[email protected] azkaban]$





[[email protected] azkaban]$ pwd
/home/hadoop/app/azkaban
[[email protected] azkaban]$ ll
total 4
-rw-r--r-- 1 hadoop hadoop 1928 May  1 11:43 azkaban-sql-script-2.5.0.tar.gz
[[email protected] azkaban]$ tar -zxvf azkaban-sql-script-2.5.0.tar.gz 





[[email protected] azkaban]$ ll
total 8
drwxrwxr-x 2 hadoop hadoop 4096 May  1 12:04 azkaban-2.5.0
-rw-r--r-- 1 hadoop hadoop 1928 May  1 11:43 azkaban-sql-script-2.5.0.tar.gz
[[email protected] azkaban]$ rm azkaban-sql-script-2.5.0.tar.gz 
[[email protected] azkaban]$ ll
total 4
drwxrwxr-x 2 hadoop hadoop 4096 May  1 12:04 azkaban-2.5.0
[[email protected] azkaban]$
           

建立使用者、配置設定權限并執行腳本

mysql –uroot –pxxxx
建立資料庫: CREATE DATABASE azkaban;
建立使用者名：CREATE USER 'azkaban'@'%' IDENTIFIED BY 'azkaban'
配置設定權限：GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,INDEX ON `azkaban`.* TO 'azkaban'@'%' WITH GRANT OPTION;
flush privileges；
quit;
通過azkaban賬号登入mysql：mysql –uazkaban –pazkaban
Use azkaban;
source ~/azkaban/azkaban-2.5.0/create-all-sql-2.5.0.sql
           

　　先開啟mysql服務程序

[[email protected] azkaban]# service mysqld start
Starting mysqld:                                           [  OK  ]
[[email protected] azkaban]# su hadoop
[[email protected] azkaban]$
           

[[email protected] azkaban]$ mysql -uroot -prootroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> CREATE DATABASE azkaban;
Query OK, 1 row affected (0.00 sec)

mysql> CREATE USER 'azkaban'@'%' IDENTIFIED BY 'azkaban';
Query OK, 0 rows affected (0.00 sec)

mysql> GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,INDEX ON 'azkaban'.* TO 'azkaban'@'%' WITH GRANT OPTION;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''azkaban'.* TO 'azkaban'@'%' WITH GRANT OPTION' at line 1
mysql> GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,INDEX ON `azkaban`.* TO 'azkaban'@'%' WITH GRANT OPTION;
Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

mysql> quit;
Bye
[[email protected] azkaban]$
           

通過azkaban賬号登入mysql
mysql –uazkaban –pazkaban
use azkaban;
source ~/app/azkaban/azkaban-2.5.0/create-all-sql-2.5.0.sql
           

[[email protected] azkaban]$ mysql -uazkaban -pazkaban
ERROR 1045 (28000): Access denied for user 'azkaban'@'localhost' (using password: YES)
[[email protected] azkaban]$ mysql -uroot -prootroot
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> GRANT ALL PRIVILEGES ON *.* to 'azkaban'@'%' IDENTIFIED BY 'azkaban' WITH GRANT OPTION;
Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON *.* to 'azkaban'@'master' IDENTIFIED BY 'azkaban' WITH GRANT OPTION;
Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON *.* to 'azkaban'@'localhost' IDENTIFIED BY 'azkaban' WITH GRANT OPTION;
Query OK, 0 rows affected (0.00 sec)

mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

mysql> select user,host,password from mysql.user;
+---------+-----------+-------------------------------------------+
| user    | host      | password                                  |
+---------+-----------+-------------------------------------------+
| root    | localhost | *6C362347EBEAA7DF44F6D34884615A35095E80EB |
| root    | master    |                                           |
| root    | 127.0.0.1 |                                           |
|         | localhost |                                           |
|         | master    |                                           |
| hive    | %         | *4DF1D66463C18D44E3B001A8FB1BBFBEA13E27FC |
| hive    | master    | *4DF1D66463C18D44E3B001A8FB1BBFBEA13E27FC |
| hive    | localhost | *4DF1D66463C18D44E3B001A8FB1BBFBEA13E27FC |
| azkaban | %         | *8E02B7AE57F9A19E165EB45CD3F705BF66985B85 |
| azkaban | master    | *8E02B7AE57F9A19E165EB45CD3F705BF66985B85 |
| azkaban | localhost | *8E02B7AE57F9A19E165EB45CD3F705BF66985B85 |
+---------+-----------+-------------------------------------------+
rows in set (0.01 sec)

mysql> exit;
Bye
[[email protected] azkaban]$
           

然後，此刻就可以通過azkaban賬号登入mysql了。

mysql –uazkaban –pazkaban
use azkaban;
source ~/app/azkaban/azkaban-2.5.0/create-all-sql-2.5.0.sql      

[[email protected] azkaban]$ pwd
/home/hadoop/app/azkaban
[[email protected] azkaban]$ ll
total 4
drwxrwxr-x 2 hadoop hadoop 4096 May  1 12:04 azkaban-2.5.0
[[email protected] azkaban]$ cd azkaban-2.5.0/
[[email protected] azkaban-2.5.0]$ pwd
/home/hadoop/app/azkaban/azkaban-2.5.0
[[email protected] azkaban-2.5.0]$ ll
total 88
-rwxr-xr-x 1 hadoop hadoop  129 Apr 22  2014 create.active_executing_flows.sql
-rwxr-xr-x 1 hadoop hadoop  216 Apr 22  2014 create.active_sla.sql
-rwxr-xr-x 1 hadoop hadoop 4694 Apr 22  2014 create-all-sql-2.5.0.sql
-rwxr-xr-x 1 hadoop hadoop  610 Apr 22  2014 create.execution_flows.sql
-rwxr-xr-x 1 hadoop hadoop  519 Apr 22  2014 create.execution_jobs.sql
-rwxr-xr-x 1 hadoop hadoop  358 Apr 22  2014 create.execution_logs.sql
-rwxr-xr-x 1 hadoop hadoop  224 Apr 22  2014 create.project_events.sql
-rwxr-xr-x 1 hadoop hadoop  227 Apr 22  2014 create.project_files.sql
-rwxr-xr-x 1 hadoop hadoop  280 Apr 22  2014 create.project_flows.sql
-rwxr-xr-x 1 hadoop hadoop  285 Apr 22  2014 create.project_permissions.sql
-rwxr-xr-x 1 hadoop hadoop  294 Apr 22  2014 create.project_properties.sql
-rwxr-xr-x 1 hadoop hadoop  380 Apr 22  2014 create.projects.sql
-rwxr-xr-x 1 hadoop hadoop  325 Apr 22  2014 create.project_versions.sql
-rwxr-xr-x 1 hadoop hadoop  155 Apr 22  2014 create.properties.sql
-rwxr-xr-x 1 hadoop hadoop  498 Apr 22  2014 create.schedules.sql
-rwxr-xr-x 1 hadoop hadoop  189 Apr 22  2014 create.triggers.sql
-rwxr-xr-x 1 hadoop hadoop   22 Apr 22  2014 database.properties
-rwxr-xr-x 1 hadoop hadoop  671 Apr 22  2014 update-all-sql-2.1.sql
-rwxr-xr-x 1 hadoop hadoop  156 Apr 22  2014 update-all-sql-2.2.sql
-rwxr-xr-x 1 hadoop hadoop  395 Apr 22  2014 update.execution_logs.2.1.sql
-rwxr-xr-x 1 hadoop hadoop   59 Apr 22  2014 update.project_properties.2.1.sql
[[email protected] azkaban-2.5.0]$
           

[[email protected] azkaban]$ mysql -uazkaban -pazkaban
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 5
Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use azkaban;
Database changed
mysql> source /home/hadoop/app/azkaban/azkaban-2.5.0/create-all-sql-2.5.0.sql
Query OK, 0 rows affected (0.03 sec)

Query OK, 0 rows affected (0.03 sec)

Query OK, 0 rows affected (0.01 sec)

Query OK, 0 rows affected (0.04 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.02 sec)

Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.03 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.02 sec)

Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.01 sec)

Query OK, 0 rows affected (0.00 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.02 sec)

Query OK, 0 rows affected (0.00 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.02 sec)

Query OK, 0 rows affected (0.00 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.01 sec)

Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.01 sec)

Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.01 sec)

Query OK, 0 rows affected (0.00 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.02 sec)

Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.01 sec)

Query OK, 0 rows affected (0.01 sec)

Query OK, 0 rows affected (0.01 sec)
Records: 0  Duplicates: 0  Warnings: 0

Query OK, 0 rows affected (0.02 sec)
　mysql> exit;
　Bye
 [[email protected] azkaban]$
           

三、Azkaban Web Server 安裝

1.下載下傳安裝包

wget https://s3.amazonaws.com/azkaban2/azkaban2/2.5.0/azkaban-web-server-2.5.0.tar.gz
tar –zxvf azkaban-web-server-2.5.0.tar.gz  –C  ~/app/azkaban/      

　　

　　我這裡，選擇離線上傳的方式，之前已經下載下傳好了。

[[email protected] azkaban]$ pwd
/home/hadoop/app/azkaban
[[email protected] azkaban]$ ll
total 4
drwxrwxr-x 2 hadoop hadoop 4096 May  1 12:04 azkaban-2.5.0
[[email protected] azkaban]$ rz

[[email protected] azkaban]$ ll
total 11716
drwxrwxr-x 2 hadoop hadoop     4096 May  1 12:04 azkaban-2.5.0
-rw-r--r-- 1 hadoop hadoop 11989669 May  1 11:44 azkaban-web-server-2.5.0.tar.gz
[[email protected] azkaban]$
           

[[email protected] azkaban]$ pwd
/home/hadoop/app/azkaban
[[email protected] azkaban]$ ll
total 11720
drwxrwxr-x 2 hadoop hadoop     4096 May  1 12:04 azkaban-2.5.0
drwxrwxr-x 8 hadoop hadoop     4096 May  1 14:29 azkaban-web-2.5.0
-rw-r--r-- 1 hadoop hadoop 11989669 May  1 11:44 azkaban-web-server-2.5.0.tar.gz
[[email protected] azkaban]$ rm azkaban-web-server-2.5.0.tar.gz 
[[email protected] azkaban]$ ll
total 8
drwxrwxr-x 2 hadoop hadoop 4096 May  1 12:04 azkaban-2.5.0
drwxrwxr-x 8 hadoop hadoop 4096 May  1 14:29 azkaban-web-2.5.0
[[email protected] azkaban]$
           

2.上傳依賴包

　　上傳mysql驅動包至 ~/app/azkaban/azkaban-web-2.5.0/extlib/

[[email protected] extlib]$ pwd
/home/hadoop/app/azkaban/azkaban-web-2.5.0/extlib
[[email protected] extlib]$ ll
total 0
[[email protected] extlib]$ rz
[[email protected] extlib]$ ll
total 812
-rw-r--r-- 1 hadoop hadoop 827942 Nov  3 09:37 mysql-connector-java-5.1.21.jar
[[email protected] extlib]$
           

3.生成jetty ssl所需密鑰檔案      

keytool -keystore keystore -alias jetty -genkey -keyalg RSA
        Enter keystore password:  mysteel
        What is your first and last name?
          [Unknown]:  mysteel
        What is the name of your organizational unit?
          [Unknown]:  mysteel
        What is the name of your organization?
          [Unknown]: mysteel
        What is the name of your City or Locality?
          [Unknown]:SH
        What is the name of your State or Province?
          [Unknown]:SH
        What is the two-letter country code for this unit?
          [Unknown]:CN
       Is CN=mysteel, OU= mysteel, O= mysteel, L=SH, ST=SH, C=CN correct?  
      [no]:  yes  
        Enter key password for < mysteel >
                (RETURN if same as keystore password):  mysteel      

[[email protected] extlib]$ pwd
/home/hadoop/app/azkaban/azkaban-web-2.5.0/extlib
[[email protected] extlib]$ keytool -keystore keystore -alias jetty -genkey -keyalg RSA
Enter keystore password:（密碼是mysteel）  
Re-enter new password: （密碼是mysteel）
What is your first and last name?
  [Unknown]:  mysteel
What is the name of your organizational unit?
  [Unknown]:  mysteel
What is the name of your organization?
  [Unknown]:  mysteel
What is the name of your City or Locality?
  [Unknown]:  SH
What is the name of your State or Province?
  [Unknown]:  SH
What is the two-letter country code for this unit?
  [Unknown]:  CN
Is CN=mysteel, OU=mysteel, O=mysteel, L=SH, ST=SH, C=CN correct?
  [no]:  yes

Enter key password for <jetty>
    (RETURN if same as keystore password):  
Re-enter new password: （密碼是mysteel）
[[email protected] extlib]$
           

注意：這裡keystore的密碼為mysteel,key的密碼為mysteel可自行修改 最後将生成的keystore檔案放到 Azkaban安裝目錄中~/app/azkaban/azkaban-web-2.5.0/ （也可通過修改配置檔案改變其預設存放位址）。

再說一個坑，這裡一定要注意，這裡的密碼要和web下conf的properties的jetty配置的密碼一緻，為啥呢，否則報錯，io exception，具體原因沒有看源碼，有興趣的可以去git上看下azkaban的源碼哦。

[[email protected] extlib]$ pwd
/home/hadoop/app/azkaban/azkaban-web-2.5.0/extlib
[[email protected] extlib]$ ll
total 816
-rw-rw-r-- 1 hadoop hadoop   2211 May  1 14:39 keystore
-rw-r--r-- 1 hadoop hadoop 827942 Nov  3 09:37 mysql-connector-java-5.1.21.jar
[[email protected] extlib]$ mv keystore /home/hadoop/app/azkaban/azkaban-web-2.5.0/
[[email protected] extlib]$ cd /home/hadoop/app/azkaban/azkaban-web-2.5.0/
[[email protected] azkaban-web-2.5.0]$ pwd
/home/hadoop/app/azkaban/azkaban-web-2.5.0
[[email protected] azkaban-web-2.5.0]$ ll
total 32
-rw-r--r-- 1 hadoop hadoop  105 Apr 22  2014 azkaban.version
drwxr-xr-x 2 hadoop hadoop 4096 Apr 22  2014 bin
drwxr-xr-x 2 hadoop hadoop 4096 May  1 14:29 conf
drwxr-xr-x 2 hadoop hadoop 4096 May  1 14:43 extlib
-rw-rw-r-- 1 hadoop hadoop 2211 May  1 14:39 keystore
drwxr-xr-x 2 hadoop hadoop 4096 May  1 14:29 lib
drwxr-xr-x 2 hadoop hadoop 4096 Apr 22  2014 plugins
drwxr-xr-x 6 hadoop hadoop 4096 May  1 14:29 web
[[email protected] azkaban-web-2.5.0]$
           

 4.修改配置檔案 (conf/azkaban.propreties)

修改首頁樣式及時區資訊

azkaban.name=Job Tasks
azkaban.label=mysteel workflow
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
default.timezone.id=Asia/Shanghai      

#Azkaban Personalization Settings
azkaban.name=Test
azkaban.label=My Local Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
default.timezone.id=America/Los_Angeles



改為

#Azkaban Personalization Settings
azkaban.name=Job Tasks
azkaban.label=mysteel workflow
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
default.timezone.id=Asia/Shanghai
           

修改mysql連接配接,可自行修改

database.type=mysql
mysql.port=3306
mysql.host=192.168.80.145
mysql.database=azkaban
mysql.user=azkaban
mysql.password=azkaban
mysql.numconnections=100      

database.type=mysql

mysql.port=3306

mysql.host=localhost

mysql.database=azkaban

mysql.user=azkaban

mysql.password=azkaban

mysql.numconnections=100

改為

database.type=mysql

mysql.port=3306

mysql.host=192.168.80.145

mysql.database=azkaban

mysql.user=azkaban

mysql.password=azkaban

mysql.numconnections=100

這裡一定要注意哦，一定要注意，密碼都要修改，不然就會報錯哦。

jetty.maxThreads=25
jetty.ssl.port=8443
jetty.port=8081
jetty.keystore=keystore
jetty.password=mysteel
jetty.keypassword= mysteel
jetty.truststore=keystore
jetty.trustpassword= mysteel      

# Azkaban Jetty server properties.
jetty.maxThreads=25
jetty.ssl.port=8443
jetty.port=8081
jetty.keystore=keystore
jetty.password=password
jetty.keypassword=password
jetty.truststore=keystore
jetty.trustpassword=password



改成


# Azkaban Jetty server properties.
jetty.maxThreads=25
jetty.ssl.port=8443
jetty.port=8071
jetty.keystore=keystore
jetty.password=mysteel
jetty.keypassword=mysteel
jetty.truststore=keystore
jetty.trustpassword=mysteel
           

 修改郵件設定（可選）

[email protected]
    mail.host=smtp.126.com
    mail.user= [email protected]
    mail.password=xxx (注:發送郵箱的密碼)      

5.使用者及權限配置

　　進入 azkaban web 伺服器 conf 目錄，修改 azkaban-users.xml ，增加管理者使用者：

< azkaban-users>
   < user username="azkaban" password="azkaban" roles="admin"  groups="azkaban" />
   < user username="metrics" password="metrics" roles="metrics"/>
   < user username="admin" password="admin" roles="admin,metrics" />
   < role name="admin" permissions="ADMIN" />
   < role name="metrics" permissions="METRICS"/>
< /azkaban-users>      

<azkaban-users>
        <user username="azkaban" password="azkaban" roles="admin" groups="azkaban" />
        <user username="metrics" password="metrics" roles="metrics"/>    
        <role name="admin" permissions="ADMIN" />
        <role name="metrics" permissions="METRICS"/>
</azkaban-users>

改為

<azkaban-users>
        <user username="azkaban" password="azkaban" roles="admin" groups="azkaban" />
        <user username="metrics" password="metrics" roles="metrics"/>
        <user username="admin" password="admin" roles="admin,metrics"/>  
        <role name="admin" permissions="ADMIN" />
        <role name="metrics" permissions="METRICS"/>
</azkaban-users>
           

四、Azkaban Executor Server安裝

1、下載下傳安裝包

wget https://s3.amazonaws.com/azkaban2/azkaban2/2.5.0/azkaban-executor-server-2.5.0.tar.gz
tar –zxvf  azkaban-executor-server-2.5.0.tar.gz  –C  ~/app/azkaban/      

　　我這裡，選擇的是離線，再上傳的方式

[[email protected] azkaban]$ pwd
/home/hadoop/app/azkaban
[[email protected] azkaban]$ ll
total 8
drwxrwxr-x 2 hadoop hadoop 4096 May  1 12:04 azkaban-2.5.0
drwxrwxr-x 8 hadoop hadoop 4096 May  1 14:43 azkaban-web-2.5.0
[[email protected] azkaban]$ rz

[[email protected] azkaban]$ ll
total 10904
drwxrwxr-x 2 hadoop hadoop     4096 May  1 12:04 azkaban-2.5.0
-rw-r--r-- 1 hadoop hadoop 11157302 May  1 11:44 azkaban-executor-server-2.5.0.tar.gz
drwxrwxr-x 8 hadoop hadoop     4096 May  1 14:43 azkaban-web-2.5.0
[[email protected] azkaban]$ clear
[[email protected] azkaban]$
           

[[email protected] azkaban]$ pwd
/home/hadoop/app/azkaban
[[email protected] azkaban]$ ll
total 10904
drwxrwxr-x 2 hadoop hadoop     4096 May  1 12:04 azkaban-2.5.0
-rw-r--r-- 1 hadoop hadoop 11157302 May  1 11:44 azkaban-executor-server-2.5.0.tar.gz
drwxrwxr-x 8 hadoop hadoop     4096 May  1 14:43 azkaban-web-2.5.0
[[email protected] azkaban]$ tar -zxvf azkaban-executor-server-2.5.0.tar.gz
           

[[email protected] azkaban]$ ll
total 10908
drwxrwxr-x 2 hadoop hadoop     4096 May  1 12:04 azkaban-2.5.0
drwxrwxr-x 7 hadoop hadoop     4096 May  1 15:21 azkaban-executor-2.5.0
-rw-r--r-- 1 hadoop hadoop 11157302 May  1 11:44 azkaban-executor-server-2.5.0.tar.gz
drwxrwxr-x 8 hadoop hadoop     4096 May  1 14:43 azkaban-web-2.5.0
[[email protected] azkaban]$ rm azkaban-executor-server-2.5.0.tar.gz
[[email protected] azkaban]$
           

2.修改配置檔案 (conf/azkaban.propreties)

修改時區資訊

default.timezone.id=Asia/Shanghai      

[[email protected] conf]$ pwd
/home/hadoop/app/azkaban/azkaban-executor-2.5.0/conf
[[email protected] conf]$ ll
total 8
-rw-r--r-- 1 hadoop hadoop  55 Apr 22  2014 azkaban.private.properties
-rw-r--r-- 1 hadoop hadoop 469 Apr 22  2014 azkaban.properties
-rw-r--r-- 1 hadoop hadoop   0 Apr 22  2014 global.properties
[[email protected] conf]$ vim azkaban.properties
           

#Azkaban
default.timezone.id=America/Los_Angeles

改為

#Azkaban
default.timezone.id=Asia/Shanghai
           

 修改mysql連接配接,可自行修改

database.type=mysql
mysql.port=3306
mysql.host=192.168.80.145
mysql.database=azkaban
mysql.user=azkaban
mysql.password=azkaban
mysql.numconnections=100      

database.type=mysql
mysql.port=3306
mysql.host=localhost
mysql.database=azkaban2
mysql.user=azkaban
mysql.password=azkaban
mysql.numconnections=100



改為


database.type=mysql
mysql.port=3306
mysql.host=192.168.80.145
mysql.database=azkaban
mysql.user=azkaban
mysql.password=azkaban
mysql.numconnections=100
           

配置端口，預設就好

executor.maxThreads=50
executor.port=12321
executor.flow.threads=30      

　　在web server 的azkaban.properties配置executor的連接配接，特别注意，如果兩個server不是在一台主機上，web server 需要配置executor.host

# Azkaban Executor settings
executor.port=12321
executor.host=ip/hostname      

　　我這裡，僅僅是都安裝在單節點master即同一台機器上，是以上述這一小步驟省略。

五、啟動驗證

啟動web server

~/app/azkaban/azkaban-web-2.5.0/bin/azkaban-web-start.sh（azkaban-web-shutdown.sh停止）
 通路 https://master:8443   出現界面安裝成功
或
      

通路 https://192.168.80.145:8443   出現界面安裝成功      

[[email protected] azkaban-web-2.5.0]$ pwd
/home/hadoop/app/azkaban/azkaban-web-2.5.0
[[email protected] azkaban-web-2.5.0]$ ll
total 32
-rw-r--r-- 1 hadoop hadoop  105 Apr 22  2014 azkaban.version
drwxr-xr-x 2 hadoop hadoop 4096 Apr 22  2014 bin
drwxr-xr-x 2 hadoop hadoop 4096 May  1 15:27 conf
drwxr-xr-x 2 hadoop hadoop 4096 May  1 14:43 extlib
-rw-rw-r-- 1 hadoop hadoop 2211 May  1 14:39 keystore
drwxr-xr-x 2 hadoop hadoop 4096 May  1 14:29 lib
drwxr-xr-x 2 hadoop hadoop 4096 Apr 22  2014 plugins
drwxr-xr-x 6 hadoop hadoop 4096 May  1 14:29 web
[[email protected] azkaban-web-2.5.0]$ bin/azkaban-web-start.sh
Using Hadoop from /home/hadoop/app/hadoop-2.6.0
Using Hive from /home/hadoop/app/hive
bin/..
/home/hadoop/app/jdk/lib:.:/home/hadoop/app/jdk/lib:/home/hadoop/app/jdk/jre/lib:bin/../lib/azkaban-2.5.0.jar:bin/../lib/commons-collections-3.2.1.jar:bin/../lib/commons-configuration-1.8.jar:bin/../lib/commons-dbcp-1.4.jar:bin/../lib/commons-dbutils-1.5.jar:bin/../lib/commons-email-1.2.jar:bin/../lib/commons-fileupload-1.2.1.jar:bin/../lib/commons-io-2.4.jar:bin/../lib/commons-jexl-2.1.1.jar:bin/../lib/commons-lang-2.6.jar:bin/../lib/commons-logging-1.1.1.jar:bin/../lib/commons-pool-1.6.jar:bin/../lib/guava-13.0.1.jar:bin/../lib/h2-1.3.170.jar:bin/../lib/httpclient-4.2.1.jar:bin/../lib/httpcore-4.2.1.jar:bin/../lib/jackson-core-asl-1.9.5.jar:bin/../lib/jackson-mapper-asl-1.9.5.jar:bin/../lib/jetty-6.1.26.jar:bin/../lib/jetty-util-6.1.26.jar:bin/../lib/joda-time-2.0.jar:bin/../lib/jopt-simple-4.3.jar:bin/../lib/junit-4.8.jar:bin/../lib/log4j-1.2.16.jar:bin/../lib/mail-1.4.5.jar:bin/../lib/mysql-connector-java-5.1.28.jar:bin/../lib/servlet-api-2.5.jar:bin/../lib/slf4j-api-1.6.1.jar:bin/../lib/slf4j-log4j12-1.6.4.jar:bin/../lib/velocity-1.7.jar:bin/../lib/velocity-tools-2.0.jar:bin/../extlib/mysql-connector-java-5.1.21.jar:bin/../plugins/*/*.jar:/home/hadoop/app/hadoop-2.6.0/conf:/home/hadoop/app/hadoop-2.6.0/*:/home/hadoop/app/hive/conf:/home/hadoop/app/hive/lib/*
[[email protected] azkaban-web-2.5.0]$ 2017/05/01 15:35:30.167 +0800 ERROR [AzkabanWebServer] [Azkaban] Starting Jetty Azkaban Executor...
2017/05/01 15:35:30.278 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban settings file from bin/../conf
2017/05/01 15:35:30.280 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban properties file
2017/05/01 15:35:30.338 +0800 INFO [AzkabanWebServer] [Azkaban] Setting up Jetty Https Server with port:8443 and numThreads:25
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/app/azkaban/azkaban-web-2.5.0/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/app/hive-1.0.0/lib/hive-jdbc-1.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

2017/05/01 15:35:30.532 +0800 INFO [log] [Azkaban] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2017/05/01 15:35:30.897 +0800 INFO [AzkabanWebServer] [Azkaban] Loading user manager class azkaban.user.XmlUserManager
2017/05/01 15:35:31.352 +0800 INFO [XmlUserManager] [Azkaban] Loading user azkaban
2017/05/01 15:35:31.359 +0800 INFO [XmlUserManager] [Azkaban] Loading user metrics
2017/05/01 15:35:31.360 +0800 INFO [XmlUserManager] [Azkaban] Loading user admin
2017/05/01 15:35:33.429 +0800 INFO [AzkabanWebServer] [Azkaban] Loading JDBC for project management
2017/05/01 15:35:33.450 +0800 INFO [ExecutorManager] [Azkaban] Cleaning old logs from execution_logs
2017/05/01 15:35:33.480 +0800 INFO [ProjectManager] [Azkaban] Project version retention is set to 3
2017/05/01 15:35:34.233 +0800 INFO [ExecutorManager] [Azkaban] Cleaning old log files before 2017-02-06T15:35:33.601+08:00
2017/05/01 15:35:34.283 +0800 INFO [ExecutorManager] [Azkaban] Cleaned up 0 log entries.
2017/05/01 15:35:34.483 +0800 INFO [TriggerManager] [Azkaban] TriggerManager loaded.
2017/05/01 15:35:34.484 +0800 INFO [AzkabanWebServer] [Azkaban] Loading built-in checker and action types
2017/05/01 15:35:34.533 +0800 INFO [CheckerTypeLoader] [Azkaban] Registering checker BasicTimeChecker
2017/05/01 15:35:34.533 +0800 INFO [CheckerTypeLoader] [Azkaban] Registering checker SlaChecker
2017/05/01 15:35:34.534 +0800 INFO [CheckerTypeLoader] [Azkaban] Registering checker ExecutionChecker
2017/05/01 15:35:34.534 +0800 INFO [ActionTypeLoader] [Azkaban] Registering action ExecuteFlowAction
2017/05/01 15:35:34.535 +0800 INFO [ActionTypeLoader] [Azkaban] Registering action KillExecutionAction
2017/05/01 15:35:34.550 +0800 INFO [ActionTypeLoader] [Azkaban] Registering action AlertAction
2017/05/01 15:35:34.566 +0800 INFO [ActionTypeLoader] [Azkaban] Registering action CreateTriggerAction
2017/05/01 15:35:34.582 +0800 INFO [AzkabanWebServer] [Azkaban] Loading trigger based scheduler
2017/05/01 15:35:34.620 +0800 INFO [AzkabanWebServer] [Azkaban] Loading plug-in checker and action types
2017/05/01 15:35:34.621 +0800 ERROR [AzkabanWebServer] [Azkaban] plugin path plugins/triggers doesn't exist!
2017/05/01 15:35:34.623 +0800 INFO [AzkabanWebServer] [Azkaban] Setting timezone to Asia/Shanghai
2017/05/01 15:35:34.627 +0800 INFO [AzkabanWebServer] [Azkaban] Registering MBeans...
2017/05/01 15:35:34.747 +0800 INFO [AzkabanWebServer] [Azkaban] Bean azkaban.jmx.JmxJettyServer registered.
2017/05/01 15:35:34.755 +0800 INFO [AzkabanWebServer] [Azkaban] Bean azkaban.jmx.JmxTriggerManager registered.
2017/05/01 15:35:34.765 +0800 INFO [AzkabanWebServer] [Azkaban] Bean azkaban.jmx.JmxExecutorManager registered.
2017/05/01 15:35:34.770 +0800 INFO [AzkabanWebServer] [Azkaban] Setting up web resource dir web/
2017/05/01 15:35:34.952 +0800 INFO [JdbcTriggerLoader] [Azkaban] Loading all triggers from db.
2017/05/01 15:35:35.047 +0800 INFO [JdbcTriggerLoader] [Azkaban] Loaded 0 triggers.
2017/05/01 15:35:35.048 +0800 INFO [log] [Azkaban] jetty-6.1.26
2017/05/01 15:35:36.445 +0800 INFO [log] [Azkaban] Started [email protected]:8443
2017/05/01 15:35:36.446 +0800 INFO [AzkabanWebServer] [Azkaban] Server running on ssl port 8443.
           

啟動 executor server

~/app/azkaban/azkaban-executor-2.5.0/bin/azkaban-executor-start.sh(azkaban-executor-shutdown.sh 停止      

[[email protected] azkaban-executor-2.5.0]$ pwd
/home/hadoop/app/azkaban/azkaban-executor-2.5.0
[[email protected] azkaban-executor-2.5.0]$ ll
total 24
-rw-r--r-- 1 hadoop hadoop  105 Apr 22  2014 azkaban.version
drwxr-xr-x 2 hadoop hadoop 4096 Apr 22  2014 bin
drwxr-xr-x 2 hadoop hadoop 4096 May  1 15:34 conf
drwxr-xr-x 2 hadoop hadoop 4096 Apr 22  2014 extlib
drwxr-xr-x 2 hadoop hadoop 4096 May  1 15:21 lib
drwxr-xr-x 2 hadoop hadoop 4096 Apr 22  2014 plugins
[[email protected] azkaban-executor-2.5.0]$ bin/azkaban-executor-start.sh
Using Hadoop from /home/hadoop/app/hadoop-2.6.0
Using Hive from /home/hadoop/app/hive
bin/..
/home/hadoop/app/jdk/lib:.:/home/hadoop/app/jdk/lib:/home/hadoop/app/jdk/jre/lib:bin/../lib/azkaban-2.5.0.jar:bin/../lib/commons-collections-3.2.1.jar:bin/../lib/commons-configuration-1.8.jar:bin/../lib/commons-dbcp-1.4.jar:bin/../lib/commons-dbutils-1.5.jar:bin/../lib/commons-email-1.2.jar:bin/../lib/commons-fileupload-1.2.1.jar:bin/../lib/commons-io-2.4.jar:bin/../lib/commons-jexl-2.1.1.jar:bin/../lib/commons-lang-2.6.jar:bin/../lib/commons-logging-1.1.1.jar:bin/../lib/commons-pool-1.6.jar:bin/../lib/guava-13.0.1.jar:bin/../lib/h2-1.3.170.jar:bin/../lib/httpclient-4.2.1.jar:bin/../lib/httpcore-4.2.1.jar:bin/../lib/jackson-core-asl-1.9.5.jar:bin/../lib/jackson-mapper-asl-1.9.5.jar:bin/../lib/jetty-6.1.26.jar:bin/../lib/jetty-util-6.1.26.jar:bin/../lib/joda-time-2.0.jar:bin/../lib/jopt-simple-4.3.jar:bin/../lib/junit-4.8.jar:bin/../lib/log4j-1.2.16.jar:bin/../lib/mail-1.4.5.jar:bin/../lib/mysql-connector-java-5.1.28.jar:bin/../lib/servlet-api-2.5.jar:bin/../lib/slf4j-api-1.6.1.jar:bin/../lib/slf4j-log4j12-1.6.4.jar:bin/../lib/velocity-1.7.jar:bin/../lib/velocity-tools-2.0.jar:bin/../extlib/*.jar:bin/../plugins/*/*.jar:/home/hadoop/app/hadoop-2.6.0/conf:/home/hadoop/app/hadoop-2.6.0/*:/home/hadoop/app/hive/conf:/home/hadoop/app/hive/lib/*
Starting AzkabanExecutorServer on port 12321 ...
[[email protected] azkaban-executor-2.5.0]$ 2017/05/01 15:43:52.101 +0800 ERROR [AzkabanExecutorServer] [Azkaban] Starting Jetty Azkaban Executor...
2017/05/01 15:43:52.180 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban settings file from bin/../conf
2017/05/01 15:43:52.182 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban private properties file
2017/05/01 15:43:52.187 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban properties file
2017/05/01 15:43:52.285 +0800 INFO [AzkabanExecutorServer] [Azkaban] Setting timezone to Asia/Shanghai
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/app/azkaban/azkaban-executor-2.5.0/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/app/hive-1.0.0/lib/hive-jdbc-1.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2017/05/01 15:43:52.430 +0800 INFO [log] [Azkaban] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2017/05/01 15:43:53.334 +0800 INFO [FlowRunnerManager] [Azkaban] Execution dir retention set to 86400000 ms
2017/05/01 15:43:53.388 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning recently finished
2017/05/01 15:43:53.389 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning old projects
2017/05/01 15:43:53.390 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning old execution dirs
2017/05/01 15:43:53.431 +0800 INFO [AzkabanExecutorServer] [Azkaban] Registering MBeans...
2017/05/01 15:43:53.610 +0800 INFO [AzkabanExecutorServer] [Azkaban] Bean azkaban.jmx.JmxJettyServer registered.
2017/05/01 15:43:53.639 +0800 INFO [AzkabanExecutorServer] [Azkaban] Bean azkaban.jmx.JmxFlowRunnerManager registered.
2017/05/01 15:43:53.640 +0800 INFO [log] [Azkaban] jetty-6.1.26
2017/05/01 15:43:53.861 +0800 INFO [log] [Azkaban] Started [email protected]:12321
2017/05/01 15:43:53.862 +0800 INFO [AzkabanExecutorServer] [Azkaban] Azkaban Executor Server started on port 12321
           

 　　如何停止

[[email protected] azkaban-web-2.5.0]$ pwd
/home/hadoop/app/azkaban/azkaban-web-2.5.0
[[email protected] azkaban-web-2.5.0]$ jps
5676 AzkabanWebServer
5866 AzkabanExecutorServer
6026 Jps      

[[email protected] azkaban-executor-2.5.0]$ pwd
/home/hadoop/app/azkaban/azkaban-executor-2.5.0
[[email protected] azkaban-executor-2.5.0]$ ll
total 44
-rw-rw-r-- 1 hadoop hadoop    0 May  1 15:43 azkaban-access.log
-rw-r--r-- 1 hadoop hadoop  105 Apr 22  2014 azkaban.version
-rw-rw-r-- 1 hadoop hadoop  302 May  1 15:43 azkaban-webserver.log
drwxr-xr-x 2 hadoop hadoop 4096 Apr 22  2014 bin
drwxr-xr-x 2 hadoop hadoop 4096 May  1 15:34 conf
-rw-rw-r-- 1 hadoop hadoop    5 May  1 15:43 currentpid
drwxrwxr-x 2 hadoop hadoop 4096 May  1 15:43 executions
drwxr-xr-x 2 hadoop hadoop 4096 Apr 22  2014 extlib
drwxr-xr-x 2 hadoop hadoop 4096 May  1 15:21 lib
drwxr-xr-x 2 hadoop hadoop 4096 Apr 22  2014 plugins
drwxrwxr-x 2 hadoop hadoop 4096 May  1 15:43 projects
drwxrwxr-x 2 hadoop hadoop 4096 May  1 15:43 temp
[[email protected] azkaban-executor-2.5.0]$ bin/azkaban-executor-shutdown.sh



[[email protected] azkaban-executor-2.5.0]$ jps
5676 AzkabanWebServer
6111 Jps
[[email protected] azkaban-executor-2.5.0]$       

[[email protected] azkaban-web-2.5.0]$ pwd
/home/hadoop/app/azkaban/azkaban-web-2.5.0
[[email protected] azkaban-web-2.5.0]$ jps
5676 AzkabanWebServer
6087 Jps
[[email protected] azkaban-web-2.5.0]$ bin/azkaban-web-shutdown.sh      

　　　　或kill掉

[[email protected] azkaban-web-2.5.0]$ jps
6128 Jps
5676 AzkabanWebServer
[[email protected] azkaban-web-2.5.0]$ kill -9 5676
[[email protected] azkaban-web-2.5.0]$ jps
6138 Jps
[[email protected] azkaban-web-2.5.0]$       

　　這裡不多贅述了。

六、Azkaban web server插件安裝（暫時沒用上，不先寫了）

　　為了後邊插件的安裝，需要在${AZKABAN_WEB_SERVER}/plugins/目錄下建立viewer目錄，用來安裝各種viewer插件。當安裝好一個插件後需要重新開機服務，讓插件生效。

安裝HDFS Viewer插件

 1.解壓插件

　　解壓插件 在viewer目錄中解壓azkaban-hdfs-viewer-2.5.0.tar.gz，得到azkaban-hdfs-viewer-2.5.0目錄，将其更名為hdfs。最終，這個插件的目錄路徑是：${AZKABAN_WEB_SERVER}/plugins/viewer/hdfs。

安裝Job Summary插件

安裝Reportal插件

七、Azkaban Executor Server 插件安裝（暫時沒用上，不先寫了）

 　　進入${AZKABAN_EXECUTOR_SERVER}/plugins目錄，在該目錄下解壓azkaban-jobtype-2.5.0.tar.gz得到目錄：azkaban-jobtype-2.5.0。将該目錄更名為jobtypes。這個目錄用來存放之後安裝的所有插件，與Web Server中的viewer目錄類似。每次插件的安裝配置需要重新開機Executor Server。

安裝Reportalhive插件

八.azkaban日常運維

啟動

1. 啟動web服務:

cdaw  #cd /home/zybigdata/azkaban/web/current的别名 bin /azkaban-web-start .sh

2. 啟動executor：

cdae  #cd /home/zybigdata/azkaban/exe/current的别名 bin /azkaban-executor-start .sh

關閉

1. 關閉web服務:

cdaw  #cd /home/zybigdata/azkaban/web/current的别名 bin /azkaban-web-shutdown .sh

2. 關閉executor：

cdae  #cd /home/zybigdata/azkaban/exe/current的别名 bin /azkaban-executor-shutdown .sh

 9.使用者管理

Azkaban通過解析 /home/zy_dw/azkaban/web/current/conf/azkaban-users.xml來實作權限管理

eg. 添加使用者名jiaqi.guo

<user username="jiaqi.guo" password="password" roles="admin" groups="azkaban" />

借鑒此文：

http://www.cnblogs.com/zlslch/p/6296283.html