hadoop安裝

1.硬體準備

2.安裝

2.1安裝jdk

1)jdk下載下傳，在官網下載下傳linux 64位的

 下載下傳到本地上傳，或者使用wget下載下傳

curl -O https://download.oracle.com/otn/java/jdk/8u261-b12/a4634525489241b9a9e1aa73d9e118e6/jdk-8u261-linux-x64.tar.gz?AuthParam=1600227784_d1d21d93eaccab801bd1324fb5089179
           

　　解壓後配置環境變量

vi /etc/profile

JAVA_HOME=/opt/modules/jdk
PATH=$PATH:$JAVA_HOME/bin

export JAVA_HOME PATH

source /etc/profile

[[email protected] jdk]# which java
/opt/modules/jdk/bin/java
           

2)配置host互通

[[email protected] jdk]# cat /etc/hosts
127.0.0.1	localhost	localhost.localdomain	localhost4	localhost4.localdomain4
::1	localhost	localhost.localdomain	localhost6	localhost6.localdomain6
172.24.67.124	hadoop-001	hadoop-001
39.100.128.158	hadoop-002	hadoop-002
39.100.146.104	hadoop-003	hadoop-003

[[email protected] jdk]# ping hadoop-003
PING hadoop-003 (39.100.146.104) 56(84) bytes of data.
64 bytes from hadoop-003 (39.100.146.104): icmp_seq=1 ttl=64 time=0.266 ms
64 bytes from hadoop-003 (39.100.146.104): icmp_seq=2 ttl=64 time=0.203 ms
           

　3)建立使用者

[[email protected] jdk]# useradd hadoop
[[email protected] jdk]# passwd hadoop
Changing password for user hadoop.
New password: 
BAD PASSWORD: it is based on a dictionary word
BAD PASSWORD: is too simple
Retype new password: 
passwd: all authentication tokens updated successfully.
[[email protected] jdk]# chown -R hadoop:hadoop /opt/modules/
[[email protected] jdk]# ll /opt/
total 8
drwxr-xr-x 3 hadoop hadoop 4096 Sep 16 14:04 modules
drwxr-xr-x 2 root   root   4096 Sep 16 11:35 soft
           

　授予sudo權限

[[email protected] jdk]# vi /etc/sudoers
 
 90 ## Allow root to run any commands anywhere
     91 root    ALL=(ALL)       ALL
     92 hadoop  ALL=(ALL)       NOPASSWD: ALL

[[email protected] jdk]# su - hadoop
[[email protected] ~]$ sudo mkdir /opt/test
[[email protected] ~]$ ll /opt/
total 12
drwxr-xr-x 3 hadoop hadoop 4096 Sep 16 14:04 modules
drwxr-xr-x 2 root   root   4096 Sep 16 11:35 soft
drwxr-xr-x 2 root   root   4096 Sep 16 14:23 test
           

[[email protected] soft]$ sudo chown -R hadoop:hadoop /opt/soft/
[[email protected] soft]$ ll /opt/
total 12
drwxr-xr-x 3 hadoop hadoop 4096 Sep 16 14:04 modules
drwxr-xr-x 2 hadoop hadoop 4096 Sep 16 14:34 soft
drwxr-xr-x 2 root   root   4096 Sep 16 14:23 test
           

2.2 安裝hadoop　　

　1)上傳和解壓

[[email protected] soft]$ tar -zxvf hadoop-2.7.2.tar.gz -C ../modules/

[[email protected] hadoop-2.7.2]$ sudo vi /etc/profile

HADOOP_HOME=/opt/modules/hadoop-2.7.2
PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin

export JAVA_HOME PATH HADOOP_HOME


[[email protected] hadoop-2.7.2]$ source /etc/profile

[[email protected] hadoop-2.7.2]$ hadoop version
Hadoop 2.7.2
Subversion Unknown -r Unknown
Compiled by root on 2017-05-22T10:49Z
Compiled with protoc 2.5.0
From source with checksum d0fda26633fa762bff87ec759ebe689c
This command was run using /opt/modules/hadoop-2.7.2/share/hadoop/common/hadoop-common-2.7.2.jar
           

2.3.叢集安裝

　1)三台機子免密登入(001免密登入002和003)　

[[email protected] ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hadoop/.ssh/id_rsa): 
Created directory '/home/hadoop/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/hadoop/.ssh/id_rsa.
Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
The key fingerprint is:
aa:ad:73:ca:f5:ab:a9:a6:99:a4:29:93:28:84:f4:39 [email protected]
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|                 |
|                 |
| .               |
|o . .   S        |
|.. E   .         |
|o.. . o          |
|*+ +o+.o         |
|=.+o**+.o.       |
+-----------------+
[[email protected] ~]$ ssh-copy-id hadoop-001
The authenticity of host 'hadoop-001 (172.24.67.124)' can't be established.
RSA key fingerprint is ae:64:82:22:d7:6c:cb:ab:5d:d8:04:8c:fe:16:a7:ea.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hadoop-001,172.24.67.124' (RSA) to the list of known hosts.
[email protected]'s password: 
Now try logging into the machine, with "ssh 'hadoop-001'", and check in:

  .ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

[[email protected] ~]$ ssh-copy-id hadoop-002
The authenticity of host 'hadoop-002 (39.100.128.158)' can't be established.
RSA key fingerprint is ea:6f:0d:c6:43:18:36:34:e7:d3:c7:2b:c2:3a:17:5b.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hadoop-002,39.100.128.158' (RSA) to the list of known hosts.
[email protected]'s password: 
Now try logging into the machine, with "ssh 'hadoop-002'", and check in:

  .ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

[[email protected] ~]$ ssh-copy-id hadoop-003
The authenticity of host 'hadoop-003 (39.100.146.104)' can't be established.
RSA key fingerprint is ae:64:82:22:d7:6c:cb:ab:5d:d8:04:8c:fe:16:a7:ea.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hadoop-003,39.100.146.104' (RSA) to the list of known hosts.
Now try logging into the machine, with "ssh 'hadoop-003'", and check in:

  .ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

[[email protected] ~]$ ssh hadoop-002 ls  /opt