1 Docker Hub
#Docker Hub被用于源代碼管理內建，也用于建構和測試工具來加速部署周期，部署周期從天減少到以分鐘計算，Docker宣稱已經允許使用者加速應用的傳輸。
#架構和技術堆棧更新對于Docker Hub的大規模和不可預知的采用是必須的。
#作為開發人員迅速采用Docker容器，IT管理者加速将其納入企業生産環境。除了管理和配置工具，還需要編排和排程軟體。
#當然有些内網環境需要通路Docker Hub，這時候就通路不了，這時就需要搭建一套内網DockerHub，官方有推出docker-registry工具，可以用于建構私有的鏡像倉庫，實作簡單。
2 環境準備
[[email protected] ~]# cat /etc/redhat-release
CentOS Linux release 7.8.2003 (Core)
[[email protected] ~]# ifconfig eth1
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.16.2.250 netmask 255.255.255.0 broadcast 172.16.2.255
2.1 docker安裝
#安裝必要的一些系統工具
yum install -y yum-utils device-mapper-persistent-data lvm2
#添加軟體源資訊
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum makecache fast
#檢視可用 docker 版本
yum list docker-ce.x86_64 --showduplicates | sort -r
#安裝docker-ce，也可以根據上面羅列的各版本使用指定版本安裝
yum -y install docker-ce
#開啟Docker服務
systemctl enable docker
systemctl start docker
2.2 docker加速
#添加加速檔案配置，這裡選用了阿裡雲的配置
#vim /etc/docker/daemon.json
#重新加載服務，重新開機docker，并檢視是否生效
systemctl daemon-reload
systemctl restart docker
docker info
3 私有倉庫搭建
3.1 檢視registry的版本
[[email protected] ~]# docker search registry
NAME DESCRIPTION STARS OFFICIAL AUTOMATED
registry The Docker Registry 2.0 implementation for s… 3062 [OK]
distribution/registry WARNING: NOT the registry official image!!! … 57 [OK]
stefanscherer/registry-windows Containerized docker registry for Windows Se… 32
budry/registry-arm Docker registry build for Raspberry PI 2 and… 18
deis/registry Docker image registry for the Deis open sour… 12
jc21/registry-ui A nice web interface for managing your Docke… 12
anoxis/registry-cli You can list and delete tags from your priva… 10 [OK]
sixeyed/registry Docker Registry 2.6.0 running on Windows - N… 10
pallet/registry-swift Add swift storage support to the official do… 4 [OK]
allingeek/registry A specialization of registry:2 configured fo… 4 [OK]
arm32v6/registry The Docker Registry 2.0 implementation for s… 3
goharbor/registry-photon 2
conjurinc/registry-oauth-server Docker registry authn/authz server backed by… 1
concourse/registry-image-resource 1
ibmcom/registry Docker Image for IBM Cloud private-CE (Commu… 1
metadata/registry Metadata Registry is a tool which helps you … 1 [OK]
webhippie/registry Docker images for Registry 1 [OK]
kontena/registry Kontena Registry 0
gisjedi/registry-proxy Reverse proxy of registry mirror image gisje… 0
dwpdigital/registry-image-resource Concourse resource type 0
lorieri/registry-ceph Ceph Rados Gateway (and any other S3 compati… 0
convox/registry 0
digillect/registry-cleaner Tool to remove unused images from Docker reg… 0 [OK]
pivnet/registry-gcloud-image 0
upmcenterprises/registry-creds
3.2 安裝registry
[[email protected] ~]# docker pull registry
Using default tag: latest
latest: Pulling from library/registry
cbdbe7a5bc2a: Pull complete
47112e65547d: Pull complete
46bcb632e506: Pull complete
c1cc712bcecd: Pull complete
3db6272dcbfa: Pull complete
Digest: sha256:8be26f81ffea54106bae012c6f349df70f4d5e7e2ec01b143c46e2c03b9e551d
Status: Downloaded newer image for registry:latest
docker.io/library/registry:latest
[[email protected] ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
registry latest 2d4f4b5309b1 2 months ago 26.2MB
#建立本地目錄，用于映射本地目錄進docker裡，目的是，當容器崩潰時，資料還在，随時以啟用一個新容器替換
mkdir -p /data/dockerhub
#啟動registry
docker run -d -v /data/dockerhub:/var/lib/registry -p 5000:5000 --restart=always --name dockerhub-registry2.0 registry
#通路網址http://172.16.1.250:5000/v2，如果出現以下頁面說明正常
[[email protected] ~]# curl http://172.16.1.250:5000/v2/
{}
3.3 上傳鏡像至私有倉庫
[[email protected] ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
nginx latest 4bb46517cac3 8 days ago 133MB
registry latest 2d4f4b5309b1 2 months ago 26.2MB
#将要推送至私有倉庫的docker鏡像做辨別
[[email protected] ~]# docker tag nginx:latest 172.16.1.250:5000/nginx:latest
[[email protected] ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
nginx latest 4bb46517cac3 8 days ago 133MB
172.16.1.250:5000/nginx latest 4bb46517cac3 8 days ago 133MB
registry latest 2d4f4b5309b1 2 months ago 26.2MB
#通過 docker push 指令将 nginx 鏡像 push到私有倉庫
[[email protected] ~]# docker push 172.16.1.250:5000/nginx:latest
The push refers to repository [172.16.1.250:5000/nginx]
550333325e31: Pushed
22ea89b1a816: Pushed
a4d893caa5c9: Pushed
0338db614b95: Pushed
d0f104dc0a1f: Pushed
latest: digest: sha256:179412c42fe3336e7cdc253ad4a2e03d32f50e3037a860cf5edbeb1aaddb915c size: 1362
#檢視是否上傳成功
[[email protected] ~]# curl http://127.0.0.1:5000/v2/_catalog
{"repositories":["nginx"]}
#檢視鏡像資訊
[[email protected] ~]# curl http://172.16.1.250:5000/v2/nginx/tags/list
{"name":"nginx","tags":["latest"]}
#從其他内網機器驗證拉取鏡像
[[email protected] ~]# docker pull 172.16.1.250:5000/nginx
Using default tag: latest
latest: Pulling from nginx
bf5952930446: Pull complete
cb9a6de05e5a: Pull complete
9513ea0afb93: Pull complete
b49ea07d2e93: Pull complete
a5e4a503d449: Pull complete
Digest: sha256:179412c42fe3336e7cdc253ad4a2e03d32f50e3037a860cf5edbeb1aaddb915c
Status: Downloaded newer image for 172.16.1.250:5000/nginx:latest
172.16.1.250:5000/nginx:latest
[[email protected] ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
172.16.1.250:5000/nginx latest 4bb46517cac3 8 days ago 133MB
4 harbor
harbor項目 github位址
#作為一個企業級私有 Registry 伺服器，Harbor 提供了更好的性能和安全。提升使用者使用 Registry 建構和運作環境傳輸鏡像的效率。Harbor 支援安裝在多個 Registry 節點的鏡像資源複制，鏡像全部儲存在私有 Registry 中， 確定資料和知識産權在公司内部網絡中管控。另外，Harbor 也提供了進階的安全特性，諸如使用者管理，通路控制和活動審計等。
#基于角色的通路控制 - 使用者與 Docker 鏡像倉庫通過“項目”進行組織管理，一個使用者可以對多個鏡像倉庫在同一命名空間（project）裡有不同的權限。
- 鏡像複制 - 鏡像可以在多個 Registry 執行個體中複制（同步）。尤其适合于負載均衡，高可用，混合雲和多雲的場景。
- 圖形化使用者界面 - 使用者可以通過浏覽器來浏覽，檢索目前 Docker 鏡像倉庫，管理項目和命名空間。
- AD/LDAP 支援 - Harbor 可以內建企業内部已有的 AD/LDAP，用于鑒權認證管理。
- 審計管理 - 所有針對鏡像倉庫的操作都可以被記錄追溯，用于審計管理。
- 國際化 - 已擁有英文、中文、德文、日文和俄文的本地化版本。更多的語言将會添加進來。
- RESTful API - RESTful API 提供給管理者對于 Harbor 更多的操控, 使得與其它管理軟體內建變得更容易。
- 部署簡單 - 提供線上和離線兩種安裝工具， 也可以安裝到 vSphere 平台(OVA 方式)虛拟裝置。
4.1 下載下傳并安裝harbor
#==harbor項目有基本要求：docker 17.06.0-ce+、golang : 1.12.0+ 和 docker-compose 1.18.0+ ==
[[email protected] ~]# yum install golang docker-ce docker-compose
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* elrepo: mirrors.tuna.tsinghua.edu.cn
* extras: mirrors.aliyun.com
* updates: mirrors.aliyun.com
Package golang-1.13.14-1.el7.x86_64 already installed and latest version
Package 3:docker-ce-19.03.12-3.el7.x86_64 already installed and latest version
Package docker-compose-1.18.0-4.el7.noarch already installed and latest version
#這裡直接選用了下載下傳最新版本的harbor離線安裝包
#下載下傳
[[email protected] ~]# wget https://github.com/goharbor/harbor/releases/download/v2.0.2/harbor-offline-installer-v2.0.2.tgz
#解壓
[[email protected] ~]# tar zxvf harbor-offline-installer-v2.0.2.tgz
#harbor預設工作方式是http，但是這隻能在頁面通路，預設harbor推送拉取鏡像時走的是https，是以需要配置下https
#生成秘鑰和自簽名證書
openssl req -newkey rsa:4096 -nodes -sha256 -keyout ca.key -x509 -days 365 -out ca.crt
#生成證書簽名請求（域名通路，就把common name的值寫為域名）：
openssl req -newkey rsa:4096 -nodes -sha256 -keyout harbor.kkcai.vip.key -out harbor.kkcai.vip.csr
#生成伺服器證書
openssl x509 -req -days 365 -in harbor.kkcai.vip.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out harbor.kkcai.vip.crt
[[email protected] harbor]# ll
total 535264
-rw-r--r-- 1 root root 2045 Aug 22 15:32 ca.crt
-rw-r--r-- 1 root root 3272 Aug 22 15:32 ca.key
-rw-r--r-- 1 root root 17 Aug 22 15:36 ca.srl
-rw-r--r-- 1 root root 3361 Jul 28 17:47 common.sh
-rw-r--r-- 1 root root 1948 Aug 22 15:36 harbor.kkcai.vip.crt
-rw-r--r-- 1 root root 1797 Aug 22 15:34 harbor.kkcai.vip.csr
-rw-r--r-- 1 root root 3272 Aug 22 15:34 harbor.kkcai.vip.key
-rw-r--r-- 1 root root 548041010 Jul 28 17:47 harbor.v2.0.2.tar.gz
-rw-r--r-- 1 root root 7829 Aug 22 12:00 harbor.yml
-rw-r--r-- 1 root root 7828 Jul 28 17:47 harbor.yml.tmpl
-rwxr-xr-x 1 root root 2523 Jul 28 17:47 install.sh
-rw-r--r-- 1 root root 11347 Jul 28 17:47 LICENSE
-rwxr-xr-x 1 root root 1856 Jul 28 17:47 prepare
#安裝證書
[[email protected] harbor]# mkdir -p /etc/cert/harbor
[[email protected] harbor]# cp harbor.kkcai.vip.crt harbor.kkcai.vip.key /etc/cert/harbor
[[email protected] harbor]# ll /etc/cert/harbor/
total 8
-rw-r--r-- 1 root root 1948 Aug 22 15:38 harbor.kkcai.vip.crt
-rw-r--r-- 1 root root 3272 Aug 22 15:38 harbor.kkcai.vip.key
#由于我内網有搭建了DNS伺服器，是以沒有設定hosts，如果内網沒有搭建DNS的同學，請vim /etc/hosts 給域名添加個路由位址
#快捷添加hosts
echo "172.16.1.250 harbor.kkcai.vip" >>/etc/hosts
#進到目錄複制預設檔案并修改配置檔案harbor.yml
[[email protected] harbor]# cp harbor.yml.tmpl harbor.yml
[[email protected] harbor]# vim harbor.yml
# DO NOT use localhost or 127.0.0.1, because Harbor needs to be accessed by external clients.
#建議使用域名
hostname: harbor.kkcai.vip
#修改證書位置
certificate: /etc/cert/harbor/harbor.kkcai.vip.crt
private_key: /etc/cert/harbor/harbor.kkcai.vip.key
#harbor密碼
harbor_admin_password: devops
# Harbor資料庫密碼
database:
# The password for the root user of Harbor DB. Change this before any production use.
password: devops
#資料存儲位置
data_volume: /data/harbor
#運作install.sh進行安裝
[[email protected] harbor]# sh install.sh
[Step 0]: checking if docker is installed ...
Note: docker version: 19.03.12
[Step 1]: checking docker-compose is installed ...
Note: docker-compose version: 1.18.0
[Step 2]: loading Harbor images ...
Loaded image: goharbor/prepare:v2.0.2
Loaded image: goharbor/harbor-jobservice:v2.0.2
Loaded image: goharbor/harbor-registryctl:v2.0.2
Loaded image: goharbor/registry-photon:v2.0.2
Loaded image: goharbor/harbor-core:v2.0.2
Loaded image: goharbor/notary-signer-photon:v2.0.2
Loaded image: goharbor/clair-photon:v2.0.2
Loaded image: goharbor/trivy-adapter-photon:v2.0.2
Loaded image: goharbor/harbor-log:v2.0.2
Loaded image: goharbor/nginx-photon:v2.0.2
Loaded image: goharbor/clair-adapter-photon:v2.0.2
Loaded image: goharbor/chartmuseum-photon:v2.0.2
Loaded image: goharbor/harbor-portal:v2.0.2
Loaded image: goharbor/harbor-db:v2.0.2
Loaded image: goharbor/redis-photon:v2.0.2
Loaded image: goharbor/notary-server-photon:v2.0.2
[Step 3]: preparing environment ...
[Step 4]: preparing harbor configs ...
prepare base dir is set to /root/harbor
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/registryctl/config.yml
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /data/secret/keys/secretkey
Creating harbor-log ... done
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
Creating harbor-db ... done
Creating harbor-core ... done
Creating network "harbor_harbor" with the default driver
Creating nginx ... done
Creating redis ...
Creating harbor-db ...
Creating registryctl ...
Creating harbor-portal ...
Creating registry ...
Creating harbor-core ...
Creating nginx ...
Creating harbor-jobservice ...
✔ ----Harbor has been installed and started successfully.----
#常用指令
4.2 驗證harbor是否安裝成功
#通路網址：https://harbor.kkcai.vip/
#預設賬号是 admin 密碼 devops
4.3 驗證harbor是否能夠正常推送鏡像
#建立docker證書存放位置，并複制ca證書
[[email protected] harbor]# mkdir -p /etc/docker/certs.d/harbor.kkcai.vip
[[email protected] harbor]# cp ca.crt /etc/docker/certs.d/harbor.kkcai.vip
#docker登入harbor
[[email protected] harbor]# docker login harbor.kkcai.vip
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
#推送鏡像至對應的項目中
[[email protected] harbor]# docker tag nginx:latest harbor.kkcai.vip/library/nginx:latest
[[email protected] harbor]# docker push harbor.kkcai.vip/library/nginx:latest
The push refers to repository [harbor.kkcai.vip/library/nginx]
550333325e31: Pushed
22ea89b1a816: Pushed
a4d893caa5c9: Pushed
0338db614b95: Pushed
d0f104dc0a1f: Pushed
latest: digest: sha256:179412c42fe3336e7cdc253ad4a2e03d32f50e3037a860cf5edbeb1aaddb915c size: 1362
#檢視harbor web頁面，已經推送成功
4.4 添加第三方倉庫
#系統管理-倉庫管理-建立目标
#提供者選擇：Docker Registry
#目标名：test
#目标URL：http://172.16.1.250:5000 （為前面建立的私有倉庫，前面建立的為不鑒權的方式，是以不需要填寫使用者名密碼）
#确定即可
X.遇到的問題
X.1 WARNING: bridge-nf-call-iptables is disabled
#問題表現
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled
#原因：由于網橋工作于資料鍊路層，在iptables沒有開啟 bridge-nf時,資料會直接經過網橋轉發，結果就是對FORWARD的設定失效；
#解決方案：編輯配置，添加如下配置儲存并重新加載
#vim /etc/sysctl.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
#sysctl -p
X.2 Get http: server gave HTTP response to HTTPS client
#問題表現
[[email protected] ~]# docker push 172.16.1.250:5000/nginx:latest
The push refers to repository [172.16.1.250:5000/nginx]
Get https://172.16.1.250:5000/v2/: http: server gave HTTP response to HTTPS client
#原因：docker registry未采用https服務，而用戶端docker使用https請求push所緻
#解決方案：編輯配置，添加 "insecure-registries":["172.16.1.250:5000"] 配置儲存并重新加載
#vim /etc/docker/daemon.json
{"registry-mirrors": ["https://registry.cn-hangzhou.aliyuncs.com"], "insecure-registries":["172.16.1.250:5000"]}
#systemctl restart docker