天天看點
傳回

Mysql8.0.18的源碼安裝一、環境準備二、安裝配置三、問題集合四、總結五、參考

Mysql的8.0版本出來已經有一段時間了，最新版本已經到了8.0.18，近期研究下，第一步就是安裝部署，以下是源碼安裝過程。

一、環境準備

OS：CentOS 7.4

DB：Mysql 8.0.18

硬體：40G以上空間，4G以上記憶體

二、安裝配置

1、下載下傳檔案

下載下傳mysql-boost-8.0.18.tar.gz

2、安裝依賴包 

# yum -y install gcc gcc-c++ ncurses-devel openssl-devel cmake3 bison wget bzip2
# ln -s /usr/bin/cmake3 /usr/bin/cmake

3、使用者組設定 

useradd mysql
groupadd mysql
mkdir -p /usr/local/mysql
mkdir -p /data/mysql8.0/log
chown -R mysql:mysql /data/mysql8.0/
chown -R mysql:mysql /usr/local/mysql

4、系統參數設定 

echo -e "
kernel.sem = 250 32000 100128
kernel.sysrq = 0
net.core.rmem_max = 33554432
net.core.wmem_max = 33554432
net.core.rmem_default = 262144
net.core.wmem_default = 262144
net.core.somaxconn = 8192
net.core.netdev_max_backlog = 16384
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.all.secure_redirects = 0
net.ipv4.conf.default.secure_redirects = 0
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
net.ipv4.tcp_rmem = 409665536 16777216
net.ipv4.tcp_wmem = 409632768 16777216
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_max_syn_backlog = 8192
net.nf_conntrack_max = 1048576
net.netfilter.nf_conntrack_tcp_timeout_established = 12000
vm.swappiness = 0
vm.zone_reclaim_mode = 0" >> /etc/sysctl.conf

sysctl –p

sed -i '/^GRUB_CMDLINE_LINUX/ s/\"$/  numa=off\"/g' /etc/default/grub
grub2-mkconfig -o /etc/grub2.cfg
--需要reboot
--檢視dmesg | grep -i numa或cat /proc/cmdline

limit="\n*\tsoft\tnofile\t65536\n*\thard\tnofile\t65536\n*\tsoft\tnproc\t131072\n*\thard\tnproc\t131072\n*\tsoft\tstack\tunlimited\n*\thard\tstack\tunlimited"
echo -e "$limit" >> /etc/security/limits.conf
echo -e "$limit" >> /etc/security/limits.d/20-nproc.conf

5、編譯安裝 

cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mysql  -DCMAKE_CXX_COMPILER=/usr/bin/g++ -DMYSQL_DATADIR=/data/mysql8.0/datanode1 -DMYSQL_UNIX_ADDR=/data/mysql8.0/mysql.sock -DWITH_BOOST=/usr/local/boost -DSYSCONFDIR=/usr/local/mysql -DFORCE_INSOURCE_BUILD=1 -DWITH_SSL=system -DMYSQL_TCP_PORT=3306 -DENABLED_LOCAL_INFILE=1 -DDEFAULT_CHARSET=utf8mb4 -DDEFAULT_COLLATION=utf8mb4_general_ci -DWITH_EXTRA_CHARSETS=all -DFORCE_INSOURCE_BUILD=1 -DCOMPILATION_COMMENT="Kenyon's MySQL 8.0"
make && make install

--PS：可以提前下載下傳boost_1_70_0.tar.gz放在/usr/local/boost下面,或者使用-DDOWNLOAD_BOOST=1 -DWITH_BOOST=/usr/local/boost下載下傳

6、配置參數 

port = 3306
socket = /data/mysql8.0/datanode1/mysql.sock
default-character-set = utf8mb4

[mysqld]
server-id = 99999            ## 适當調整，可以選取ipv4位址
port = 3306
user = mysql
basedir = /usr/local/mysql
datadir =/data/mysql8.0/datanode1
tmpdir = /tmp
socket = /data/mysql8.0/datanode1/mysql.sock
pid-file = /data/mysql8.0/datanode1/mysql.pid
default-storage-engine = INNODB
lower_case_table_names = 1
transaction_isolation = READ-COMMITTED
secure_file_priv =

# connection
back_log = 400
max_connections = 2000
interactive_timeout = 28800
wait_timeout = 28800
slave_net_timeout = 300
skip_name_resolve = on

# binlog
log-bin = mysql-bin
log-bin-index = mysql-bin
relay-log = relay-bin
relay-log-index = relay-bin
expire_logs_days = 7
binlog_cache_size = 8M
binlog_format = ROW
log_slave_updates = 1
relay_log_purge = 1
log-bin-trust-function-creators = 1
#slave_parallel_workers = 4

# caches & limits
bulk_insert_buffer_size = 8M
innodb_buffer_pool_size = 18G    ## 适當調整:記憶體60%
innodb_log_buffer_size = 32M
innodb_sort_buffer_size = 4M
join_buffer_size = 4M
sort_buffer_size = 4M
key_buffer_size = 16M
innodb_max_dirty_pages_pct = 15
max_allowed_packet = 16M
max_heap_table_size = 16M
read_buffer_size = 512K
read_rnd_buffer_size = 512K
open-files-limit = 10240
table_definition_cache = 4096
table_open_cache = 4096
table_open_cache_instances = 16

# thread
thread_cache_size = 300
innodb_thread_concurrency = 0
innodb_io_capacity = 500
innodb_io_capacity_max = 1000

# innodb log
innodb_flush_method = O_DIRECT
innodb_log_file_size = 1G
innodb_autoextend_increment = 128
innodb_flush_log_at_trx_commit = 2
innodb_log_files_in_group = 3
innodb_log_group_home_dir = /data/mysql8.0/datanode1/
innodb_print_all_deadlocks
sync_binlog = 1000

# innodb file
innodb_data_home_dir = /data/mysql8.0/datanode1/
innodb_data_file_path = ibdata:2G:autoextend
innodb_file_per_table = 1
tmp_table_size = 4M

# data
innodb_autoinc_lock_mode = 1
auto_increment_increment = 3        ## 适當調整
auto_increment_offset = 1           ## 适當調整
max_prepared_stmt_count = 65530

# character
character-set-server = utf8mb4
character-set-client = utf8mb4

# logging
slow-query-log = 1
slow-query-log-file = /data/mysql8.0/log/mysql-slow.log
log-error = /data/mysql8.0/log/mysql-error.log
long_query_time    = 1

# Recommended in standard MySQL setup
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES

7、初始化 

/usr/local/mysql/bin/mysqld --initialize --basedir=/usr/local/mysql --datadir=/data/mysql8.0/datanode1  --lower-case-table-names=1 --user=mysql

8、設定自啟動 

# chkconfig --add mysqld
# chkconfig mysqld on

9、啟動關閉 

mysql> select version();
+-----------+
| version() |
+-----------+
| 8.0.18    |
+-----------+
1 row in set (0.00 sec)

三、問題集合

1、cmake版本過低 

[root@kenyon mysql-8.0.18]#  cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mysql -DMYSQL_DATADIR=/data/mysql8.0/data -DMYSQL_UNIX_ADDR=/data/mysql8.0/mysql.sock -DWITH_BOOST=/root/mysql-8.0.0-dmr/boost -DSYSCONFDIR=/usr/local/mysql -DWITH_SSL=system -DMYSQL_TCP_PORT=3306  -DENABLED_LOCAL_INFILE=1 -DENABLE_DTRACE=0 -DDEFAULT_CHARSET=utf8mb4 -DDEFAULT_COLLATION=utf8mb4_general_ci -DWITH_EXTRA_CHARSETS=all -DWITH_EMBEDDED_SERVER=1 -DCOMPILATION_COMMENT="Kenyon's MySQL 8.0"
-- Running cmake version 2.8.12.2
CMake Warning at CMakeLists.txt:43 (MESSAGE):
  Please use cmake3 rather than cmake on this platform


-- Please install cmake3 (yum install cmake3)
CMake Error at CMakeLists.txt:73 (CMAKE_MINIMUM_REQUIRED):
  CMake 3.5.1 or higher is required.  You are running version 2.8.12.2


-- Configuring incomplete, errors occurred!


--解決辦法是用删除老的cmake版本，用新的cmake3版本替換cmake2版本
rm -f /usr/bin/cmake2
rm -f /usr/bin/cmake

yum install cmake3
ln -s /usr/bin/cmake3 /usr/bin/cmake

2、強制源碼安裝 

-- MySQL 8.0.18
-- Source directory /root/software/mysql-8.0.18
-- Binary directory /root/software/mysql-8.0.18
CMake Error at CMakeLists.txt:341 (MESSAGE):
  Please do not build in-source.  Out-of source builds are highly
  recommended: you can have multiple builds for the same source, and there is
  an easy way to do cleanup, simply remove the build directory (note that
  'make clean' or 'make distclean' does *not* work)

  You *can* force in-source build by invoking cmake with
  -DFORCE_INSOURCE_BUILD=1

根據提示，在cmake編譯的時候加上-DFORCE_INSOURCE_BUILD=1參數

3、gcc版本過低 

-- Source directory /root/software/mysql-8.0.18
-- Binary directory /root/software/mysql-8.0.18
CMake Warning at CMakeLists.txt:339 (MESSAGE):
  This is an in-source build


-- CMAKE_GENERATOR: Unix Makefiles
CMake Error at cmake/os/Linux.cmake:59 (MESSAGE):
  GCC 5.3 or newer is required (-dumpversion says 4.8.5)
Call Stack (most recent call first):
  CMakeLists.txt:442 (INCLUDE)
解決辦法是更新gcc，這是個很漫長的過程

--直接安裝gcc可能會=會報錯，需要安裝gmp、mpfr、mpc，各元件前後有關系，按此順序執行
configure: error: Building GCC requires GMP 4.2+, MPFR 2.4.0+ and MPC 0.8.0+.
Try the --with-gmp, --with-mpfr and/or --with-mpc options to specify

wget ftp://ftp.gnu.org/gnu/mpc/mpc-1.1.0.tar.gz     -P /usr/local/src
wget ftp://ftp.gnu.org/gnu/gmp/gmp-6.1.2.tar.bz2    -P /usr/local/src
wget http://ftp.gnu.org/gnu/mpfr/mpfr-4.0.2.tar.gz  -P /usr/local/src
wget http://ftp.gnu.org/gnu/m4/m4-1.4.18.tar.gz     -P /usr/local/src
wget http://ftp.gnu.org/gnu/gcc/gcc-9.2.0/gcc-9.2.0.tar.gz -P /usr/local/src

tar -zxvf mpc-1.1.0.tar.gz
tar -zxvf mpfr-4.0.2.tar.gz
tar -jxvf gmp-6.1.2.tar.bz2
tar -zxvf m4-1.4.18.tar.gz
tar -zxvf gcc-9.2.0.tar.gz

--解壓gmp需要yum install bzip2 -y

mkdir -p /usr/local/m4-1.4.18
mkdir -p /usr/local/gmp-6.1.2
mkdir -p /usr/local/mpfr-4.0.2
mkdir -p /usr/local/mpc-1.1.0
mkdir -p /usr/local/gcc-9.2.0

cd m4-1.4.18
./configure --prefix=/usr/local/m4-1.4.18
make && make install
ln -s /usr/local/m4-1.4.18/bin/m4 /usr/bin/m4

cd  gmp-6.1.2
./configure --prefix=/usr/local/gmp-6.1.2 
make && make install

cd mpfr-4.0.2
./configure --prefix=/usr/local/mpfr-4.0.2/ --with-gmp=/usr/local/gmp-6.1.2/
make && make install

cd mpc-1.1.0
./configure --prefix=/usr/local/mpc-1.1.0 --with-gmp=/usr/local/gmp-6.1.2/ --with-mpfr=/usr/local/mpfr-4.0.2/
make && make install

[root@kenyon ]# more /etc/ld.so.conf
include ld.so.conf.d/*.conf

/usr/local/gmp-6.1.2/lib
/usr/local/mpfr-4.0.2/lib
/usr/local/mpc-1.1.0/lib

# ldconfig

cd gcc-9.2.0
./configure --prefix=/usr/local/gcc-9.2.0 --with-gmp=/usr/local/gmp-6.1.2  --with-mpfr=/usr/local/mpfr-4.0.2 --with-mpc=/usr/local/mpc-1.1.0 --disable-multilib 
make && make install
--編譯好久....3個小時左右....

mv /usr/bin/gcc /usr/bin/gcc4.8.5
mv /usr/bin/g++ /usr/bin/g++4.8.5

ln -s /usr/local/gcc-9.2.0/bin/gcc /usr/bin/gcc
ln -s /usr/local/gcc-9.2.0/bin/g++ /usr/bin/g++

4、編譯參數問題 

c++: error: unrecognized command line option ‘-std=c++14’

解決辦法是編譯時添加  -DCMAKE_CXX_COMPILER=/usr/bin/g++

5、動态庫問題 

```../runtime_output_directory/uca9dump: /lib64/libstdc++.so.6: version `CXXABI_1.3.9' not found (required by ../runtime_output_directory/uca9dump)
--原因是更新了gcc，卻沒有将更新後的gcc的動态庫去替換老版本的gcc動态庫所緻

解決辦法：
cp /usr/local/src/gcc-9.2.0/x86_64-pc-linux-gnu/libstdc++-v3/src/.libs/libstdc++.so.6.0.27 /usr/lib64
cd /usr/lib64
ln -s libstdc++.so.6.0.27 libstdc++.so.6

6、源碼syscall問題

在函數‘void buf_pool_create(buf_pool_t*, ulint, ulint, std::mutex*, dberr_t&)’中: /root/mysql-8.0.17/storage/innobase/buf/buf0buf.cc:1220:44: 錯誤：‘SYS_gettid’在此作用域中尚未聲明 setpriority(PRIO_PROCESS, (pid_t)syscall(SYS_gettid), -20);

解決辦法：在源檔案storage/innobase/buf/buf0buf.cc的開頭添加： #include "sys/syscall.h"

7、swap_lint問題 

/root/mysql-8.0.18/storage/innobase/lock/lock0lock.cc:1359:9: error: ‘os_compare_and_swap_thread_id’ was not declared in this scope; did you mean ‘os_compare_and_swap_uint64’

解決辦法：替換源檔案lock0lock.cc中os_compare_and_swap_thread_id為os_compare_and_swap_lint

b.
/root/mysql-8.0.18/storage/innobase/trx/trx0trx.cc: In function ‘void trx_init(trx_t*)’:
/root/mysql-8.0.18/storage/innobase/trx/trx0trx.cc:222:5: error: ‘os_compare_and_swap_thread_id’ was not declared in this scope; did you mean ‘os_compare_and_swap_lint’?
  222 |     os_compare_and_swap_thread_id(&trx->killed_by, thread_id, 0);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |     os_compare_and_swap_lint
/root/mysql-8.0.18/storage/innobase/trx/trx0trx.cc: In function ‘void trx_kill_blocking(trx_t*)’:
/root/mysql-8.0.18/storage/innobase/trx/trx0trx.cc:3230:5: error: ‘os_compare_and_swap_thread_id’ was not declared in this scope; did you mean ‘os_compare_and_swap_lint’?
 3230 |     os_compare_and_swap_thread_id(&victim_trx->killed_by, thread_id, 0);

解決辦法：
修改storage/innobase/trx/trx0trx.cc中替換os_compare_and_swap_thread_id為os_compare_and_swap_lint，注意有兩處修改

8、初始化失敗 

2019-12-02T11:31:16.776210+08:00 1 [ERROR] [MY-011011] [Server] Failed to find valid data directory.
2019-12-02T11:31:16.776500+08:00 0 [ERROR] [MY-010020] [Server] Data Dictionary initialization failed.
2019-12-02T11:31:16.776570+08:00 0 [ERROR] [MY-010119] [Server] Aborting
2019-12-02T11:31:16.777890+08:00 0 [System] [MY-010910] [Server] /usr/local/mysql/bin/mysqld: Shutdown complete (mysqld 8.0.18)  Kenyon's MySQL 8.0.18

解決辦法：
在以上問題都處理掉後，重新cmake、make和make install

四、總結

1、mysql8版本源碼安裝較以前的7版本要求更高，尤其是編譯器的要求更高，編譯時間也更長

2、以前低版本（5.7以前）可以定制化各種引擎和初始化參數，新版本已經預設安裝，不再需要編譯該類參數

3、官方目前不推薦進行源碼編譯，從時間成本上來看，建議還是選擇rpm安裝，也可以自己編譯完成後打包rpm自行安裝

4、目前發現的一個坑是想設定大小寫敏感需要在初始化資料庫時就要指定（initialize --lower-case-table-names=1）,否則隻能導出重新初始化

5、初始化完成後密碼會列印在螢幕日志末尾，首次登陸必須先改密碼才能使用

五、參考

https://www.twblogs.net/a/5d7351bebd9eee5327ff7d28/zh-cn http://www.zhongweicheng.com/?p=2503 https://www.mysql.com https://www.cnblogs.com/NanZhiHan/p/11017158.html
javascript 安全 關系型資料庫 mysql 編譯器 資料庫 c語言 資料安全/隐私保護 c++ perl mysql源碼安裝安裝問題 mysql源碼安裝 安裝mysql源碼 源碼編譯安裝mysql mysql源碼安裝安裝問題安全
上一篇: 阿裡雲Link ID² 安全領域再拿殊榮
下一篇: 阿裡技術 2019 年度熱文 Top10 出爐，你錯過哪些？| 開發者必讀（128期）

繼續閱讀