探索Linux核心：Kconfig/kbuild的秘密-Exploring the Linux kernel: The secrets of Kconfig/kbuild

FORM：https://www.sohu.com/a/272455844_610730

英文原版FROM：https://opensource.com/article/18/10/kbuild-and-kconfig

深入了解Linux配置/建構系統的工作原理。

自從Linux核心代碼遷移到Git以來，Linux核心配置/建構系統（也稱為Kconfig /kbuild）已經存在了很長時間。然而，作為支撐基礎設施，它很少成為人們關注的焦點；甚至在日常工作中使用它的核心開發人員也從未真正過它。

為了探索如何編譯Linux核心，本文将深入研究Kconfig/kbuild内部過程，解釋如何生成.config檔案和vmlinux/bzImage檔案，并介紹依賴性跟蹤的智能技巧。

Kconfig

建構核心的第一步始終是Kconfig，Kconfig有助于使Linux核心高度子產品化和可定制。 Kconfig為使用者提供了許多配置目标：

config	使用基于行的程式更新目前配置
nconfig	使用ncurses基于菜單的程式更新目前配置
menuconfig	使用基于菜單的程式更新目前配置
xconfig	使用基于Qt的前端更新目前配置
gconfig	使用基于GTK +的前端更新目前配置
oldconfig	使用提供的.config作為基礎更新目前配置
localmodconfig	更新目前配置禁用未加載的子產品
localyesconfig	更新目前配置，将本地mod轉換到核心
defconfig	預設來自Arch提供的defconfig的新配置
savedefconfig	将目前配置儲存為./defconfig（最小配置）
allnoconfig	新配置，其中所有選項均以“否”回答
allyesconfig	新配置，其中所有選項都被'yes'接受
allmodconfig	盡可能新配置選擇子產品
alldefconfig	新配置，所有符号都設定為預設值
randconfig	新配置，随機回答所有選項
listnewconfig	列出新選項
olddefconfig	與oldconfig相同，但在沒有提示的情況下将新符号設定為其預設值
kvmconfig	為KVM虛拟機核心支援啟用其他選項
xenconfig	為xen dom0和虛拟機核心支援啟用其他選項
tinyconfig	配置最小的核心

我認為menuconfig是這些選項中最受歡迎的。目标由不同的主程式處理，這些程式由核心提供并在核心建構期間建構。一些目标有一個GUI（為了友善使用者），而大多數沒有。與Kconfig相關的工具和源代碼主要位于核心源代碼中的s/kconfig /下。正如我們從s/kconfig/Makefile中看到的，有幾個主機程式，包括conf，mconf和nconf。除了conf之外，它們中的每一個都負責基于GUI的配置目标之一，是以，conf處理它們中的大多數。

從邏輯上講，Kconfig的基礎結構有兩部分：一部分實作一種新語言來定義配置項（參見核心源代碼下的Kconfig檔案），另一部分解析Kconfig語言并處理配置操作。

大多數配置目标具有大緻相同的内部過程（如下所示）：

請注意，所有配置項都具有預設值。

第一步是讀取源根目錄下的Kconfig檔案，建構初始配置資料庫; 然後它通過根據此優先級讀取現有配置檔案來更新初始資料庫：

.config

/lib/modules/$(shell,uname -r)/.config

/etc/kernel-config

/boot/config-$(shell,uname -r)

ARCH_DEFCONFIG

arch/$(ARCH)/defconfig

如果你通過menuconfig進行基于GUI的配置或通過oldconfig進行基于指令行的配置，則會根據你的自定義更新資料庫。最後，将配置資料庫轉儲到.config檔案中。

但.config檔案不是核心建構的最終配置；這就是syncconfig目标存在的原因。syncconfig曾經是一個名為silentoldconfig的配置選項，但它不會執行舊名稱所說的内容，是以它已重命名。此外，因為它是供内部使用（不适用于使用者），是以它已從清單中删除。

以下是syncconfig的作用：

syncconfig将.config作為輸入并輸出許多其他檔案，這些檔案分為三類：

• auto.conf和tristate.conf用于makefile文本處理。 例如，可以在元件的makefile中看到這樣的語句：

obj-$(CONFIG_GENERIC_CALIBRATE_DELAY) += calibrate.o

• autoconf.h用于C語言源檔案。
• include / config /下的空頭檔案用于kbuild期間的配置依賴性跟蹤，如下所述。

配置完成後，我們将知道哪些檔案和代碼片段未編譯。

kbuild

元件式建構（稱為遞歸make）是GNU make管理大型項目的常用方法。Kbuild是遞歸make的一個很好的例子。通過将源檔案劃分為不同的子產品/元件，每個元件都由其自己的makefile管理。當開始建構時，頂級makefile以正确的順序調用每個元件的makefile，建構元件，并将它們收集到最終的執行程式中。

Kbuild指的是不同類型的makefile：

• Makefile是位于源根目錄中的頂級makefile。
• .config是核心配置檔案。
• arch / $（ARCH）/Makefile是arch makefile，它是top makefile的補充。
• s/Makefile。*描述了所有kbuild makefile的通用規則。
• 最後，大約有500個kbuild makefile。

top makefile包含arch makefile，讀取.config檔案，下載下傳到子目錄，在s/Makefile。*中定義的例程的幫助下，在每個元件的makefile上調用make，建構每個中間對象，并将所有中間對象連結到vmlinux中。核心文檔Documentation/kbuild/makefiles.txt描述了這些makefile的所有方面。

作為示例，讓我們看看如何在x86-64上生成vmlinux：

（該插圖基于Richard Y. Steven的部落格。它已更新，并在作者允許的情況下使用。）

進入vmlinux的所有.o檔案首先進入他們自己的内置.a，通過變量KBUILD_VMLINUX_INIT，KBUILD_VMLINUX_MAIN，KBUILD_VMLINUX_LIBS訓示，然後收集到vmlinux檔案中。

在簡化的makefile代碼的幫助下，了解如何在Linux核心中實作遞歸make：

# In top Makefile

vmlinux: s/link-vmlinux.sh $(vmlinux-deps)

+$(call if_changed,link-vmlinux)

# Variable assignments

vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN) $(KBUILD_VMLINUX_LIBS)

export KBUILD_VMLINUX_INIT := $(head-y) $(init-y)

export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y2) $(drivers-y) $(net-y) $(virt-y)

export KBUILD_VMLINUX_LIBS := $(libs-y1)

export KBUILD_LDS := arch/$(SRCARCH)/kernel/vmlinux.lds

init-y := init/

drivers-y := drivers/ sound/ firmware/

net-y := net/

libs-y := lib/

core-y := usr/

virt-y := virt/

# Transform to corresponding built-in.a

init-y := $(patsubst %/, %/built-in.a, $(init-y))

core-y := $(patsubst %/, %/built-in.a, $(core-y))

drivers-y := $(patsubst %/, %/built-in.a, $(drivers-y))

net-y := $(patsubst %/, %/built-in.a, $(net-y))

libs-y1 := $(patsubst %/, %/lib.a, $(libs-y))

libs-y2 := $(patsubst %/, %/built-in.a, $(filter-out %.a, $(libs-y)))

virt-y := $(patsubst %/, %/built-in.a, $(virt-y))

# Setup the dependency. vmlinux-deps are all intermediate objects, vmlinux-dirs

# are phony targets, so every time comes to this rule, the recipe of vmlinux-dirs

# will be executed. Refer "4.6 Phony Targets" of `info make`

$(sort $(vmlinux-deps)): $(vmlinux-dirs) ;

# Variable vmlinux-dirs is the directory part of each built-in.a

vmlinux-dirs := $(patsubst %/,%,$(filter %/, $(init-y) $(init-m)

$(core-y) $(core-m) $(drivers-y) $(drivers-m)

$(net-y) $(net-m) $(libs-y) $(libs-m) $(virt-y)))

# The entry of recursive make

$(vmlinux-dirs):

$(Q)$(MAKE) $(build)[email protected] need-builtin=1

例如，擴充了遞歸制作配方

make -f s/Makefile.build obj=init need-builtin=1

這意味着make将進入s/Makefile.build繼續建構每個内置的工作.a。 在s/link-vmlinux.sh的幫助下，vmlinux檔案最終位于源根目錄下。

了解vmlinux與bzImage

許多Linux核心開發人員可能不清楚vmlinux和bzImage之間的關系。例如，這是他們在x86-64中的關系：

根vmlinux被剝離，壓縮，放入piggy.S，然後與其他對等對象連結到arch/x86/boot/compressed/vmlinux。同時，在arch / x86 / boot下生成一個名為setup.bin的檔案。可能存在具有重定位資訊的可選第三檔案，具體取決于CONFIG_X86_NEED_RELOCS的配置。

由核心提供的稱為build的宿主程式将這兩個（或三個）部分建構到最終的bzImage檔案中。

依賴性跟蹤

Kbuild跟蹤三種依賴關系：

第一個很容易了解，但第二個和第三個呢？ 核心開發人員經常會看到如下代碼：

#ifdef CONFIG_SMP

__boot_cpu_id = cpu;

#endif

當CONFIG_SMP更改時，應該重新編譯這段代碼。 編譯源檔案的指令行也很重要，因為不同的指令行可能會導緻不同的目标檔案。

當.c檔案通過#include指令使用頭檔案時，您需要編寫如下規則：

main.o: defs.h

recipe...

管理大型項目時，需要大量的這些規則；把它們全部寫下來會很乏味和乏味。幸運的是，大多數現代C編譯器都可以通過檢視源檔案中的#include行來編寫這些規則。對于GNU編譯器集合（GCC），隻需添加指令行參數：-MD depfile

# In s/Makefile.lib

c_flags = -Wp,-MD,$(depfile) $(NOSTDINC_FLAGS) $(LINUXINCLUDE)

-include $(srctree)/include/linux/compiler_types.h

$(__c_flags) $(modkern_cflags)

$(basename_flags) $(modname_flags)

這将生成一個.d檔案，内容如下：

init_task.o: init/init_task.c include/linux/kconfig.h

include/generated/autoconf.h include/linux/init_task.h

include/linux/rcupdate.h include/linux/types.h

...

然後主機程式fixdep通過将depfile和指令行作為輸入來處理其他兩個依賴項，然後以makefile文法輸出。<target> .cmd檔案，該檔案記錄指令行和所有先決條件（包括配置） 為目标。 它看起來像這樣：

# The command line used to compile the target

cmd_init/init_task.o := gcc -Wp,-MD,init/.init_task.o.d -nostdinc ...

...

# The dependency files

deps_init/init_task.o :=

$(wildcard include/config/posix/timers.h)

$(wildcard include/config/arch/task/struct/on/stack.h)

$(wildcard include/config/thread/info/in/task.h)

...

include/uapi/linux/types.h

arch/x86/include/uapi/asm/types.h

include/uapi/asm-generic/types.h

...

在遞歸make中将包含一個。<target> .cmd檔案，提供所有依賴關系資訊并幫助決定是否重建目标。

這背後的秘密是fixdep将解析depfile（.d檔案），然後解析内部的所有依賴檔案，搜尋所有CONFIG_字元串的文本，将它們轉換為相應的空頭檔案，并将它們添加到目标的先決條件。每次配置更改時，相應的空頭檔案也将更新，是以kbuild可以檢測到該更改并重建依賴于它的目标。 因為還記錄了指令行，是以很容易比較最後和目前的編譯參數。

展望未來

Kconfig/kbuild在很長一段時間内保持不變，直到新的維護者Masahiro Yamada于2017年初加入，現在kbuild再次正在積極開發。 如果你很快就會看到與本文中的内容不同的内容，請不要感到驚訝。

英文原版

FROM：https://opensource.com/article/18/10/kbuild-and-kconfig

The Linux kernel config/build system, also known as Kconfig/kbuild, has been around for a long time, ever since the Linux kernel code migrated to Git. As supporting infrastructure, however, it is seldom in the spotlight; even kernel developers who use it in their daily work never really think about it.

To explore how the Linux kernel is compiled, this article will dive into the Kconfig/kbuild internal process, explain how the .config file and the vmlinux/bzImage files are produced, and introduce a smart trick for dependency tracking.

Kconfig

The first step in building a kernel is always configuration. Kconfig helps make the Linux kernel highly modular and customizable. Kconfig offers the user many config targets:

config	Update current config utilizing a line-oriented program
nconfig	Update current config utilizing a ncurses menu-based program
menuconfig	Update current config utilizing a menu-based program
xconfig	Update current config utilizing a Qt-based frontend
gconfig	Update current config utilizing a GTK+ based frontend
oldconfig	Update current config utilizing a provided .config as base
localmodconfig	Update current config disabling modules not loaded
localyesconfig	Update current config converting local mods to core
defconfig	New config with default from Arch-supplied defconfig
savedefconfig	Save current config as ./defconfig (minimal config)
allnoconfig	New config where all options are answered with 'no'
allyesconfig	New config where all options are accepted with 'yes'
allmodconfig	New config selecting modules when possible
alldefconfig	New config with all symbols set to default
randconfig	New config with a random answer to all options
listnewconfig	List new options
olddefconfig	Same as oldconfig but sets new symbols to their default value without prompting
kvmconfig	Enable additional options for KVM guest kernel support
xenconfig	Enable additional options for xen dom0 and guest kernel support
tinyconfig	Configure the tiniest possible kernel

I think menuconfig is the most popular of these targets. The targets are processed by different host programs, which are provided by the kernel and built during kernel building. Some targets have a GUI (for the user's convenience) while most don't. Kconfig-related tools and source code reside mainly under scripts/kconfig/ in the kernel source. As we can see from scripts/kconfig/Makefile, there are several host programs, including conf, mconf, and nconf. Except for conf, each of them is responsible for one of the GUI-based config targets, so, conf deals with most of them.

Logically, Kconfig's infrastructure has two parts: one implements a new language to define the configuration items (see the Kconfig files under the kernel source), and the other parses the Kconfig language and deals with configuration actions.

Most of the config targets have roughly the same internal process (shown below):

kconfig_process.png

The Linux Terminal

• Top 7 terminal emulators for Linux
• 10 command-line tools for data analysis in Linux
• Download Now: SSH cheat sheet
• Advanced Linux commands cheat sheet
• Linux command line tutorials

Note that all configuration items have a default value.

The first step reads the Kconfig file under source root to construct an initial configuration database; then it updates the initial database by reading an existing configuration file according to this priority:

.config

/lib/modules/$(shell,uname -r)/.config

/etc/kernel-config

/boot/config-$(shell,uname -r)

ARCH_DEFCONFIG

arch/$(ARCH)/defconfig

If you are doing GUI-based configuration via menuconfig or command-line-based configuration via oldconfig, the database is updated according to your customization. Finally, the configuration database is dumped into the .config file.

But the .config file is not the final fodder for kernel building; this is why the syncconfigtarget exists. syncconfig used to be a config target called silentoldconfig, but it doesn't do what the old name says, so it was renamed. Also, because it is for internal use (not for users), it was dropped from the list.

Here is an illustration of what syncconfig does:

syncconfig.png

syncconfig takes .config as input and outputs many other files, which fall into three categories:

• auto.conf & tristate.conf are used for makefile text processing. For example, you may see statements like this in a component's makefile: 

  obj-$(CONFIG_GENERIC_CALIBRATE_DELAY) += calibrate.o
             

• autoconf.h is used in C-language source files.
• Empty header files under include/config/ are used for configuration-dependency tracking during kbuild, which is explained below.

After configuration, we will know which files and code pieces are not compiled.

kbuild

Component-wise building, called recursive make, is a common way for GNU 

make

 to manage a large project. Kbuild is a good example of recursive make. By dividing source files into different modules/components, each component is managed by its own makefile. When you start building, a top makefile invokes each component's makefile in the proper order, builds the components, and collects them into the final executive.

Kbuild refers to different kinds of makefiles:

• Makefile is the top makefile located in source root.
• .config is the kernel configuration file.
• arch/$(ARCH)/Makefile is the arch makefile, which is the supplement to the top makefile.
• scripts/Makefile.* describes common rules for all kbuild makefiles.
• Finally, there are about 500 kbuild makefiles.

The top makefile includes the arch makefile, reads the .config file, descends into subdirectories, invokes make on each component's makefile with the help of routines defined in scripts/Makefile.*, builds up each intermediate object, and links all the intermediate objects into vmlinux. Kernel document Documentation/kbuild/makefiles.txtdescribes all aspects of these makefiles.

As an example, let's look at how vmlinux is produced on x86-64:

vmlinux_generation_process.png

(The illustration is based on Richard Y. Steven's blog. It was updated and is used with the author's permission.)

All the .o files that go into vmlinux first go into their own built-in.a, which is indicated via variables KBUILD_VMLINUX_INIT, KBUILD_VMLINUX_MAIN, KBUILD_VMLINUX_LIBS, then are collected into the vmlinux file.

Take a look at how recursive make is implemented in the Linux kernel, with the help of simplified makefile code:

# In top Makefile vmlinux: scripts/link-vmlinux.sh $(vmlinux-deps) +$(call if_changed,link-vmlinux) # Variable assignments vmlinux-deps := $(KBUILD_LDS) $(KBUILD_VMLINUX_INIT) $(KBUILD_VMLINUX_MAIN) $(KBUILD_VMLINUX_LIBS) export KBUILD_VMLINUX_INIT := $(head-y) $(init-y) export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y2) $(drivers-y) $(net-y) $(virt-y) export KBUILD_VMLINUX_LIBS := $(libs-y1) export KBUILD_LDS := arch/$(SRCARCH)/kernel/vmlinux.lds init-y := init/ drivers-y := drivers/ sound/ firmware/ net-y := net/ libs-y := lib/ core-y := usr/ virt-y := virt/ # Transform to corresponding built-in.a init-y := $(patsubst %/, %/built-in.a, $(init-y)) core-y := $(patsubst %/, %/built-in.a, $(core-y)) drivers-y := $(patsubst %/, %/built-in.a, $(drivers-y)) net-y := $(patsubst %/, %/built-in.a, $(net-y)) libs-y1 := $(patsubst %/, %/lib.a, $(libs-y)) libs-y2 := $(patsubst %/, %/built-in.a, $(filter-out %.a, $(libs-y))) virt-y := $(patsubst %/, %/built-in.a, $(virt-y)) # Setup the dependency. vmlinux-deps are all intermediate objects, vmlinux-dirs # are phony targets, so every time comes to this rule, the recipe of vmlinux-dirs # will be executed. Refer "4.6 Phony Targets" of `info make` $(sort $(vmlinux-deps)): $(vmlinux-dirs) ; # Variable vmlinux-dirs is the directory part of each built-in.a vmlinux-dirs := $(patsubst %/,%,$(filter %/, $(init-y) $(init-m) \ $(core-y) $(core-m) $(drivers-y) $(drivers-m) \ $(net-y) $(net-m) $(libs-y) $(libs-m) $(virt-y))) # The entry of recursive make $(vmlinux-dirs): $(Q)$(MAKE) $(build)[email protected] need-builtin=1
           

The recursive make recipe is expanded, for example:

make -f scripts/Makefile.build obj=init need-builtin=1
           

This means make will go into scripts/Makefile.build to continue the work of building each built-in.a. With the help of scripts/link-vmlinux.sh, the vmlinux file is finally under source root.

Understanding vmlinux vs. bzImage

Many Linux kernel developers may not be clear about the relationship between vmlinux and bzImage. For example, here is their relationship in x86-64:

vmlinux-bzimage.png

The source root vmlinux is stripped, compressed, put into piggy.S, then linked with other peer objects into arch/x86/boot/compressed/vmlinux. Meanwhile, a file called setup.bin is produced under arch/x86/boot. There may be an optional third file that has relocation info, depending on the configuration of CONFIG_X86_NEED_RELOCS.

A host program called build, provided by the kernel, builds these two (or three) parts into the final bzImage file.

Dependency tracking

Kbuild tracks three kinds of dependencies:

1. All prerequisite files (both *.c and *.h)
2. CONFIG_ options used in all prerequisite files
3. Command-line dependencies used to compile the target

The first one is easy to understand, but what about the second and third? Kernel developers often see code pieces like this:

#ifdef CONFIG_SMP __boot_cpu_id = cpu; #endif
           

When CONFIG_SMP changes, this piece of code should be recompiled. The command line for compiling a source file also matters, because different command lines may result in different object files.

When a .c file uses a header file via a #include directive, you need write a rule like this:

main.o: defs.h recipe...
           

When managing a large project, you need a lot of these kinds of rules; writing them all would be tedious and boring. Fortunately, most modern C compilers can write these rules for you by looking at the #include lines in the source file. For the GNU Compiler Collection (GCC), it is just a matter of adding a command-line parameter: -MD depfile

# In scripts/Makefile.lib c_flags = -Wp,-MD,$(depfile) $(NOSTDINC_FLAGS) $(LINUXINCLUDE) \ -include $(srctree)/include/linux/compiler_types.h \ $(__c_flags) $(modkern_cflags) \ $(basename_flags) $(modname_flags)
           

This would generate a .d file with content like:

init_task.o: init/init_task.c include/linux/kconfig.h \ include/generated/autoconf.h include/linux/init_task.h \ include/linux/rcupdate.h include/linux/types.h \ ...
           

Then the host program fixdep takes care of the other two dependencies by taking the depfile and command line as input, then outputting a .<target>.cmd file in makefile syntax, which records the command line and all the prerequisites (including the configuration) for a target. It looks like this:

# The command line used to compile the target cmd_init/init_task.o := gcc -Wp,-MD,init/.init_task.o.d -nostdinc ... ... # The dependency files deps_init/init_task.o := \ $(wildcard include/config/posix/timers.h) \ $(wildcard include/config/arch/task/struct/on/stack.h) \ $(wildcard include/config/thread/info/in/task.h) \ ... include/uapi/linux/types.h \ arch/x86/include/uapi/asm/types.h \ include/uapi/asm-generic/types.h \ ...
           

A .<target>.cmd file will be included during recursive make, providing all the dependency info and helping to decide whether to rebuild a target or not.

The secret behind this is that fixdep will parse the depfile (.d file), then parse all the dependency files inside, search the text for all the CONFIG_ strings, convert them to the corresponding empty header file, and add them to the target's prerequisites. Every time the configuration changes, the corresponding empty header file will be updated, too, so kbuild can detect that change and rebuild the target that depends on it. Because the command line is also recorded, it is easy to compare the last and current compiling parameters.

Looking ahead

Kconfig/kbuild remained the same for a long time until the new maintainer, Masahiro Yamada, joined in early 2017, and now kbuild is under active development again. Don't be surprised if you soon see something different from what's in this article.