通過對苯叔中斷處理流程的學習，将學習心得總結為如下文檔
linux ARM V8異常/中斷處理流程
- 1 異常向量表
-
- 1.1 Vector table offsets from vector table base address
- 1.2 linux arm64 異常向量表代碼
-
- 1.2.1 異常向量表
- 1.2.2 kernel_ventry
- 2 中斷處理
-
- 2.1 硬體中斷号和軟體中斷号的差別
- 2.2 中斷注冊函數
-
- 2.2.1 request_irq
- 2.2.2 request_threaded_irq
- 2.3 中斷上半部處理流程
-
- 2.3.1 el1_irq
- 2.3.2 kernel_entry
- 2.3.3 kernel_exit
- 2.3.4 irq_handler
-
- 2.3.4.1 irq_stack_entry
- 2.3.4.2 irq_stack_exit
- 2.3.4.3 handle_arch_irq
- 2.3.4.4 gic_init_bases
- 2.3.4.5 gic_handle_irq
- 2.3.4.6 handle_domain_irq
- 2.3.4.7 __handle_domain_irq
- 2.3.4.8 irq_enter和__irq_enter
- 2.3.4.9 irq_exit
- 2.3.4.9 generic_handle_irq
- 2.3.4.10 generic_handle_irq_desc
1 異常向量表
當中斷被觸發之後首先會到異常向量表中去執行對應的異常處理分支。
下面是在linux 核心中ARM V8的異常向量表
1.1 Vector table offsets from vector table base address
對于AArch64 exception table在《ARM_v8_architecture_Programmer Guide v1.0.pdf》一書的10.4 AArch64 exception table有具體的介紹，其異常向量表如下圖所示。
異常向量表的解釋：
- 表1表示在目前異常等級下，使用的SP寄存器為EL0的SP寄存器
- 表2表示在目前異常等級下，使用目前異常等級的SP_ELx
- 表3表示從低異常等級發生異常并且在AArch64模式下執行，進入到高優先級去執行異常處理流程
- 表4表示從低異常等級發生異常并且在AArch32模式下執行，進入到高優先級去執行異常處理流程
1.2 linux arm64 異常向量表代碼
異常向量表要按照2K對齊的方式去放置，如下所示的
.align 11
就是要求下面的代碼段在放置到
.entry.text
時，要按照2^11(即2K)的大小去放置。
在arm64 異常向量表中需要關注kernel_ventry和如何組成異常向量的處理分支
- kernel_ventry
- 如何組成異常向量的處理分支
1.2.1 異常向量表
/*
* Exception vectors.
*/
.pushsection ".entry.text", "ax"
.align 11
ENTRY(vectors)
kernel_ventry 1, sync_invalid // Synchronous EL1t
kernel_ventry 1, irq_invalid // IRQ EL1t
kernel_ventry 1, fiq_invalid // FIQ EL1t
kernel_ventry 1, error_invalid // Error EL1t
kernel_ventry 1, sync // Synchronous EL1h
kernel_ventry 1, irq // IRQ EL1h
kernel_ventry 1, fiq_invalid // FIQ EL1h
kernel_ventry 1, error // Error EL1h
kernel_ventry 0, sync // Synchronous 64-bit EL0
kernel_ventry 0, irq // IRQ 64-bit EL0
kernel_ventry 0, fiq_invalid // FIQ 64-bit EL0
kernel_ventry 0, error // Error 64-bit EL0
#ifdef CONFIG_COMPAT
kernel_ventry 0, sync_compat, 32 // Synchronous 32-bit EL0
kernel_ventry 0, irq_compat, 32 // IRQ 32-bit EL0
kernel_ventry 0, fiq_invalid_compat, 32 // FIQ 32-bit EL0
kernel_ventry 0, error_compat, 32 // Error 32-bit EL0
#else
kernel_ventry 0, sync_invalid, 32 // Synchronous 32-bit EL0
kernel_ventry 0, irq_invalid, 32 // IRQ 32-bit EL0
kernel_ventry 0, fiq_invalid, 32 // FIQ 32-bit EL0
kernel_ventry 0, error_invalid, 32 // Error 32-bit EL0
#endif
END(vectors)
1.2.2 kernel_ventry
-
.align 7
-
b el\()\el\()_\label
kernel_ventry 1, irq
b el1_irq
.macro kernel_ventry, el, label, regsize = 64
.align 7
#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
.if \el == 0
alternative_if ARM64_UNMAP_KERNEL_AT_EL0
.if \regsize == 64
mrs x30, tpidrro_el0
msr tpidrro_el0, xzr
.else
mov x30, xzr
.endif
alternative_else_nop_endif
.endif
#endif
sub sp, sp, #S_FRAME_SIZE
#ifdef CONFIG_VMAP_STACK
/*
* Test whether the SP has overflowed, without corrupting a GPR.
* Task and IRQ stacks are aligned so that SP & (1 << THREAD_SHIFT)
* should always be zero.
*/
add sp, sp, x0 // sp' = sp + x0
sub x0, sp, x0 // x0' = sp' - x0 = (sp + x0) - x0 = sp
tbnz x0, #THREAD_SHIFT, 0f
sub x0, sp, x0 // x0'' = sp' - x0' = (sp + x0) - sp = x0
sub sp, sp, x0 // sp'' = sp' - x0 = (sp + x0) - x0 = sp
b el\()\el\()_\label
0:
/*
* Either we've just detected an overflow, or we've taken an exception
* while on the overflow stack. Either way, we won't return to
* userspace, and can clobber EL0 registers to free up GPRs.
*/
/* Stash the original SP (minus S_FRAME_SIZE) in tpidr_el0. */
msr tpidr_el0, x0
/* Recover the original x0 value and stash it in tpidrro_el0 */
sub x0, sp, x0
msr tpidrro_el0, x0
/* Switch to the overflow stack */
adr_this_cpu sp, overflow_stack + OVERFLOW_STACK_SIZE, x0
/*
* Check whether we were already on the overflow stack. This may happen
* after panic() re-enables interrupts.
*/
mrs x0, tpidr_el0 // sp of interrupted context
sub x0, sp, x0 // delta with top of overflow stack
tst x0, #~(OVERFLOW_STACK_SIZE - 1) // within range?
b.ne __bad_stack // no? -> bad stack pointer
/* We were already on the overflow stack. Restore sp/x0 and carry on. */
sub sp, sp, x0
mrs x0, tpidrro_el0
#endif
b el\()\el\()_\label
.endm
2 中斷處理
中斷處理需要關注一下幾點：
- 硬體中斷号和軟體中斷号的差別
- 中斷注冊函數
- 中斷上半部處理流程
- 中斷線程化
- 中斷親和性
- 中斷搶占
2.1 硬體中斷号和軟體中斷号的差別
linux核心相容了目前市面上的主流的處理器架構，不同的處理器架構可能采用不同的中斷處理器，linux為了屏蔽硬體上的差異，抽象出了軟體中斷号，友善在軟體上做統一的中斷處理。
硬體中斷号和軟體中斷号确定的時間點：
- 硬體中斷号：在SOC設計階段就确定了
- 軟體中斷号：在中斷注冊的過程中，通過查找allocated_irq 的連續cnt個bit位申請到對應的軟體中斷号，在申請到軟體中斷号之後，會把hwirq和virq記錄到irq_domain中。
2.2 中斷注冊函數
中斷注冊函數分為兩個函數，
request_irq
和
request_threaded_irq
，而
request_irq
是基于
request_threaded_irq
的封裝，将
thread_fn
置空形成的函數。
需要重點說明的兩點是：（1）在
request_irq
和
request_threaded_irq
兩個函數中使用的中斷号都是經過map之後的virq；（2）使用
request_threaded_irq
注冊中斷并且，thread_fn函數不為空，則核心會起一個核心線程去進行中斷相關的事務，這樣的應用場景存在于RTLinux中，目的就是為了防止頻繁的中斷阻止高優先任務的執行，增加系統的實時性。
2.2.1 request_irq
request_irq在注冊中斷時，預設是不采用中斷線程化處理的，其在調用request_threaded_irq函數時，将thread_fn參數置為NULL了。
/**
* request_irq - Add a handler for an interrupt line
* @irq: The interrupt line to allocate
* @handler: Function to be called when the IRQ occurs.
* Primary handler for threaded interrupts
* If NULL, the default primary handler is installed
* @flags: Handling flags
* @name: Name of the device generating this interrupt
* @dev: A cookie passed to the handler function
* * This call allocates an interrupt and establishes a handler; see
* the documentation for request_threaded_irq() for details.
*/
static inline int __must_check
request_irq(unsigned int irq, irq_handler_t handler, unsigned long flags,
const char *name, void *dev)
{
return request_threaded_irq(irq, handler, NULL, flags, name, dev);
}
2.2.2 request_threaded_irq
request_threaded_irq函數主要包括一下功能：
-
desc = irq_to_desc(irq);
- 填充irqaction結構
- action->handler = handler;
- action->thread_fn = thread_fn;
- action->flags = irqflags;
- action->name = devname;
- action->dev_id = dev_id;
- __setup_irq注冊irqaction
/**
* request_threaded_irq - allocate an interrupt line
* @irq: Interrupt line to allocate
* @handler: Function to be called when the IRQ occurs.
* Primary handler for threaded interrupts
* If NULL and thread_fn != NULL the default
* primary handler is installed
* @thread_fn: Function called from the irq handler thread
* If NULL, no irq thread is created
* @irqflags: Interrupt type flags
* @devname: An ascii name for the claiming device
* @dev_id: A cookie passed back to the handler function
*
* This call allocates interrupt resources and enables the
* interrupt line and IRQ handling. From the point this
* call is made your handler function may be invoked. Since
* your handler function must clear any interrupt the board
* raises, you must take care both to initialise your hardware
* and to set up the interrupt handler in the right order.
*
* If you want to set up a threaded irq handler for your device
* then you need to supply @handler and @thread_fn. @handler is
* still called in hard interrupt context and has to check
* whether the interrupt originates from the device. If yes it
* needs to disable the interrupt on the device and return
* IRQ_WAKE_THREAD which will wake up the handler thread and run
* @thread_fn. This split handler design is necessary to support
* shared interrupts.
*
* Dev_id must be globally unique. Normally the address of the
* device data structure is used as the cookie. Since the handler
* receives this value it makes sense to use it.
*
* If your interrupt is shared you must pass a non NULL dev_id
* as this is required when freeing the interrupt.
*
* Flags:
*
* IRQF_SHARED Interrupt is shared
* IRQF_TRIGGER_* Specify active edge(s) or level
*
*/
int request_threaded_irq(unsigned int irq, irq_handler_t handler,
irq_handler_t thread_fn, unsigned long irqflags,
const char *devname, void *dev_id)
{
struct irqaction *action;
struct irq_desc *desc;
int retval;
if (irq == IRQ_NOTCONNECTED)
return -ENOTCONN;
/*
* Sanity-check: shared interrupts must pass in a real dev-ID,
* otherwise we'll have trouble later trying to figure out
* which interrupt is which (messes up the interrupt freeing
* logic etc).
*
* Also IRQF_COND_SUSPEND only makes sense for shared interrupts and
* it cannot be set along with IRQF_NO_SUSPEND.
*/
if (((irqflags & IRQF_SHARED) && !dev_id) ||
(!(irqflags & IRQF_SHARED) && (irqflags & IRQF_COND_SUSPEND)) ||
((irqflags & IRQF_NO_SUSPEND) && (irqflags & IRQF_COND_SUSPEND)))
return -EINVAL;
desc = irq_to_desc(irq);
if (!desc)
return -EINVAL;
if (!irq_settings_can_request(desc) ||
WARN_ON(irq_settings_is_per_cpu_devid(desc)))
return -EINVAL;
if (!handler) {
if (!thread_fn)
return -EINVAL;
handler = irq_default_primary_handler;
}
action = kzalloc(sizeof(struct irqaction), GFP_KERNEL);
if (!action)
return -ENOMEM;
action->handler = handler;
action->thread_fn = thread_fn;
action->flags = irqflags;
action->name = devname;
action->dev_id = dev_id;
retval = irq_chip_pm_get(&desc->irq_data);
if (retval < 0) {
kfree(action);
return retval;
}
retval = __setup_irq(irq, desc, action);
if (retval) {
irq_chip_pm_put(&desc->irq_data);
kfree(action->secondary);
kfree(action);
}
#ifdef CONFIG_DEBUG_SHIRQ_FIXME
if (!retval && (irqflags & IRQF_SHARED)) {
/*
* It's a shared IRQ -- the driver ought to be prepared for it
* to happen immediately, so let's make sure....
* We disable the irq to make sure that a 'real' IRQ doesn't
* run in parallel with our fake.
*/
unsigned long flags;
disable_irq(irq);
local_irq_save(flags);
handler(irq, dev_id);
local_irq_restore(flags);
enable_irq(irq);
}
#endif
return retval;
}
2.3 中斷上半部處理流程
為了介紹該部分，下面以在EL1工作場景下，産生一個中斷，對中斷的處理流程去分析整個處理流程，其中除
el1_irq
函數中的
irq_handler
函數處理流程中包含中斷的低層處理，其餘部分均是中斷的高層處理。
當在el1異常等級下産生中斷，則會執行到異常向量表的
kernel_ventry 1, irq // IRQ EL1h
的分支，解析之後，該分支會執行到
b el1_irq
分支。
2.3.1 el1_irq
el1_irq
有以下幾點需要重點關注一下：
-
kernel_entry 1
-
.align 6
-
irq_handler
-
kernel_exit 1
el1_irq
.align 6
el1_irq:
kernel_entry 1
gic_prio_irq_setup pmr=x20, tmp=x1
enable_da_f
#ifdef CONFIG_ARM64_PSEUDO_NMI
test_irqs_unmasked res=x0, pmr=x20
cbz x0, 1f
bl asm_nmi_enter
1:
#endif
#ifdef CONFIG_TRACE_IRQFLAGS
bl trace_hardirqs_off
#endif
irq_handler
#ifdef CONFIG_PREEMPTION
ldr x24, [tsk, #TSK_TI_PREEMPT] // get preempt count
alternative_if ARM64_HAS_IRQ_PRIO_MASKING
/*
* DA_F were cleared at start of handling. If anything is set in DAIF,
* we come back from an NMI, so skip preemption
*/
mrs x0, daif
orr x24, x24, x0
alternative_else_nop_endif
cbnz x24, 1f // preempt count != 0 || NMI return path
bl arm64_preempt_schedule_irq // irq en/disable is done inside
1:
#endif
#ifdef CONFIG_ARM64_PSEUDO_NMI
/*
* When using IRQ priority masking, we can get spurious interrupts while
* PMR is set to GIC_PRIO_IRQOFF. An NMI might also have occurred in a
* section with interrupts disabled. Skip tracing in those cases.
*/
test_irqs_unmasked res=x0, pmr=x20
cbz x0, 1f
bl asm_nmi_exit
1:
#endif
#ifdef CONFIG_TRACE_IRQFLAGS
#ifdef CONFIG_ARM64_PSEUDO_NMI
test_irqs_unmasked res=x0, pmr=x20
cbnz x0, 1f
#endif
bl trace_hardirqs_on
1:
#endif
kernel_exit 1
ENDPROC(el1_irq)
2.3.2 kernel_entry
kernel_entry函數的主要功能如下所示：
- 将通用寄存器X0 ~ X30 儲存到棧中
- 将系統寄存器sp，fp，elr_el1，spsr_el1儲存到棧中
.macro kernel_entry, el, regsize = 64
.if \regsize == 32
mov w0, w0 // zero upper 32 bits of x0
.endif
stp x0, x1, [sp, #16 * 0]
stp x2, x3, [sp, #16 * 1]
stp x4, x5, [sp, #16 * 2]
stp x6, x7, [sp, #16 * 3]
stp x8, x9, [sp, #16 * 4]
stp x10, x11, [sp, #16 * 5]
stp x12, x13, [sp, #16 * 6]
stp x14, x15, [sp, #16 * 7]
stp x16, x17, [sp, #16 * 8]
stp x18, x19, [sp, #16 * 9]
stp x20, x21, [sp, #16 * 10]
stp x22, x23, [sp, #16 * 11]
stp x24, x25, [sp, #16 * 12]
stp x26, x27, [sp, #16 * 13]
stp x28, x29, [sp, #16 * 14]
.if \el == 0
clear_gp_regs
mrs x21, sp_el0
ldr_this_cpu tsk, __entry_task, x20
msr sp_el0, tsk
// Ensure MDSCR_EL1.SS is clear, since we can unmask debug exceptions
// when scheduling.
ldr x19, [tsk, #TSK_TI_FLAGS]
disable_step_tsk x19, x20
apply_ssbd 1, x22, x23
.else
add x21, sp, #S_FRAME_SIZE
get_current_task tsk
/* Save the task's original addr_limit and set USER_DS */
ldr x20, [tsk, #TSK_TI_ADDR_LIMIT]
str x20, [sp, #S_ORIG_ADDR_LIMIT]
mov x20, #USER_DS
str x20, [tsk, #TSK_TI_ADDR_LIMIT]
/* No need to reset PSTATE.UAO, hardware's already set it to 0 for us */
.endif /* \el == 0 */
mrs x22, elr_el1
mrs x23, spsr_el1
stp lr, x21, [sp, #S_LR]
/*
* In order to be able to dump the contents of struct pt_regs at the
* time the exception was taken (in case we attempt to walk the call
* stack later), chain it together with the stack frames.
*/
.if \el == 0
stp xzr, xzr, [sp, #S_STACKFRAME]
.else
stp x29, x22, [sp, #S_STACKFRAME]
.endif
add x29, sp, #S_STACKFRAME
#ifdef CONFIG_ARM64_SW_TTBR0_PAN
/*
* Set the TTBR0 PAN bit in SPSR. When the exception is taken from
* EL0, there is no need to check the state of TTBR0_EL1 since
* accesses are always enabled.
* Note that the meaning of this bit differs from the ARMv8.1 PAN
* feature as all TTBR0_EL1 accesses are disabled, not just those to
* user mappings.
*/
alternative_if ARM64_HAS_PAN
b 1f // skip TTBR0 PAN
alternative_else_nop_endif
.if \el != 0
mrs x21, ttbr0_el1
tst x21, #TTBR_ASID_MASK // Check for the reserved ASID
orr x23, x23, #PSR_PAN_BIT // Set the emulated PAN in the saved SPSR
b.eq 1f // TTBR0 access already disabled
and x23, x23, #~PSR_PAN_BIT // Clear the emulated PAN in the saved SPSR
.endif
__uaccess_ttbr0_disable x21
1:
#endif
stp x22, x23, [sp, #S_PC]
/* Not in a syscall by default (el0_svc overwrites for real syscall) */
.if \el == 0
mov w21, #NO_SYSCALL
str w21, [sp, #S_SYSCALLNO]
.endif
/* Save pmr */
alternative_if ARM64_HAS_IRQ_PRIO_MASKING
mrs_s x20, SYS_ICC_PMR_EL1
str x20, [sp, #S_PMR_SAVE]
alternative_else_nop_endif
/*
* Registers that may be useful after this macro is invoked:
*
* x20 - ICC_PMR_EL1
* x21 - aborted SP
* x22 - aborted PC
* x23 - aborted PSTATE
*/
.endm
2.3.3 kernel_exit
kernel_exit
函數的主要功能如下所示：
- 将通用寄存器X0 ~ X30 的值從棧中恢複到寄存器中
- 将系統寄存器sp，fp，elr_el1，spsr_el1的值從棧中恢複到對應的寄存器中
.macro kernel_exit, el
.if \el != 0
disable_daif
/* Restore the task's original addr_limit. */
ldr x20, [sp, #S_ORIG_ADDR_LIMIT]
str x20, [tsk, #TSK_TI_ADDR_LIMIT]
/* No need to restore UAO, it will be restored from SPSR_EL1 */
.endif
/* Restore pmr */
alternative_if ARM64_HAS_IRQ_PRIO_MASKING
ldr x20, [sp, #S_PMR_SAVE]
msr_s SYS_ICC_PMR_EL1, x20
mrs_s x21, SYS_ICC_CTLR_EL1
tbz x21, #6, .L__skip_pmr_sync\@ // Check for ICC_CTLR_EL1.PMHE
dsb sy // Ensure priority change is seen by redistributor
.L__skip_pmr_sync\@:
alternative_else_nop_endif
ldp x21, x22, [sp, #S_PC] // load ELR, SPSR
.if \el == 0
ct_user_enter
.endif
#ifdef CONFIG_ARM64_SW_TTBR0_PAN
/*
* Restore access to TTBR0_EL1. If returning to EL0, no need for SPSR
* PAN bit checking.
*/
alternative_if ARM64_HAS_PAN
b 2f // skip TTBR0 PAN
alternative_else_nop_endif
.if \el != 0
tbnz x22, #22, 1f // Skip re-enabling TTBR0 access if the PSR_PAN_BIT is set
.endif
__uaccess_ttbr0_enable x0, x1
.if \el == 0
/*
* Enable errata workarounds only if returning to user. The only
* workaround currently required for TTBR0_EL1 changes are for the
* Cavium erratum 27456 (broadcast TLBI instructions may cause I-cache
* corruption).
*/
bl post_ttbr_update_workaround
.endif
1:
.if \el != 0
and x22, x22, #~PSR_PAN_BIT // ARMv8.0 CPUs do not understand this bit
.endif
2:
#endif
.if \el == 0
ldr x23, [sp, #S_SP] // load return stack pointer
msr sp_el0, x23
tst x22, #PSR_MODE32_BIT // native task?
b.eq 3f
#ifdef CONFIG_ARM64_ERRATUM_845719
alternative_if ARM64_WORKAROUND_845719
#ifdef CONFIG_PID_IN_CONTEXTIDR
mrs x29, contextidr_el1
msr contextidr_el1, x29
#else
msr contextidr_el1, xzr
#endif
alternative_else_nop_endif
#endif
3:
#ifdef CONFIG_ARM64_ERRATUM_1418040
alternative_if_not ARM64_WORKAROUND_1418040
b 4f
alternative_else_nop_endif
/*
* if (x22.mode32 == cntkctl_el1.el0vcten)
* cntkctl_el1.el0vcten = ~cntkctl_el1.el0vcten
*/
mrs x1, cntkctl_el1
eon x0, x1, x22, lsr #3
tbz x0, #1, 4f
eor x1, x1, #2 // ARCH_TIMER_USR_VCT_ACCESS_EN
msr cntkctl_el1, x1
4:
#endif
apply_ssbd 0, x0, x1
.endif
msr elr_el1, x21 // set up the return data
msr spsr_el1, x22
ldp x0, x1, [sp, #16 * 0]
ldp x2, x3, [sp, #16 * 1]
ldp x4, x5, [sp, #16 * 2]
ldp x6, x7, [sp, #16 * 3]
ldp x8, x9, [sp, #16 * 4]
ldp x10, x11, [sp, #16 * 5]
ldp x12, x13, [sp, #16 * 6]
ldp x14, x15, [sp, #16 * 7]
ldp x16, x17, [sp, #16 * 8]
ldp x18, x19, [sp, #16 * 9]
ldp x20, x21, [sp, #16 * 10]
ldp x22, x23, [sp, #16 * 11]
ldp x24, x25, [sp, #16 * 12]
ldp x26, x27, [sp, #16 * 13]
ldp x28, x29, [sp, #16 * 14]
ldr lr, [sp, #S_LR]
add sp, sp, #S_FRAME_SIZE // restore sp
.if \el == 0
alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0
#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
bne 5f
msr far_el1, x30
tramp_alias x30, tramp_exit_native
br x30
5:
tramp_alias x30, tramp_exit_compat
br x30
#endif
.else
eret
.endif
sb
.endm
2.3.4 irq_handler
irq_handler
函數主要功能是調用
handle_arch_irq
中斷處理函數
- irq_stack_entry核心棧的儲存
- blr handle_arch_irq
- irq_stack_exit核心棧的恢複
/*
* Interrupt handling.
*/
.macro irq_handler
ldr_l x1, handle_arch_irq
mov x0, sp
irq_stack_entry
blr x1
irq_stack_exit
.endm
2.3.4.1 irq_stack_entry
.macro irq_stack_entry
mov x19, sp // preserve the original sp
/*
* Compare sp with the base of the task stack.
* If the top ~(THREAD_SIZE - 1) bits match, we are on a task stack,
* and should switch to the irq stack.
*/
ldr x25, [tsk, TSK_STACK]
eor x25, x25, x19
and x25, x25, #~(THREAD_SIZE - 1)
cbnz x25, 9998f
ldr_this_cpu x25, irq_stack_ptr, x26
mov x26, #IRQ_STACK_SIZE
add x26, x25, x26
/* switch to the irq stack */
mov sp, x26
9998:
.endm
2.3.4.2 irq_stack_exit
/*
* x19 should be preserved between irq_stack_entry and
* irq_stack_exit.
*/
.macro irq_stack_exit
mov sp, x19
.endm
2.3.4.3 handle_arch_irq
handle_arch_irq
的設定是在
set_handle_irq
中實作的，
set_handle_irq
函數的實作如下所示
#ifdef CONFIG_GENERIC_IRQ_MULTI_HANDLER
int __init set_handle_irq(void (*handle_irq)(struct pt_regs *))
{
if (handle_arch_irq)
return -EBUSY;
handle_arch_irq = handle_irq;
return 0;
}
#endif
2.3.4.4 gic_init_bases
對于AArch64來說，通常采用gic v3的中斷控制器去管理複雜的中斷系統，而gic v3在初始化過程中通過
gic_init_bases
注冊了
set_handle_irq(gic_handle_irq);
函數實體，進行将
handle_arch_irq
指向
gic_handle_irq
函數，
gic_init_bases
函數實作如下所示：
static int __init gic_init_bases(void __iomem *dist_base,
struct redist_region *rdist_regs,
u32 nr_redist_regions,
u64 redist_stride,
struct fwnode_handle *handle)
{
u32 typer;
int err;
if (!is_hyp_mode_available())
static_branch_disable(&supports_deactivate_key);
if (static_branch_likely(&supports_deactivate_key))
pr_info("GIC: Using split EOI/Deactivate mode\n");
gic_data.fwnode = handle;
gic_data.dist_base = dist_base;
gic_data.redist_regions = rdist_regs;
gic_data.nr_redist_regions = nr_redist_regions;
gic_data.redist_stride = redist_stride;
/*
* Find out how many interrupts are supported.
*/
typer = readl_relaxed(gic_data.dist_base + GICD_TYPER);
gic_data.rdists.gicd_typer = typer;
gic_enable_quirks(readl_relaxed(gic_data.dist_base + GICD_IIDR),
gic_quirks, &gic_data);
pr_info("%d SPIs implemented\n", GIC_LINE_NR - 32);
pr_info("%d Extended SPIs implemented\n", GIC_ESPI_NR);
/*
* ThunderX1 explodes on reading GICD_TYPER2, in violation of the
* architecture spec (which says that reserved registers are RES0).
*/
if (!(gic_data.flags & FLAGS_WORKAROUND_CAVIUM_ERRATUM_38539))
gic_data.rdists.gicd_typer2 = readl_relaxed(gic_data.dist_base + GICD_TYPER2);
gic_data.domain = irq_domain_create_tree(handle, &gic_irq_domain_ops,
&gic_data);
irq_domain_update_bus_token(gic_data.domain, DOMAIN_BUS_WIRED);
gic_data.rdists.rdist = alloc_percpu(typeof(*gic_data.rdists.rdist));
gic_data.rdists.has_rvpeid = true;
gic_data.rdists.has_vlpis = true;
gic_data.rdists.has_direct_lpi = true;
gic_data.rdists.has_vpend_valid_dirty = true;
if (WARN_ON(!gic_data.domain) || WARN_ON(!gic_data.rdists.rdist)) {
err = -ENOMEM;
goto out_free;
}
gic_data.has_rss = !!(typer & GICD_TYPER_RSS);
pr_info("Distributor has %sRange Selector support\n",
gic_data.has_rss ? "" : "no ");
if (typer & GICD_TYPER_MBIS) {
err = mbi_init(handle, gic_data.domain);
if (err)
pr_err("Failed to initialize MBIs\n");
}
set_handle_irq(gic_handle_irq);
gic_update_rdist_properties();
gic_smp_init();
gic_dist_init();
gic_cpu_init();
gic_cpu_pm_init();
if (gic_dist_supports_lpis()) {
its_init(handle, &gic_data.rdists, gic_data.domain);
its_cpu_init();
} else {
if (IS_ENABLED(CONFIG_ARM_GIC_V2M))
gicv2m_init(handle, gic_data.domain);
}
gic_enable_nmi_support();
return 0;
out_free:
if (gic_data.domain)
irq_domain_remove(gic_data.domain);
free_percpu(gic_data.rdists.rdist);
return err;
}
2.3.4.5 gic_handle_irq
當中斷出發之後，到了gic中斷控制器之後，中斷控制器的中斷處理函數就會做如下幾件事情：
- 通過
gic_read_iar
- 判斷中斷号是否有效
- 對于PPI和SPI中斷，執行
handle_domain_irq
- 對于SGI(IPI)中斷，則執行
handle_IPI
static asmlinkage void __exception_irq_entry gic_handle_irq(struct pt_regs *regs)
{
u32 irqnr;
irqnr = gic_read_iar();
if (gic_supports_nmi() &&
unlikely(gic_read_rpr() == GICD_INT_NMI_PRI)) {
gic_handle_nmi(irqnr, regs);
return;
}
if (gic_prio_masking_enabled()) {
gic_pmr_mask_irqs();
gic_arch_enable_irqs();
}
/* Check for special IDs first */
if ((irqnr >= 1020 && irqnr <= 1023))
return;
/* Treat anything but SGIs in a uniform way */
if (likely(irqnr > 15)) {
int err;
if (static_branch_likely(&supports_deactivate_key))
gic_write_eoir(irqnr);
else
isb();
err = handle_domain_irq(gic_data.domain, irqnr, regs);
if (err) {
WARN_ONCE(true, "Unexpected interrupt received!\n");
gic_deactivate_unhandled(irqnr);
}
return;
}
if (irqnr < 16) {
gic_write_eoir(irqnr);
if (static_branch_likely(&supports_deactivate_key))
gic_write_dir(irqnr);
#ifdef CONFIG_SMP
/*
* Unlike GICv2, we don't need an smp_rmb() here.
* The control dependency from gic_read_iar to
* the ISB in gic_write_eoir is enough to ensure
* that any shared data read by handle_IPI will
* be read after the ACK.
*/
handle_IPI(irqnr, regs);
#else
WARN_ONCE(true, "Unexpected SGI received!\n");
#endif
}
}
2.3.4.6 handle_domain_irq
在__handle_domain_irq函數中，通過irq_domain中和hwirq對應的virq去調用對應的中斷處理函數。
注：irq_domain是中斷控制器的軟體抽象，用于描述中斷控制器，每一個中斷控制器都有一個對應的軟體上的irq_domain。
/*
* Convert a HW interrupt number to a logical one using a IRQ domain,
* and handle the result interrupt number. Return -EINVAL if
* conversion failed. Providing a NULL domain indicates that the
* conversion has already been done.
*/
int __handle_domain_irq(struct irq_domain *domain, unsigned int hwirq,
bool lookup, struct pt_regs *regs);
static inline int handle_domain_irq(struct irq_domain *domain,
unsigned int hwirq, struct pt_regs *regs)
{
return __handle_domain_irq(domain, hwirq, true, regs);
}
2.3.4.7 __handle_domain_irq
該函數主要的功能就是調用和目前觸發中斷的hwirq所對應的中斷處理函數，在該函數中有以下幾個重要的點需要關注：
- irq_enter
- 通過irq_find_mapping函數在irq_domain中找到和hwirq對應的virq
- generic_handle_irq
- irq_exit
/**
* __handle_domain_irq - Invoke the handler for a HW irq belonging to a domain
* @domain: The domain where to perform the lookup
* @hwirq: The HW irq number to convert to a logical one
* @lookup: Whether to perform the domain lookup or not
* @regs: Register file coming from the low-level handling code
*
* Returns: 0 on success, or -EINVAL if conversion has failed
*/
int __handle_domain_irq(struct irq_domain *domain, unsigned int hwirq,
bool lookup, struct pt_regs *regs)
{
struct pt_regs *old_regs = set_irq_regs(regs);
unsigned int irq = hwirq;
int ret = 0;
irq_enter();
#ifdef CONFIG_IRQ_DOMAIN
if (lookup)
irq = irq_find_mapping(domain, hwirq);
#endif
/*
* Some hardware gives randomly wrong interrupts. Rather
* than crashing, do something sensible.
*/
if (unlikely(!irq || irq >= nr_irqs)) {
ack_bad_irq(irq);
ret = -EINVAL;
} else {
generic_handle_irq(irq);
}
irq_exit();
set_irq_regs(old_regs);
return ret;
}
2.3.4.8 irq_enter和__irq_enter
__irq_enter會做如下幾件事情：
- 通過
account_irq_enter_time
- 通過
preempt_count_add
HARDIRQ_OFFSET
-
trace_hardirq_enter
current->hardirq_context++
/*
* It is safe to do non-atomic ops on ->hardirq_context,
* because NMI handlers may not preempt and the ops are
* always balanced, so the interrupted value of ->hardirq_context
* will always be restored.
*/
#define __irq_enter() \
do { \
account_irq_enter_time(current); \
preempt_count_add(HARDIRQ_OFFSET); \
trace_hardirq_enter(); \
} while (0)
/*
* Enter an interrupt context.
*/
void irq_enter(void)
{
rcu_irq_enter();
if (is_idle_task(current) && !in_interrupt()) {
/*
* Prevent raise_softirq from needlessly waking up ksoftirqd
* here, as softirq will be serviced on return from interrupt.
*/
local_bh_disable();
tick_irq_enter();
_local_bh_enable();
}
__irq_enter();
}
2.3.4.9 irq_exit
該函數是退出中斷，該函數的執行标志着退出硬中斷上下文（中斷上半部），此時會使能中斷（
local_irq_disable
），并且會檢查目前是否有軟中斷需要處理（
!in_interrupt() && local_softirq_pending()
），如果目前不處于中斷上下文中(
in_interrupt()
)，并且有軟中斷需要處理，則會調用
invoke_softirq
函數去處理軟中斷。
關于in_interrupt的解析可以參考https://blog.csdn.net/u014100559/article/details/106819336?spm=1001.2014.3001.5502
中斷下半部的了解可以參考https://blog.csdn.net/u014100559/article/details/90779456
/*
* Exit an interrupt context. Process softirqs if needed and possible:
*/
void irq_exit(void)
{
#ifndef __ARCH_IRQ_EXIT_IRQS_DISABLED
local_irq_disable();
#else
lockdep_assert_irqs_disabled();
#endif
account_irq_exit_time(current);
preempt_count_sub(HARDIRQ_OFFSET);
if (!in_interrupt() && local_softirq_pending())
invoke_softirq();
tick_irq_exit();
rcu_irq_exit();
trace_hardirq_exit(); /* must be last! */
}
2.3.4.9 generic_handle_irq
通過調用generic_handle_irq_desc函數去調用對應的中斷處理函數，在這個函數中需要關注的點是
*desc = irq_to_desc(irq)
，此處的含義是用virq找到對應的irq_desc，進而去調用
desc->handle_irq
/**
* generic_handle_irq - Invoke the handler for a particular irq
* @irq: The irq number to handle
*
*/
int generic_handle_irq(unsigned int irq)
{
struct irq_desc *desc = irq_to_desc(irq);
if (!desc)
return -EINVAL;
generic_handle_irq_desc(desc);
return 0;
}
2.3.4.10 generic_handle_irq_desc
調用
desc->handle_irq
，此時irq_desc所對應的是具體硬體裝置的軟體抽象，它在軟體上描述了一個具體硬體在發生中斷時，所需要處理的所有的事情以及對應的中斷初始化的資訊。
/*
* Architectures call this to let the generic IRQ layer
* handle an interrupt.
*/
static inline void generic_handle_irq_desc(struct irq_desc *desc)
{
desc->handle_irq(desc);
}