调试遗漏IoStartNextPacket引起的阻塞

前面 driver verifier检测驱动死锁一文中本想检测一下驱动中潜在的死锁来解决驱动无响应的bug，然而并没有实质性的进展。后来通过一系列的调试终于找到了根源所在，本文用于记录查找问题的过程。

下面是测试程序，主线程循环读驱动，子线程循环写驱动，由于是异步操作，每次读写结束后，线程都要调用WaitForSingleObject等待驱动完成IRP。可是由于驱动处理有误，ReadFile调用2次后，主线程就彻底卡死在Wait函数上了:

hDev = CreateFileA(interfaceBuff, 
        GENERIC_ALL, 
        FILE_SHARE_READ | FILE_SHARE_WRITE,
        NULL, OPEN_EXISTING,
        FILE_ATTRIBUTE_NORMAL | FILE_FLAG_OVERLAPPED, 
        NULL);

  InitializeCriticalSection(&cs);

  CreateThread(NULL, 0, (LPTHREAD_START_ROUTINE)ThreadProc, NULL, 0, &threadId);

  while (1)
  {
    memset(readBuff, 0, 4096);
    BOOL bRead = ReadFile(hDev, readBuff, 4096, &readLen, &overlapRd); 1)
    WaitForSingleObject(overlapRd.hEvent, INFINITE);                   2)

    if (readLen)
    {
      EnterCriticalSection(&cs);
      printf("_tmain read:%s\n",readBuff);
      LeaveCriticalSection(&cs);
    }
      
  }

  CloseHandle(hDev);
  return 0;
}

DWORD ThreadProc(void* param)
{
  OVERLAPPED overlapRd = { 0 },overlapWr = {0};
  overlapRd.hEvent = CreateEvent(NULL, FALSE, FALSE, NULL);
  overlapWr.hEvent = CreateEvent(NULL, FALSE, FALSE, NULL);

  char writeBuff[4096] = { "_ThreadProc" }, readBuff[4096] = { 0 };
  DWORD len = 0, writeLen, readLen;

  while (1)
  {
    WriteFile(hDev, writeBuff, strlen(writeBuff), &writeLen, &overlapWr);
    WaitForSingleObject(overlapWr.hEvent, INFINITE);

  }
}

对于异步IRP，调用ReadFile会传入OVERLAPPED结构，之后WaitForSingleObject会在OVERLAPPED!HEVENT上等待。当驱动中完成读请求(调用IoCompleteRequest)时，会调用KeSetEvnet(irp->UserEvent);使得用户层的等待函数返回。基于上述分析，可以推知，ReadFile的IRP请求被挂起后一直没有被完成(悬而未决的IRP请求)。为了验证我的猜测，可以查看一下线程的IRP请求:

1.首先查看R3下线程ID，当前进程有3个线程，0/1分别对应读写线程，其ID号分别为:75c.3a8/75c.828;2号线程可能是windbg调试线程

0:002> ~*kb ;注意这里的0:002> 暗示当前是在R3下调试

   0  Id: 75c.3a8 Suspend: 1 Teb: 7ffdf000 Unfrozen
ChildEBP RetAddr  Args to Child              
0042d53c 77165e6c 754f179c 0000007c 00000000 ntdll!KiFastSystemCallRet
0042d540 754f179c 0000007c 00000000 00000000 ntdll!NtWaitForSingleObject+0xc
0042f7d0 001b400a 00000001 005f96d0 005f9738 onlyForWrite!main+0x22b [c:\studio\samplechar\onlyforwrite\onlyforwrite\testsamplechar.cpp @ 173]

   1  Id: 75c.828 Suspend: 1 Teb: 7ffde000 Unfrozen
ChildEBP RetAddr  Args to Child              
01c7db38 77165e6c 754f179c 00000090 00000000 ntdll!KiFastSystemCallRet
01c7db3c 754f179c 00000090 00000000 00000000 ntdll!NtWaitForSingleObject+0xc
01c7fd20 76ef1174 00000000 01c7fd6c 7717b3f5 onlyForWrite!ThreadProc+0x139 [c:\studio\samplechar\onlyforwrite\onlyforwrite\testsamplechar.cpp @ 200]

#  2  Id: 75c.8dc Suspend: 1 Teb: 7ffdd000 Unfrozen
ChildEBP RetAddr  Args to Child              
007efa94 771bd279 7764a5c0 00000000 00000000 ntdll!DbgBreakPoint
007efac4 76ef1174 00000000 007efb10 7717b3f5 ntdll!DbgUiRemoteBreakin+0x3c

2.利用上面的线程号，在R0 windbg上查看线程挂起的IRP:

kd> !process 0 0 ;查找所有进程  注意这里调试开头是kd>
**** NT ACTIVE PROCESS DUMP ****
PROCESS 89a4bd40  SessionId: 1  Cid: 075c    Peb: 7ffd4000  ParentCid: 0f8c
    DirBase: 7ff7e480  ObjectTable: 8a2a16d0  HandleCount:  57.
    Image: onlyForWrite.exe ;找到onlyForWrite.exe进程的信息


kd> !process 89a4bd40 6 ;查找进程onlyForWrite.exe下所有线程
PROCESS 89a4bd40  SessionId: 1  Cid: 075c    Peb: 7ffd4000  ParentCid: 0f8c
    DirBase: 7ff7e480  ObjectTable: 8a2a16d0  HandleCount:  57.
    Image: onlyForWrite.exe

        THREAD 8a927d48  Cid 075c.03a8  Teb: 7ffdf000 Win32Thread: fd3c7dd8 WAIT: (Suspended) KernelMode Non-Alertable
SuspendCount 1
FreezeCount 1
            8a927f10  Semaphore Limit 0x2
        IRP List: ;这个线程有IRP队列
            8773eef8: (0006,0100) Flags: 00060970  Mdl: 00000000
        

        THREAD 8b3a37d8  Cid 075c.0828  Teb: 7ffde000 Win32Thread: 00000000 WAIT: (Suspended) KernelMode Non-Alertable
SuspendCount 1
FreezeCount 1
            8b3a39a0  Semaphore Limit 0x2
        Not impersonating
        DeviceMap                 946136a0
        Owning Process            89a4bd40       Image:         onlyForWrite.exe
        Attached Process          N/A            Image:         N/A
        

        THREAD 8b328cd8  Cid 075c.08dc  Teb: 7ffdd000 Win32Thread: 00000000 WAIT: (Executive) KernelMode Non-Alertable
SuspendCount 1
            92ddd6e0  SynchronizationEvent
        Not impersonating
        DeviceMap                 946136a0
        Owning Process            89a4bd40       Image:         onlyForWrite.exe
        Attached Process          N/A            Image:         N/A

对比windbg输出的3个线程信息，发现唯独Cid==075c.03a8线程有IRP队列。结合R3 windbg中查到的读线程ID:0x3a8，可以确定读线程的确出了什么问题。再进一步确认IRP是不是应用层发出的:

kd> !irp 8773eef8 ；8773ee8从前面THREAD 8a927d48的IRP List中获取
Irp is active with 2 stacks 2 is current (= 0x8773ef8c)
 No Mdl: System buffer=8776e000: Thread 8a927d48:  Irp stack trace.  
     cmd  flg cl Device   File     Completion-Context
 [N/A(0), N/A(0)]
            0  0 00000000 00000000 00000000-00000000    

      Args: 00000000 00000000 00000000 00000000
>[IRP_MJ_READ(3), N/A(0)]
            0  1 8a7762b8 876c7988 00000000-00000000    pending -->irp状态是挂起的 
         \Driver\SampleChar
      Args: 00001000 00000000 00000000 00000000
kd> da 8776e000
8776e000  "................................"
8776e020  "................................"

果不其然，这是程序发出的IRP请求未被处理。至少可以肯定应该是驱动中StartIo函数有错误了。

#pragma code_seg()
void SampleStartIo(PDEVICE_OBJECT devObj, PIRP irp)
{
  KEVENT workEvt, completeEvt;
  KIRQL origIrql;
  NTSTATUS status = STATUS_SUCCESS;
  unsigned long readLen;
  SampleCharDevContext* devCtx = (SampleCharDevContext*)devObj->DeviceExtension;
  IO_STACK_LOCATION* curStack = IoGetCurrentIrpStackLocation(irp);

  KeInitializeEvent(&workEvt,SynchronizationEvent,FALSE);
  KeInitializeEvent(&completeEvt, NotificationEvent, FALSE);

  if (curStack->Parameters.Read.Length > 4096)
  {
    status = irp->IoStatus.Status = STATUS_BUFFER_OVERFLOW;
    irp->IoStatus.Information = 0;

    IoCompleteRequest(irp, IO_NO_INCREMENT);
    IoStartNextPacket(devObj, FALSE);
    return;
  }

  KeAcquireSpinLock(&devCtx->devSpinLock, &origIrql);
  
  if(devCtx->buffPos != 0x00UL)
  {
    readLen = devCtx->buffPos >= curStack->Parameters.Read.Length ? curStack->Parameters.Read.Length : devCtx->buffPos;
    RtlCopyMemory(irp->AssociatedIrp.SystemBuffer,
      devCtx->SampleBuff,
      readLen);
    devCtx->buffRemained += readLen;
    devCtx->buffPos -= readLen;
  }
  else
  {
    KeReleaseSpinLock(&devCtx->devSpinLock, origIrql);
    
    irp->IoStatus.Status = STATUS_SUCCESS;
    irp->IoStatus.Information = 0x00UL;
    IoCompleteRequest(irp, IO_NO_INCREMENT); ->1)
    //maybe blocked at here,if without this
    //IoStartNextPacket(devObj, FALSE);  ->4)
    return; ->2)
  }

  KeReleaseSpinLock(&devCtx->devSpinLock, origIrql);

  IoCopyCurrentIrpStackLocationToNext(irp);
  IoSetCompletionRoutine(irp, IrpAsyncReadCompleteRoutine, &completeEvt, TRUE, TRUE, TRUE);
  
  status = IoCallDriver(devCtx->lowerDev,irp);
  if (status == STATUS_PENDING)
  {
    KeWaitForSingleObject(&completeEvt, Executive, KernelMode, FALSE, NULL);
  }
  
  irp->IoStatus.Status = STATUS_SUCCESS;
  irp->IoStatus.Information = readLen;
  IoCompleteRequest(irp, IO_NO_INCREMENT);

  IoStartNextPacket(devObj,FALSE);
}

NTSTATUS SampleCharReadAsync(PDEVICE_OBJECT devObj, PIRP irp)
{
  IoMarkIrpPending(irp);
  IoStartPacket(devObj,irp,NULL,NULL); ->3)
  return STATUS_PENDING;
}

kd> !devobj 8a7762b8 -------------->8a7762b8是前面!irp 命令输出的irp相关的devobj
Device object (8a7762b8) is for:
 SampleChar \Driver\SampleChar DriverObject 8774cb18
Current Irp 8773eef8 RefCount 0 Type 00000022 Flags 0000204c
Dacl 8fb049c0 DevExt 8a776370 DevObjExt 8a7763f0 
ExtensionFlags (0x30000000)  DOE_RAW_FDO
                             Unknown flags 0x10000000
Characteristics (0x00000100)  FILE_DEVICE_SECURE_OPEN
AttachedTo (Lower) 8761a400 \Driver\PnpManager
Device queue is busy - Queue flink = blink  ------->windbg显示设备队列忙

调试遗漏IoStartNextPacket引起的阻塞

继续阅读

ZOJ 1032 Area 2

UVA 590 Always on the run

FZU 1978 Repair the brackets

UVA 10344- 23 out of 5

POJ 2115 C Looooops

ZOJ 3700 Ever Dream

编程高手：VC 6.0下利用消息实现内部进程通讯

Compile workrave under windows &ndash; My exprience 在Windows上编译Workrave

UVA 620 Cellular Structure

Windows下使用GSL（GNU Scientific Library）

swift资料合集

苹果审核 Guideline 2.1 - Information Needed

windows不能在本地计算机上运行oracleDbConsoleorcl

Windows下VS开发环境环境安装工程项目设置关于Debug和Release的提示

Windows下配置Apache的SSL服务

Mac｜Windows系统本地照片自动上传到服务器