python ctypes總結

核心概念

ctypes官方

• 類型一覽

ctypes是python定義的為實作類型轉換的中間層，是純python資料類型與c類型通信的橋梁.

除了None,integer,string，bytes，(隐式轉換), 其他都需要轉換成ctypes類型作為參數.

None -->NULL
string bytes -- > char* wchar_t*
integer -- > int
           

#c_double 必須顯示轉換
printf(b'Hello %s %f\n', b'Wisdom', c_double(1.23))
           

标準類型裡唯一要注意的是c_char_p,很像C裡的字元串類型， 它隻能轉換integer或者bytes作為輸入.

普通的’char *'類型用POINTER(c_char)

class ctypes.c_char_p
Represents the C char * datatype when it points to a zero-terminated string. For a general character pointer that may also point to binary data, POINTER(c_char) must be used. The constructor accepts an integer address, or a bytes object.
           

• 嚴格區分類型

不像c指針那麼萬能,python裡一切皆對象.數組，指針都是要按不同的類型來分開的。

1 int a = 100;  
    ----> ct_a = c_int(100)
2 int array[100] = {0,1,2,...} 
    ---->ct_array = (c_int* 100)(0,1,2,...)
         for i in ct_array: print(i)
3 int *ptr = &a;  
    ----> ct_ptr = POINTER(c_int)(ct_a)
          ct_ptr.contents
          c_int(10)
4 *ptr = 200 ;
// ptr[0] = 200
    ----> ct_ptr[0] = 200 or ct_ptr.contents.value = 300
5 int *ptr_arr = array;
    ---->  ctype做不到! 對python 來講，數組和指針是不同的類型 
           

• Access dll function and values

function見下面例2.

value:

>>> opt_flag = c_int.in_dll(pythonapi, "Py_OptimizeFlag")
>>> print(opt_flag)
c_long(0)
           

• resttype argstype

load c的lib後，必須告訴Python,一個ctype函數的形參類型和傳回的值的類型, 函數才能正确地在python裡調用.

libusb_alloc_transfer = libusb.libusb_alloc_transfer
libusb_alloc_transfer.argtypes = [c_int]
libusb_alloc_transfer.restype = libusb_transfer_p
           

• convert python type to ctype

1.構造ctype對象時輸入;

2.通過value輸入輸入;

ct_int = ctypes.c_int(100)
ct_int.value
100
ct_int.value = 200
ct_int
c_int(200)
           

如果是ctype指針，也是要通過指派:

但是python的bytes是immutable的，也就是bytes内容變化，是指向了另外的記憶體，而不是記憶體内容本身發生了變化，這和c的一些要求是不符合的:

ct_str = ctypes.c_wchar_p('Hi!')
ct_str.value
'Hi!'
ct_str
c_wchar_p(140093580736096)
ct_str = ctypes.c_wchar_p('Hi again')
ct_str.value
'Hi again'
ct_str
c_wchar_p(140093530479112)
           

用下面的 create_string_buffer解決問題

• create_string_buffer

  建立可修改的ctype記憶體空間

>>buf = create_string_buffer(b'12345')
>>buf.value
>>b'12345'
           

• byref and pointer and POINTER

byref

:

拿到ctype instance的指針對象，,offset是偏移的位址. 但是隻能用作ctype函數的參數,比pointer(obj)更快.

pointer

:

pointer則是構造ctype對象的ctype pointer執行個體對象,

普通對象變成指針對象

ctypes.pointer(obj)
Pointer instances are created by calling the pointer() function on a ctypes type.
This function creates a new pointer instance, pointing to obj. The returned object is of the
type POINTER(type(obj)).
           

用contents取出内容,類似于

*a

:

• POINTER

POINTER僅僅是ctype層面的指針類型轉換，比如1個ctypes類型轉成其指針的類型,

類型變成指針類型

ctypes.POINTER(type)
This factory function creates and returns a new ctypes pointer type. Pointer types are
cached and reused internally, so calling this function repeatedly is cheap. type must be a
ctypes type.
           

例子:

libc.printf(b'%x',byref(c_int(123)))
>>bf60f9188
type(byref(c_int(123)))

ct_int = ct_int(123)
>> <class 'CArgObject'>
ip = pointer(ct_int)
type(ip)
>> <class '__main__.LP_c_int'>
ip.contents
>>c_int(0)
type_int_p = POINTER(c_int)
type(type_int_p)
>> <class '_ctypes.PyCPointerType'>
ip_1 = type_int_p(c_int(123))
ip_1.contents
>>c_int(123)
type(ip_1)
<class '__main__.LP_c_int'>
>>False
           

• Structure

參考下面例子裡定義.

1.注意一點，位元組序是native bytes order.

2.前向聲明結構體的方式:

>>> from ctypes import *
#先聲明1個空的
>>> class cell(Structure):
...     pass
...
#直接指派_fields_的屬性
>>> cell._fields_ = [("name", c_char_p),
...                  ("next", POINTER(cell))]
>>>
           

• Array

不同類型再乘以1個數即得到數組.

ct_int_array = c_int * 10
type(ct_int_array)
<class '_ctypes.PyCArrayType'>
           

• memmove memset

ctypes.memmove(dst, src, count)
Same as the standard C memmove library function: copies count bytes from src to dst. dst and src must be integers or ctypes instances that can be converted to pointers.

ctypes.memset(dst, c, count)
Same as the standard C memset library function: fills the memory block at address dst with count bytes of value c. dst must be an integer specifying an address, or a ctypes instance.
           

• cast:

類似c裡的強制指針類型轉換,将1個可轉換為指針類型的object，轉換為第2個參數指定的ctype pointer類型, 傳回新的ctype pointer的執行個體.

必須是

One POINTER to another POINTER

.如Array POINTER object轉Structure POINTER的object:

(理論上，可以Structure轉Array,但是沒有這樣的接口，隻有cast通過POINTER轉換才能做到)

length = 100
#某個Structure pointer object，轉為Array pointer object.
p = cast(pointer(s), POINTER(c_char * length))
#得到Structure的原始資料
raw = p.contents.raw

arr = (c_char * len(raw))()  
arr.raw = raw

s1 = cast(pointer(arr),POINTER(Structure))
#又把bytes 恢複成了ctype的structure
s1.contents
           

相當于c裡的:

Structure * s = new Structure();
char** p = (char*)&s;
// (*p)相當于Array ,Array POINTER應該是**p;
Structure * s1 = (Structure*)(*p);
           

• addressof

獲得資料的記憶體存儲位址

a = c_int(100)
addressof(a)
139795135170432
hex(addressof(a))
'0x7f24975f8780'

byref(a)
<cparam 'P' (0x7f24975f8780)>
           

神奇的幾個位址

下面3個位址分别是什麼?

ct_arr_ptr =  pointer((c_int*5)(1,2,3,4,5))
# 這是ct_arr_ptr對象的位址
<__main__.LP_c_int_Array_5 object at 0x7f2496542d90>

# 這是ct_arr_ptr所指向的内容在記憶體中的真實位址 
hex(addressof(ct_arr_ptr.contents))
'0x7f24966101e0'
hex(addressof(ct_arr_ptr.contents))
'0x7f24966101e0'
# 這是contents對象的位址，每次都臨時生成新的，但是都引用上面那個不變的記憶體裡的東西.
ct_arr_ptr.contents
<__main__.c_int_Array_5 object at 0x7f24965ae158>
ct_arr_ptr.contents
<__main__.c_int_Array_5 object at 0x7f24975f8620>

           

• CFUNCTYPE

用來定義ctype的函數指針，指向python函數,實際是傳給c語言調用的.

到這可以看到python和c的互相調用方式了:

1 python裡loadlibrary方式調用c; 
2 python提供FUNCTYPE的python回調給c語言調用;
           

libc = CDLL('/lib/x86_64-linux-gnu/libc.so.6')
qsort = libc.qsort
qsort.restype
>> <class 'ctypes.c_int'>
CMPFUNCP = CFUNCTYPE(c_int,POINTER(c_int),POINTER(c_int))
type(CMPFUNCP)
>> <class '_ctypes.PyCFuncPtrType'>
def python_cmp(a,b):
    print('cmp in python')
    return a[0]-b[0]
ia = (c_int * 10)(1,3,5,7,9,2,4,6,8,10)
qsort(id,len(ia),sizeof(c_int),CMPFUNCP(python_cmp()))
ia
1 2 3 4 5 6 7 8 9 10 <__main__.c_int_Array_10 object at 0x7f6a121c26a8>
#更簡單的寫法是用decorator:
@CFUNCTYPE(c_int,POINTER(c_int),POINTER(c_int))
def python_cmp(a,b):
    print('%s: %s',a[0],b[0])
    return a[0]-b[0]
qsort(ia,len(ia),sizeof(c_int),python_cmp)

           

• FUNCTYPE 和PFUNCTYPE的差別?

FUNCTYPE是封裝的python函數是給c語言調用的，它将不受GIL影響，純c的.

而PFUNCTYPE封裝的python函數仍然受GIL影響，這個差別還是很大的.

c希望調用的python不要卡在GIL裡的話，用FUNCTYPE;如果有些GIL操作再c裡不可忽略，用PFUNCTYPE.

執行個體1: ctypes + socket

1. 對端用c語言tcp socket發送
2. python socket.recv , 得到bytes(b’xxxx’)
3. ctypes Structure
4. 轉換bytes to ctypes 解析資料,用Structure按c的方式解析資料.

def struct2stream(s):
    length = ctypes.sizeof(s)
    p = ctypes.cast(ctypes.pointer(s), ctypes.POINTER(ctypes.c_char * length))
    return p.contents.raw


def stream2struct(string, stype):
    if not issubclass(stype, ctypes.Structure):
        raise ValueError('Not a ctypes.Structure')
    length = ctypes.sizeof(stype)
    stream = (ctypes.c_char * length)()
    stream.raw = string
    p = ctypes.cast(stream, ctypes.POINTER(stype))
    return p.contents


class CommandHeader(ctypes.Structure):
    _pack_ = 4
    _fields_ = [
        # Size of this descriptor (in bytes)
        ('MsgCommand', ctypes.c_int),
        ('MsgParam', ctypes.c_int),
        ('unkown', ctypes.c_short),
        ('unkown1', ctypes.c_short),
        ('startx', ctypes.c_short),
        ('starty', ctypes.c_short),
        ('width', ctypes.c_short),
        ('height', ctypes.c_short),
        ('len', ctypes.c_int)
    ]

class StructConverter(object):
    def __init__(self):
        pass

    @classmethod
    def encoding(cls, raw, structs):
        """
            'encode' means raw binary stream to ctype structure. 
        """
        if raw is not None and structs is not None:
            return stream2struct(raw, structs)
        else:
            return None

    @classmethod
    def decoding(cls, data):
        """
            'decode means ctpye structure to raw binary stream
        """
        if data is not None:
            return struct2stream(data)
        else:
            return None
           

收發過程:

#receive
try:
    raw = fds.recv(ctypes.sizeof(CommandHeader))
except socket.error as e:
    exit(1)
header = StructConverter.encoding(raw, CommandHeader)

#send
resp = CommandHeader()
try:
    fds.send(StructConverter.decoding(data=resp))
except socket.error as e:
    LOGGER.info('%s', e)
    return
           

執行個體2 libusb使用

參考python libusb1

并以自己實作的python調用libusb底層庫的實作為例子:

def aoa_update_point(self, action, x, y, ops=0):

    global report
    if ops == 0:
        # left = up =0
        x, y = self.axis_convert(x, y)
        real_x = x - self.ref_x
        real_y = y - self.ref_y
    else:
        real_x = x
        real_y = y

    # LOGGER.info('real point(%d %d)',real_x,real_y)

    if action == 'move' or action == 'down':
        report = Report(REPORT_ID, 1, 0, 0, 0, int(real_x), int(real_y))
    if action == 'up':
        report = Report(REPORT_ID, 0, 0, 0, 0, int(real_x), int(real_y))

    if ops == 0:
        self.set_ref(x, y)

    #transfer是1個Structurue pointer obj
    transfer = U.libusb_alloc_transfer(0)
    # contents是實體 
    transfer.contents.actual_length = sizeof(Report)
    # p_report = cast(pointer(report), c_void_p)
    transfer.contents.buffer = cast(pointer(report), c_void_p)

    # put report buffer into control_buffer
    control_buffer = create_string_buffer(sizeof(Report) + LIBUSB_CONTROL_SETUP_SIZE)

    # python級别的記憶體填充,memmove + addressof
    memmove(addressof(control_buffer) +
            LIBUSB_CONTROL_SETUP_SIZE, addressof(report),
            sizeof(report))

    # 可以看出這個是signed char 0x1 ---> 0x1 0x0 小端!
    # 實際調用了:
    # setup = cast(addressof(control_buffer), libusb_control_setup_p).contents
    U.libusb_fill_control_setup(
        addressof(control_buffer),
        U.LIBUSB_ENDPOINT_OUT | U.LIBUSB_REQUEST_TYPE_VENDOR,
        AndroidA0AProtocl.AOA_SEND_HID_EVENT.value[0],
        1,
        0,
        6)

    # LOGGER.info(control_buffer.raw)

    U.libusb_fill_control_transfer(
        transfer,
        self.aoa_handle,
        pointer(control_buffer),
        null_callback,
        None,
        0)

    transfer.contents.flags = U.LIBUSB_TRANSFER_FREE_BUFFER | U.LIBUSB_TRANSFER_FREE_TRANSFER

    rets: int = U.libusb_submit_transfer(transfer)
    if rets < 0:
        LOGGER.info(U.libusb_error_name(rets))
        return rets
    return rets
           

末了

_CData---->_CSimpleData---->(c_int,c_char,....c_long) which has value attributes
      ---->_Pointers , which has contents attributes.
      ---->Array
      ---->Structure
      ---->Union