大綱
- 參考
- 目标
- 背景:
-
- 這樣做的目的是什麼?
- 怎麼做?
- 1 研究一下V4L2的sub device的初始化
- 2 具體到我使用的sensor mlx75027,具體的初始化流程
- 3.V4L2 cmd的類型
- 4.不同類型指令的處理過程
-
- 1. VIDIOC_XXX 的處理
- 2. V4L2_CID_XXX 的處理
- 3. 自定義cmd的處理
- 4. v4l2_subdev_call() 這個是實作自定義cmd的重點
- 5.遇到的問題
-
- 1. is_isp_used 和 is_bayer_raw 的值會影響 V4L2_CID_XXX 的處理邏輯
- 2. linux 核心宏container_of的意思
- 6. 全志提供了V4L2裝置直接讀寫I2C的做法
參考
我非常嚴重的參考了以下的文章,提前列出來:
非常好的文章
Linux V4L2 draft
v4l2的學習建議和流程解析
V4L2架構解析
Linux V4L2 draft
Linux V4L2 draft
作者: 嵌入式Max
V4L2架構全解析
00 - V4L2架構概述
01 - V4L2架構-v4l2 device
02 - V4L2架構-media-device
03 - V4L2架構-videobuf2
V4L2架構-control
V4L2架構-control的資料結構
作者: Letcos
[Camera]v4l2架構核心空間解析
下面的文章作為一般參考,這裡記錄一下:
V4L2源代碼之旅一:struct v4l2_subdev
V4L2源代碼之旅二:V4L2 sub-device userspace API
V4L2源代碼之旅三:I2C sub-device drivers
V4L2源代碼之旅四:struct video_device
V4L2源代碼之旅五:V4L2的起點和終點
V4L2源代碼之旅六:源碼追蹤
V4L2源代碼之旅七:controls queryctrl()
V4L2源代碼之旅八:ioctl
V4L2源代碼之旅九:videobuf
V4L2源代碼之旅十:videobuf主要結構體
Xilinx Linux V4L2視訊管道(Video Pipeline)驅動程式分析
目标
對一個封裝為V4L2的Sensor,在AP層增加一個直接操作 I2C的API.
背景:
這裡使用的是全志T507的SDK,linux版本是4.9.191
因為在全志平台,sensor裝置并不是預設做為一個标準的I2C 裝置,可以通過 /dev/i2c-x 可以直接對裝置進行讀寫.而是注冊為V4L2的裝置( /dev/videox). 這樣AP層就無法通過正常的I2C操作來操作Sensor 的 Register.
全志平台的I2C有自己的标準 :TWI : Normal Two Wire Interface :“全志平台相容I2C标準協定的總線控制協定”.
但是軟體層面,用的還是I2C的描述 ,這樣就太惡心了,我認為完全沒有必要新發明一套私有的體系,這樣違背了設立I2C的初衷.
這樣做的目的是什麼?
之是以注冊為/dev/videox,就是為了避免AP層對sensor直接進行I2C操作而可能引發問題.
而I2C的操作,全志自己封裝了1層cci來進行操作.
那麼具體是使用cci還是使用twi(也就是I2C),從kernel裡面的配置可以得知是可以選擇的,預設是保留現在的cci方式,可選是将V4L2做為一個正常的I2C裝置.
.config - Linux/arm64 4.9.191 Kernel Configuration
Device Drivers > Multimedia support > V4L platform devices > select cci or cci to twi (use internal cci) —>
(X) use internal cci
( ) chenge cci to twi
怎麼做?
一開始我也不知道V4L2的指令是如何運作的,那麼隻能通過解剖麻雀的方法進行研究.
我對subdevice的了解是裝置可以作為I2C, sensor, v4l2的子裝置,可以互相指向和轉化,是以叫做subdevice,這個有新的了解再補充.
// 從v4l2_subdev擷取i2c_client:
struct i2c_client *client = v4l2_get_subdevdata(sd);
// 從i2c_client擷取v4l2_subdev:
struct v4l2_subdev *sd = i2c_get_clientdata(client);
## 從sd成員來擷取父成員的位址
struct v4l2_subdev *sd;
struct sensor_info *info = to_state(sd);
## 擷取info的成員成員sd的位址
struct sensor_info *info;
struct v4l2_subdev *sd = &info->sd;
## 這2個是一個互為 相反的操作
1 研究一下V4L2的sub device的初始化
在sunxi平台, /dev/video0 的裝置,大緻是經過下面的流程進行了初始化.
module_init(vin_init);
└──>vin_init(void);
│ // drivers/media/platform/sunxi-vin/vin-video/vin_core.c
├── sunxi_csi_platform_register();
└──>sunxi_vin_core_register_driver();
└──>platform_driver_register(&vin_core_driver);
└──>static struct platform_driver vin_core_driver = {
.probe = vin_core_probe,
...
}
vin_core_probe();
│ //drivers/media/platform/sunxi-vin/vin-video/vin_video.c
└──>vin_initialize_capture_subdev();
│ // drivers/media/v4l2-core/v4l2-subdev.c
│ // 此時将device作為連結清單加入init序列,并開始初始化具體的device,下面是間接的調用
├──>v4l2_subdev_init();
│ sd->internal_ops = &vin_capture_sd_internal_ops;
│ struct v4l2_subdev_internal_ops vin_capture_sd_internal_ops = {
│ .registered = vin_capture_subdev_registered,
│ .unregistered = vin_capture_subdev_unregistered,
│};
└──v4l2_set_subdevdata(sd)
└──>vin_capture_subdev_registered()
| // drivers/media/platform/sunxi-vin/vin-video/vin_video.c
├──>vin_init_controls()
│ │ //将3A之類的指令注冊一下,并給初始值
│ ├──>v4l2_ctrl_new_std(V4L2_CID_XXX)
│ ├──>v4l2_ctrl_new_std_menu(V4L2_CID_XXX)
│ └──>v4l2_ctrl_new_custom(V4L2_CID_XXX)
└──>vin_init_video()
├──>cap->vdev.name // vin_video0
├──>cap->vdev.fops = &vin_fops; // 檔案接口的ioctl
├──>cap->vdev.ioctl_ops = &vin_ioctl_ops; // ioctl的指令清單
├──>video_register_device(cap->vdev)
├──>video_set_drvdata()
└──>vb2_queue_init()
2 具體到我使用的sensor mlx75027,具體的初始化流程
// drivers/media/platform/sunxi-vin/modules/sensor/mlx75027_mipi.c
module_init(init_sensor);
│ static struct i2c_driver sensor_driver = {
│ .probe = sensor_probe,
│ ...
│ };
└──>init_sensor(void);
└──>cci_dev_init_helper(&sensor_driver);
└──>sensor_driver->probe()
└──>sensor_probe()
├──>sensor_init_controls(sensor_ctrl_ops);
│ │ // 定義sensor對3A指令處理的邏輯,可覆寫父級别
│ └──>v4l2_ctrl_new_std(V4L2_CID_XXX);
└──>cci_dev_probe_helper(sensor_ops); // 設定sensor_ops
│ // 定義sensor對3A指令處理的邏輯,可覆寫父級别
└──>struct v4l2_subdev_ops sensor_ops = {
.core = &sensor_core_ops,
...
};
└──>struct v4l2_subdev_core_ops sensor_core_ops = {
.ioctl = sensor_ioctl, // 上層通過ioctl,最終會調用到這裡,我實作的自定義指令,也會到這個處理
...
};
3.V4L2 cmd的類型
-
第一種是定義在 /include/uapi/linux/videodev2.h 中 VIDIOC_XXX
這種指令的個數不超過 #define BASE_VIDIOC_PRIVATE 192 做為v4l2保留的指令,自定義的指令,不建議加在這個區間内.
這些指令是v4l2操作的基本函數,包括裝置的打開和關閉,流資料的請求等
/*
* I O C T L C O D E S F O R V I D E O D E V I C E S
*
*/
#define VIDIOC_QUERYCAP _IOR('V', 0, struct v4l2_capability)
#define VIDIOC_RESERVED _IO('V', 1)
#define VIDIOC_ENUM_FMT _IOWR('V', 2, struct v4l2_fmtdesc)
#define VIDIOC_G_FMT _IOWR('V', 4, struct v4l2_format)
#define VIDIOC_S_FMT _IOWR('V', 5, struct v4l2_format)
#define VIDIOC_REQBUFS _IOWR('V', 8, struct v4l2_requestbuffers)
#define VIDIOC_QUERYBUF _IOWR('V', 9, struct v4l2_buffer)
#define VIDIOC_G_FBUF _IOR('V', 10, struct v4l2_framebuffer)
#define VIDIOC_S_FBUF _IOW('V', 11, struct v4l2_framebuffer)
#define VIDIOC_OVERLAY _IOW('V', 14, int)
#define VIDIOC_QBUF _IOWR('V', 15, struct v4l2_buffer)
#define VIDIOC_EXPBUF _IOWR('V', 16, struct v4l2_exportbuffer)
#define VIDIOC_DQBUF _IOWR('V', 17, struct v4l2_buffer)
#define VIDIOC_STREAMON _IOW('V', 18, int)
#define VIDIOC_STREAMOFF _IOW('V', 19, int)
-
第二種是定義在 /include/uapi/linux/v4l2-controls.h 中 V4L2_CID_XXX
因為是應用層使用,是以這裡定義了位址,并按照一定的大小來區分不同的應用可以使用的範圍.
#define V4L2_CTRL_CLASS_USER 0x00980000 /* Old-style 'user' controls */
#define V4L2_CID_BASE (V4L2_CTRL_CLASS_USER | 0x900)
#define V4L2_CID_USER_BASE V4L2_CID_BASE
#define V4L2_CID_USER_CLASS (V4L2_CTRL_CLASS_USER | 1)
#define V4L2_CID_BRIGHTNESS (V4L2_CID_BASE+0)
#define V4L2_CID_CONTRAST (V4L2_CID_BASE+1)
-
第三種是定義在 /include/media/sunxi_camera_v2.h 裡面給各個Camera使用的cmd.
可以看到在私有的192之後,可以添加sensor的私有資料 VIDIOC_ISP_XXX 和 VIDIOC_VIN_SENSOR_XXX .
#define BASE_VIDIOC_PRIVATE 192
#define VIDIOC_ISP_AE_STAT_REQ _IOWR('V', BASE_VIDIOC_PRIVATE + 1, struct isp_stat_buf)
#define VIDIOC_ISP_HIST_STAT_REQ _IOWR('V', BASE_VIDIOC_PRIVATE + 2, struct isp_stat_buf)
...
#define VIDIOC_VIN_SENSOR_CFG_REQ _IOWR('V', BASE_VIDIOC_PRIVATE + 60, struct sensor_config)
#define VIDIOC_VIN_SENSOR_EXP_GAIN _IOWR('V', BASE_VIDIOC_PRIVATE + 61, struct sensor_exp_gain)
#define VIDIOC_VIN_SENSOR_SET_FPS _IOWR('V', BASE_VIDIOC_PRIVATE + 62, struct sensor_fps)
...
// 我新增的指令放在了最後
struct msg_i2c {
unsigned short addr;
unsigned short value;
};
#define VIDIOC_VIN_SET_I2C_DATA _IOWR('V', BASE_VIDIOC_PRIVATE + 76, struct msg_i2c)
4.不同類型指令的處理過程
1. VIDIOC_XXX 的處理
int sel = 0;
int mode = 5;
enum v4l2_buf_type type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE;
if (-1 == camera_init(sel, mode))
return -1;
if (-1 == camera_fmt_set(mode))
return -1;
if (-1 == req_frame_buffers())
return -1;
pixformat = TVD_PL_YUV420;
disp_init(input_size.width, input_size.height, pixformat);
if (-1 == ioctl(fd, VIDIOC_STREAMON, &type))
{
printf("VIDIOC_STREAMON failed\n");
return -1;
}
在 ioctl 之後,從kernel 到 driver 的流程如下
// 在 第1節已經知道了 vin_init_video() 會注冊 cap->vdev.fops = &vin_fops;
// 是以對vin裝置的ioctl都會到這個函數
static struct v4l2_file_operations vin_fops = {
.unlocked_ioctl = video_ioctl2,
}
static struct v4l2_ioctl_info v4l2_ioctls[] = {
IOCTL_INFO_FNC(VIDIOC_STREAMON, v4l_streamon, v4l_print_buftype, INFO_FL_PRIO | INFO_FL_QUEUE),
IOCTL_INFO_FNC(VIDIOC_STREAMOFF, v4l_streamoff, v4l_print_buftype, INFO_FL_PRIO | INFO_FL_QUEUE),
}
EXPORT_SYMBOL(video_ioctl2);
video_ioctl2()
└──>__video_do_ioctl()
├──>v4l2_is_known_ioctl() // 如果是在已知cmd清單裡面,就執行注冊的func
├──>struct v4l2_ioctl_info *info = &v4l2_ioctls[_IOC_NR(cmd)];
└──>info->func(ops, file, fh, arg);
└──>v4l_streamon()
└──>ops->vidioc_streamon(file, fh, *(unsigned int *)arg);
└──>static const struct v4l2_ioctl_ops vin_ioctl_ops = {
.vidioc_streamon = vidioc_streamon,
.vidioc_streamoff = vidioc_streamoff,
}
// /drivers/media/platform/sunxi-vin/vin-video/vin_video.c
vidioc_streamon()
└──>vb2_streamon()
│ // /drivers/media/v4l2-core/videobuf2-core.c
└──>vb2_core_streamon()
├──>v4l_vb2q_enable_media_source()
└──>vb2_start_streaming()
│ // /drivers/media/platform/sunxi-vin/modules/sensor/mlx75027_mipi.c
└──>struct v4l2_subdev_video_ops sensor_video_ops = {
.s_stream = sensor_s_stream,
}
sensor_s_stream()
└──>sensor_s_streamon(sd,1);
└──>sensor_write(sd, 0x1001, 0x01); // 實際操作寄存器
2. V4L2_CID_XXX 的處理
// drivers/media/platform/sunxi-vin/modules/sensor/mlx75027_mipi.c
module_init(init_sensor);
│ static struct i2c_driver sensor_driver = {
│ .probe = sensor_probe,
│ ...
│ };
└──>init_sensor(void);
└──>cci_dev_init_helper(&sensor_driver);
└──>sensor_driver->probe()
└──>sensor_probe()
├──>sensor_init_controls(sensor_video_ops);
│ │ // 定義sensor對3A指令處理的邏輯,可覆寫父級别
│ └──>v4l2_ctrl_new_std(V4L2_CID_XXX);
└──>cci_dev_probe_helper(sensor_ops); // 設定sensor_ops
│ // 定義sensor對3A指令處理的邏輯,可覆寫父級别
└──>v4l2_subdev_video_ops sensor_video_ops = {
.s_parm = sensor_s_parm,
.g_parm = sensor_g_parm,
...
};
└──>struct v4l2_ioctl_ops soc_camera_ioctl_ops = {
.vidioc_g_parm = soc_camera_g_parm,
}
└──>soc_camera_g_parm()
└──>default_g_parm()
└──>vidioc_g_parm()
└──>v4l2_subdev_call(vinc->vid_cap.pipe.sd[VIN_IND_SENSOR], video, g_parm, parms);
└──>in_ctrl_ops = {
.g_volatile_ctrl = vin_g_volatile_ctrl,
};
└──>vin_g_volatile_ctrl()
// /drivers/media/platform/sunxi-vin/modules/sensor/mlx75027_mipi.c
v4l2_ctrl_ops sensor_ctrl_ops = {
.g_volatile_ctrl = sensor_g_ctrl,
};
└──>int sensor_g_ctrl(struct v4l2_ctrl *ctrl)
switch (ctrl->id)
case V4L2_CID_EXPOSURE:
sensor_g_exp(); // 實際的處理函數
3. 自定義cmd的處理
// /include/media/sunxi_camera_v2.h 在頭檔案增加新的cmd和傳入的參數定義
struct msg_i2c {
unsigned short addr;
unsigned short value;
};
#define VIDIOC_VIN_SET_I2C_DATA _IOWR('V', BASE_VIDIOC_PRIVATE + 76, struct msg_i2c)
// /drivers/media/platform/sunxi-vin/vin-video/vin_video.c
struct v4l2_ioctl_ops vin_ioctl_ops = {
.vidioc_default = vin_param_handler, // 走的是這個函數
└──>vin_param_handler()
└──>switch (cmd) {
case VIDIOC_VIN_SET_I2C_DATA:
vidioc_set_i2c_zhang(struct file *file, struct v4l2_fh *fh, struct msg_i2c *i2c);
└──>struct vin_core *vinc = video_drvdata(file);
v4l2_subdev_call(vinc->vid_cap.pipe.sd[VIN_IND_SENSOR], core, ioctl, VIDIOC_VIN_SET_I2C_DATA, i2c);
│// /drivers/media/platform/sunxi-vin/modules/sensor/mlx75027_mipi.c
└──>struct v4l2_subdev_ops sensor_ops = {
.core = &sensor_core_ops,
};
struct v4l2_subdev_core_ops sensor_core_ops = {
.ioctl = sensor_ioctl,
}
└──>sensor_ioctl()
└──>switch (cmd) {
case VIDIOC_VIN_SET_I2C_DATA:
sensor_s_i2c_data(sd, (struct msg_i2c *)arg);
└──> sensor_write(sd, i2c->addr, i2c->value);
4. v4l2_subdev_call() 這個是實作自定義cmd的重點
/*
* Call an ops of a v4l2_subdev, doing the right checks against
* NULL pointers.
*
* Example: err = v4l2_subdev_call(sd, video, s_std, norm);
*/
#define v4l2_subdev_call(sd, o, f, args...) \
(!(sd) ? -ENODEV : (((sd)->ops->o && (sd)->ops->o->f) ? \
(sd)->ops->o->f((sd), ##args) : -ENOIOCTLCMD))
vidioc_set_i2c_zhang(struct file *file, struct v4l2_fh *fh, struct msg_i2c *i2c);
└──>v4l2_subdev_call(vinc->vid_cap.pipe.sd[VIN_IND_SENSOR], core, ioctl, VIDIOC_VIN_SET_I2C_DATA, i2c);
struct v4l2_subdev_ops sensor_ops = {
.core = &sensor_core_ops,
};
struct v4l2_subdev_core_ops sensor_core_ops = {
.ioctl = sensor_ioctl,
}
v4l2_subdev_call(vinc->vid_cap.pipe.sd[VIN_IND_SENSOR], core, ioctl, VIDIOC_VIN_SET_I2C_DATA, i2c);
實際上等價于
v4l2_subdev_call(subdevice, sensor_ops , ioctl, VIDIOC_VIN_SET_I2C_DATA, struct msg_i2c);
翻譯過來就是通過subdevice來調用 sensor_ops 的 ioctl 方法, 傳入的參數是VIDIOC_VIN_SET_I2C_DATA, 方法的結構體是struct msg_i2c.
5.遇到的問題
1. is_isp_used 和 is_bayer_raw 的值會影響 V4L2_CID_XXX 的處理邏輯
static int vin_g_volatile_ctrl(struct v4l2_ctrl *ctrl)
{
struct vin_vid_cap *cap = container_of(ctrl->handler, struct vin_vid_cap, ctrl_handler);
struct sensor_instance *inst = get_valid_sensor(cap->vinc);
if (inst->is_isp_used && inst->is_bayer_raw) {
// 一般而言,V4L2_CID_XXX 會走這個分支
} else {
}
// /drivers/media/platform/sunxi-vin/vin.c
static int __vin_handle_sensor_info(struct sensor_instance *inst)
{
if (inst->cam_type == SENSOR_RAW) {
inst->is_bayer_raw = 1;
inst->is_isp_used = 1;
} else if (inst->cam_type == SENSOR_YUV) {
inst->is_bayer_raw = 0;
inst->is_isp_used = 0;
} else {
inst->is_bayer_raw = 0;
inst->is_isp_used = 0;
}
return 0;
}
// 從這個函數可以看到,隻有sensor的type是SENSOR_RAW的時候,這2個屬性才都是1.
// 是以就需要在初始化清單裡面将sensor的屬性配置好,如果是新增的sensor,這2個值都是0.
// /drivers/media/platform/sunxi-vfe/utility/sensor_info.c
struct sensor_item sensor_list_t[] = {
/* name i2c_addr sensor type sensor size sensor max pclk */
{ "ov2640", 0x60, SENSOR_YUV, PIXEL_NUM_2M, CORE_CLK_RATE_FOR_2M},
{ "ov5647_mipi", 0x6c, SENSOR_RAW, PIXEL_NUM_5M, CORE_CLK_RATE_FOR_5M},
}
// 可以把新增的sensor添加在這個結構體的後面.
2. linux 核心宏container_of的意思
linux 核心宏container_of剖析
關于linux container_of用法
container_of用法及實作
linux 驅動程式中 container_of宏解析
詳解Linux核心之位操作
struct struct *struct_p= container_of(struct_member_p, struct struct, struct_member);
//container_of 的作用是:已知 struct結構中某個成員struct_member的指針struct_member_p,就可以知道整個struct的指針struct_p
6. 全志提供了V4L2裝置直接讀寫I2C的做法
我是在封裝完畢API之後,才在全志的官網看到了這個方法…現在分享出來.
【FAQ440】Tina線上讀寫Sensor寄存器示例
釋出日期 2021-10-30 15:47:18
平台 V536 V533 V833 V831 就是不知道T507是否也支援,實際看了下,sensor目錄下的所有檔案都是一緻的.
【指令示例】
1)cd /sys/devices/gc2053_mipi(進入目标 sensor 節點目錄)
2)echo 16 > addr_width; echo 8 > data_width(輸入目标 sensor 寄存器位址/資料位寬,請查閱 datasheet 擷取)
3)echo 0 > read_flag(read_flag:讀寫控制節點,使能為1表示後續操作為讀動作,使能為0表示後續操作為寫動作)
4)echo 30350021 > cci_client(“30350021”:0x3035【目标寄存器位址】,0x0021【将要寫入的寄存器值】,在 read_flag = 1 情況下,寫入值為無效狀态)
5)cat read_value(列印上一步操作的結果:寄存器值)
【注意事項】
1)IIC 在系統和硬體上正常工作;
2)上述指令需要在 Camera 正常工作狀态下執行才會有效,如 /sys/devices/sensor 節點是否存在,Camera 是否上電等等;
3)上述指令隻在 tina-linux 系統有效,rtos 系統暫不支援;
有幾個圖不錯,也分享出來