Camera driver notes - madisongh/meta-rk3588 GitHub Wiki (2024)

Based on a machine-translated copy of Rockchip_Driver_Guide_VI_CN_v1.1.1.pdf. Covers device tree configuration only, and only for RK3588/RK3588S. Has some paraphrasing/rewording for clarity, hopefully without loss of accuracy.

• One VICAP
• MIPI dphy configured as either 4 2-lane interfaces or 2 4-lane interfaces, 2.5Gbps/lane
• MIPI dcphy configured as either 4 2-lane dphy or 2 3-lan cphy interfaces, 2.5Gbps/lane
• DVP: BT610/BT656/BT1120 pclk 150MHz
• 4672x3504 single ISP, 8192x6144 dual ISP synthesis
• Up to 3 frames of HDR

• One VICAP
• MIPI dphy configured as either 2 2-lane interfaces or 1 4-lane interface, 2.5Gbps/lane
• MIPI dcphy configured as either 2 4-lane dphy or 2 3-lan cphy interfaces, 2.5Gbps/lane
• DVP: BT610/BT656/BT1120 pclk 150MHz
• 4672x3504 single ISP, 8192x6144 dual ISP synthesis
• Up to 3 frames of HDR

The RK3588 supports two DCPHYs, node names csi2_dcphy0 and csi2_dcphy1. Each DCPHY supportssimultaneous use of RX/TX, and RX is used for camera input. The hardware supports DPHY/CPHY protocol multiplexing; it should be noted thatthe TX/RX of the same dcphy can only use DPHY or CPHY at the same time. For otherDCPHY parameters, please refer to the RK3588 data sheet.

The RK3588 supports 2 DPHYs, dphy0_hw and dphy1_hw, which support "full" and "split" modes.

• Full mode: The node name used is csi2_dphy0, and supports up to 4 lanes.
• Split mode: split into two phys for use, named csi2_dphy1 (lanes 0 and 1) and csi2_dphy2 (lanes 2 and 3). Each phy supports up to 2 lanes.

When configuring dphy0_hw for full mode, the link needs to be configured on csi2_dphy1, but the node name csi2_dphy1 needs to be changed to csi2_dphy0, so the software can identify that full mode is being used.

• Full mode: The node name used is csi2_dphy3, and supports up to 4 lanes.
• Split mode: split into two phys for use, named csi2_dphy4 (lanes 0 and 1) and csi2_dphy5 (lanes 2 and 3). Each phy supports up to 2 lanes

When configuring dphy1_hw for full mode, the link needs to be configured according on csi2_dphy4, but the node namecsi2_dphy4 needs to be changed to csi2_dphy3, so the software can identify that full mode is being used.

• To use the above MIPI PHY nodes, you need to configure the corresponding physical node (csi2_dcphy0_hw/csi2_dcphy1_hw/csi2_dphy0_hw/csi2_dphy1_hw).
• Each MIPI PHY needs a CSI2 module to parse the MIPI protocol (node names mipi0_csi2 through mipi5_csi2).
• All camera data needs to pass through VICAP, and then link to ISP. RK3588 has one VICAP hardware, which supports up to 7 simultaneous input channels, 6 MIPI (node names rkcif_mipi_lvds0 through rkcif_mipi_lvds5) and 1 DVP (rkcif_dvp). The binding relationship of each node needs to be strictly in accordance with the block diagram node numbering.
• The link relationship between each VICAP node and the ISP is indicated by corresponding virtual XXX_sditf.
• The RK3588 has 2 ISP hardware blocks. Each ISP can supports multiple virtual nodes; the software reads the image data of each channel from the DDR in turn through the read-back method. For a multi-camera setup, it is recommended to evenly distribute thedata streams between the two ISPs.

The ISP blocks support "pass-through" (or "straight-through") and "read-back" modes.

• Pass-through: Image data passes directly from VICAP to ISP without intermediate storage in DDR. It should be noted that during HDR pass-through, only short frames are true pass-through, long frames need to exist in DDR, and the ISP reads from DDR.
• Readback: refers to the data collected by VICAP to DDR. After the application obtains the data, it pushes the buffer address to the ISP, and the ISP then obtains the image data from DDR.

When configuring the device tree, if an ISP is only configured with one virtual node, the pass-through mode is used by default, and if multiple virtual nodes are configured, the readback mode is used.

Example DTS fragment for configuring an IMX464 sensor connected to DPHY1.

/{vcc_mipicsi1: vcc-mipicsi1-regulator {compatible = "regulator-fixed";gpio = <&gpio4 RK_PA6 GPIO_ACTIVE_HIGH>;pinctrl-names = "default";pinctrl-0 = <&mipicsi1_pwr>;regulator-name = "vcc_mipicsi1";enable-active-high;};};&csi2_dphy1_hw {status = "okay";};&csi2_dphy3 {status = "okay";ports {#address-cells = <1>;#size-cells = <0>;port@0 {reg = <0>;#address-cells = <1>;#size-cells = <0>;mipi_in_ucam: endpoint@1{reg = <1>;remote-endpoint = <&imx464_out>;data-lanes = <1 2 3 4>;};};port@1 {reg = <1>;#address-cells = <1>;#size-cells = <0>;csidphy3_out: endpoint@0{reg = <0>;remote-endpoint = <&mipi4_csi2_input>;};};};};&i2c4 {status = "okay";pinctrl-0 = <&i2c4m3_xfer>;imx464: imx464@36 {compatible = "sony,imx464";status = "okay";reg = <0x36>;clocks = <&cru CLK_MIPI_CAMARAOUT_M4>;clock-names = "xvclk";pinctrl-names = "default";pinctrl-0 = <&mipim0_camera4_clk>;avdd-supply = <&vcc_mipicsi1>;reset-gpios = <&gpio1 RK_PD6 GPIO_ACTIVE_HIGH>;pwdn-gpios = <&gpio3 RK_PC1 GPIO_ACTIVE_HIGH>;rockchip,camera-module-index = <0>;rockchip,camera-module-facing = "back";rockchip,camera-module-name = "CMK-OT1980-PX1";rockchip,camera-module-lens-name = "SHG102";port {imx464_out: endpoint {remote-endpoint = <&mipi_in_ucam>;data-lanes = <1 2 3 4>;};};};};&mipi4_csi2 {status = "okay";ports {#address-cells = <1>;#size-cells = <0>;port@0 {reg = <0>;#address-cells = <1>;#size-cells = <0>;mipi4_csi2_input: endpoint@1 {reg = <1>;remote-endpoint = <&csidphy3_out>;};};port@1 {reg = <1>;#address-cells = <1>;#size-cells = <0>;mipi4_csi2_output: endpoint@0 {reg = <0>;remote-endpoint = <&cif_mipi_in4>;};};};};&pinctrl {cam {mipicsi1_pwr: mipicsi1-pwr {rockchip,pins = /*camera power en */ <4 RK_PA6 RK_FUNC_GPIO &pcfg_pull_none>;};};};&rkcif{status = "okay";};&rkcif_mipi_lvds4 {status = "okay";port { cif_mipi_in4: endpoint {remote-endpoint = <&mipi4_csi2_output>;};};};&rkcif_mipi_lvds4_sditf {status = "okay";port {mipi4_lvds_sditf: endpoint {remote-endpoint = <&isp0_vir0>;};};};&rkcif_mmu {status = "okay";};#if 1&rkisp0 {status = "okay";/* the max input wh and fps of multi sensor *///max-input = <2688 152030 >; /* multi-cam sensor resolution is different, need to configure */};&isp0_mmu {status = "okay";};#else /* * Sensor resolution greater than 16M (4672x3504) requires 2 isp composite * processing, only supports single camera * Dual isp case need width 32 align */&rkisp_unite_mmu {status = "okay";};&rkisp_unite {status = "okay";};&rkisp0_vir0 {status = "okay";rockchip,hw = <&rkisp_unite>;};#endif&rkisp0_vir0 {status = "okay";port {#address-cells = <1>;#size-cells = <0>;isp0_vir0: endpoint@0 {reg = <0>;remote-endpoint = <&mipi4_lvds_sditf>;};};};

• data-lanes must specify the number of lanes used, otherwise it cannot be recognized as MIPI type
• dphy needs to be linked to the csi host node, and csi2_dphy3 corresponds to mipi4_csi2; csi2_dphy3 is onlya logical node and needs to rely on the physical node csi2_dphy1_hw
• rkcif_mipi_lvds4 is one of the logical nodes of VICAP, and the physical node rkcif and the corresponding iommu need to be configured. * rkcif_mipi_lvds4_sditf is a virtual child node, a virtual node of rkcif_mipi_lvds4, used to link to the ISP.

The sensor driver generally implements the three power supply operations: avdd, dvdd, and dovdd. If you use a power supply similar to the RK809, you can configure it directly on the sensor node. If you use an external power supply such as an LDO, the enable pin is controlled by a GPIO. You can refer to vcc_mipicsi1 to configure it. The power node can be powered on and off byreference counting, which is suitable for multiple devices using the same power supply.

It is recommended to supply the dvdd power supply separately when multiple cameras are used. Avdd/dovdd can be shared. When dvdd is shared, if the power draw is relatively large, there may be a momentary shortage of supply, which will affect the image quality and even fail to produce pictures.