Android Camera

VAR-SOM-MX6 - Camera

Overview

i.MX6 platforms are capable of handling multiple cameras.(With some restriction)
See page: 20 - 30 http://cache.freescale.com/files/training/doc/ftf/2014/FTF-CON-F0119.pdf
Android Application can leverage the use of multiple cameras for capturing the video frames.
VAR-SOM-MX6 / DART-MX6 / Solo-Dual have MIPI / Parallel camera support, available on the development board.

Hardware Availability and Setup

• VAR-SOM-MX6 Custom board:
  

CSI1 - MIPI Clock + 4 Data lanes are accessible via Carrier board header.
J17: MIPI - 4 Lane + CLK + I2C + GPIOs
CSI0 - Parallel camera pins are accesible via carrier board header.
J18: Parallel - 8 bit + I2C

For more details refer to: https://www.variscite.com/wp-content/uploads/2017/12/V2_VAR-MX6CustomBoard-Schematics.pdf
MIPI Camera Accessories Required: https://www.variscite.com/products/accessories/i-mx6-camera-board/

• VAR-DT6CustomBoard:

CSI1 - MIPI Clock + 2 Data lanes are accessible via carrier board header.
J13: MIPI - 2 Lane + CLK + I2C
CSI0 - Parallel camera pins are accesible via carrier board header.
J11: Parallel - 16 bit + I2C
For more details refer to: https://www.variscite.com/wp-content/uploads/2017/11/VAR-DT6CustomBoard-Schematics.pdf

• VAR-SOLOCustomBoard:

CSI1 - MIPI Clock +4 Data lanes are accessible via carrier board Headers. J12: MIPI 2 Lane + CLK + I2C
CSI0 - Parallel camera 8 bit pins are accesible via carrier board header.
J8: Parallel - 8 bit + I2C
For more details refer to: https://www.variscite.com/wp-content/uploads/2017/12/VAR-SOLOCustomBoard-Schematics.pdf

MIPI Camera Accessories Required: https://www.variscite.com/products/accessories/426-vcam-ov5640-v5/

For maximum, supported resolution and combinations refer to http://cache.freescale.com/files/training/doc/ftf/2014/FTF-CON-F0119.pdf
page no 21 for more details and combination.

Note: i.MX6 platform doesn't have the SOC based Image Signal Processing (ISP) unit to convert bayer data to RGB and YUV.
i.MX6 platform relies on MIPI camera to convert the Bayer data to YUV format.

Connect the relevant camera accessories to the right connector.

Software Setup

Make sure you have the required hardware as per above.

MIPI Cameras

By default, MIPI cameras are supported out of the box.
Android System / Vendor images contain Camera HAL already support for OV5640 5MP support available.
You don't need to do any special build/addition below HAL if you are using the standard camera.

• For Android 7.1.1 and 7.1.2:

Source Path: <android_build>/hardware/imx/libcamera3/

• For Android 8.0.0:

Source Path: <android_build>/vendor/nxp-opensource/imx/libcamera3/

Adding a parallel camera

To add a parallel camera support to the kernel:
As usual, make sure the driver for your device (camera) is included in the kernel configuration (use menuconfig), and define in the device tree the pins and driver to be used.
The device tree for VAR-SOM-MX6 is arch/arm/boot/dts/imx6qdl-var-som.dtsi
The device tree for DART-MX6 is arch/arm/boot/dts/imx6qdl-var-dart.dtsi
For example:

diff --git a/arch/arm/boot/dts/imx6qdl-var-dart.dtsi b/arch/arm/boot/dts/imx6qdl-var-dart.dtsi
index 7afa15e7813b..975c30233e87 100644
--- a/arch/arm/boot/dts/imx6qdl-var-dart.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-var-dart.dtsi
@@ -171,21 +171,28 @@
 		brightness-levels = <0 4 8 16 32 64 128 248>;
 		default-brightness-level = <7>;
 		status = "okay";
 	};
 
 	v4l2_cap_0 {
 		compatible = "fsl,imx6q-v4l2-capture";
 		ipu_id = <0>;
-		csi_id = <1>;
+		csi_id = <0>;
 		mclk_source = <0>;
 		status = "okay";
 	};
 
+	v4l2_cap_1 {
+		compatible = "fsl,imx6q-v4l2-capture";
+		ipu_id = <1>;
+		csi_id = <1>;
+		mclk_source = <0>;
+		status = "okay";
+	};
 
 
 	v4l2_out {
 		compatible = "fsl,mxc_v4l2_output";
 		status = "okay";
 	};
 };
 
@@ -222,17 +229,17 @@
 	
 	ov564x_mipi: ov564x_mipi@3c {
 		compatible = "ovti,ov564x_mipi";
 		reg = <0x3c>;
 		clocks = <&clks 200>;
 		clock-names = "csi_mclk";
 		pwn-gpios = <&gpio3 13 1>;
 		rst-gpios = <&gpio4 10 0>;
-		csi_id = <1>;
+		csi_id = <0>;
 		mclk = <24000000>;
 		mclk_source = <0>;
 	};
 };
 
 &i2c2 {
 	clock-frequency = <100000>;
 	pinctrl-names = "default";
@@ -266,16 +273,30 @@
 	pinctrl-0 = <&pinctrl_i2c3_3>;
 	status = "okay";
 
 	/* DS1307 RTC module */
 	rtc@0x68 {
 		compatible = "dallas,ds1307";
 		reg = <0x68>;
 	};
+
+	ov564x: ov564x@3c {
+		compatible = "ovti,ov564x";
+		reg = <0x3c>;
+		pinctrl-names = "default";
+		pinctrl-0 = <&pinctrl_ipu2>;
+		clocks = <&clks 200>;
+		clock-names = "csi_mclk";
+		pwn-gpios = <&gpio3 4 0>;
+		rst-gpios = <&gpio3 5 0>;
+		csi_id = <1>;
+		mclk = <24000000>;
+		mclk_source=<0>;
+	};
 };
 
 &iomuxc {
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_hog>;
 
 	imx6qdl-var-som-mx6 {
 
@@ -395,16 +416,35 @@
 				MX6QDL_PAD_DISP0_DAT19__IPU1_DISP0_DATA19	0x10
 				MX6QDL_PAD_DISP0_DAT20__IPU1_DISP0_DATA20	0x10
 				MX6QDL_PAD_DISP0_DAT21__IPU1_DISP0_DATA21	0x10
 				MX6QDL_PAD_DISP0_DAT22__IPU1_DISP0_DATA22	0x10
 				MX6QDL_PAD_DISP0_DAT23__IPU1_DISP0_DATA23	0x10
 			>;
 		};
 
+		pinctrl_ipu2: ipu2grp { /* parallel camera */
+			fsl,pins = <
+				MX6QDL_PAD_EIM_A17__IPU2_CSI1_DATA12		0x0000b0b1
+				MX6QDL_PAD_EIM_A18__IPU2_CSI1_DATA13		0x0000b0b1
+				MX6QDL_PAD_EIM_A19__IPU2_CSI1_DATA14		0x0000b0b1
+				MX6QDL_PAD_EIM_A20__IPU2_CSI1_DATA15		0x0000b0b1
+				MX6QDL_PAD_EIM_A21__IPU2_CSI1_DATA16		0x0000b0b1
+				MX6QDL_PAD_EIM_A22__IPU2_CSI1_DATA17		0x0000b0b1
+				MX6QDL_PAD_EIM_A23__IPU2_CSI1_DATA18		0x0000b0b1
+				MX6QDL_PAD_EIM_A24__IPU2_CSI1_DATA19		0x0000b0b1
+				MX6QDL_PAD_EIM_DA10__IPU2_CSI1_DATA_EN		0x80000000
+				MX6QDL_PAD_EIM_A16__IPU2_CSI1_PIXCLK		0x0000b0b1
+				MX6QDL_PAD_EIM_DA11__IPU2_CSI1_HSYNC		0x0000b0b1
+				MX6QDL_PAD_EIM_DA12__IPU2_CSI1_VSYNC		0x0000b0b1
+				MX6QDL_PAD_EIM_DA4__GPIO3_IO04			0x80000000
+				MX6QDL_PAD_EIM_DA5__GPIO3_IO05			0x80000000
+			>;
+		};
+
 		pinctrl_pwm1_1: pwm1grp {
 			fsl,pins = <
 				MX6QDL_PAD_DISP0_DAT9__PWM2_OUT		0x1b0b1
 			>;
 		};
 
 		/* Linux Console */
 		pinctrl_uart1_1: uart1grp-1 { /* RX/TX only */
@@ -593,17 +633,17 @@
 	fsl,phy_reg_vlev = <0x0294>;
 	fsl,phy_reg_cksymtx = <0x800d>;
 	status = "okay";
 };
 
 &mipi_csi {
 	status = "okay";
 	ipu_id = <0>;
-	csi_id = <1>;
+	csi_id = <0>;
 	v_channel = <0>;
 	lanes = <2>;
 };
 
 &pcie {
 	reset-gpio    = <&gpio4 11 0>;	/* gpio pin num of power good signal */
 	wake-up-gpio  = <&gpio4 31 1>;	/* gpio pin num of incoming wakeup signal */
 	disable-gpio  = <&gpio5 5 0>;	/* gpio pin num of outgoing rfkill/endpoint disable signal */
diff --git a/arch/arm/boot/dts/imx6qdl-var-som.dtsi b/arch/arm/boot/dts/imx6qdl-var-som.dtsi
index 043b9332af05..0edad80b22c5 100644
--- a/arch/arm/boot/dts/imx6qdl-var-som.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-var-som.dtsi
@@ -159,30 +159,38 @@
 		compatible = "pwm-backlight";
 		pwms = <&pwm2 0 50000>;
 		brightness-levels = <0 4 8 16 32 64 128 248>;
 		default-brightness-level = <7>;
 		status = "okay";
 	};
 
 	v4l2_cap_0 {
 		compatible = "fsl,imx6q-v4l2-capture";
 		ipu_id = <0>;
 		csi_id = <1>;
 		mclk_source = <0>;
 		status = "okay";
 	};
 
+	v4l2_cap_1 {
+		compatible = "fsl,imx6q-v4l2-capture";
+		ipu_id = <0>;
+		csi_id = <0>;
+		mclk_source = <0>;
+		status = "okay";
+	};
+
 	v4l2_out {
 		compatible = "fsl,mxc_v4l2_output";
 		status = "okay";
 	};
 };
 
 &audmux {
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_audmux_2>;
 	status = "okay";
 };
 
 &cpu0 {
 	arm-supply = <&sw1a_reg>;
 	soc-supply = <&sw1c_reg>;
@@ -297,30 +305,44 @@
 		>;
 	};
 };
 
 &i2c3 {
 	clock-frequency = <100000>;
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_i2c3_3>;
 	status = "okay";
 
 	/* DS1307 RTC module */
 	rtc@0x68 {
 		compatible = "dallas,ds1307";
 		reg = <0x68>;
 	};
+
+	ov564x: ov564x@3c {
+		compatible = "ovti,ov564x";
+		reg = <0x3c>;
+		pinctrl-names = "default";
+		pinctrl-0 = <&pinctrl_ipu1_2>;
+		clocks = <&clks 200>;
+		clock-names = "csi_mclk";
+		pwn-gpios = <&gpio3 4 0>;
+		rst-gpios = <&gpio3 5 0>;
+		csi_id = <0>;
+		mclk = <24000000>;
+		mclk_source=<0>;
+	};
 };
 
 &iomuxc {
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_hog>;
 
 	imx6qdl-var-som-mx6 {
 
 		pinctrl_hog: hoggrp {
 			fsl,pins = <
 				/* CTW6120 IRQ */
 				MX6QDL_PAD_EIM_DA7__GPIO3_IO07 		0x80000000
 				/* for Bluetooth/wifi enable */
 				MX6QDL_PAD_SD3_DAT6__GPIO6_IO18		0x1b0b1
 				/* SDMMC2 CD/WP */
@@ -448,30 +470,32 @@
 
 		pinctrl_ipu1_2: ipu1grp-2 { /* parallel camera */
 			fsl,pins = <
 				MX6QDL_PAD_CSI0_DAT12__IPU1_CSI0_DATA12		0x80000000
 				MX6QDL_PAD_CSI0_DAT13__IPU1_CSI0_DATA13		0x80000000
 				MX6QDL_PAD_CSI0_DAT14__IPU1_CSI0_DATA14		0x80000000
 				MX6QDL_PAD_CSI0_DAT15__IPU1_CSI0_DATA15		0x80000000
 				MX6QDL_PAD_CSI0_DAT16__IPU1_CSI0_DATA16		0x80000000
 				MX6QDL_PAD_CSI0_DAT17__IPU1_CSI0_DATA17		0x80000000
 				MX6QDL_PAD_CSI0_DAT18__IPU1_CSI0_DATA18		0x80000000
 				MX6QDL_PAD_CSI0_DAT19__IPU1_CSI0_DATA19		0x80000000
 				MX6QDL_PAD_CSI0_DATA_EN__IPU1_CSI0_DATA_EN	0x80000000
 				MX6QDL_PAD_CSI0_PIXCLK__IPU1_CSI0_PIXCLK	0x80000000
 				MX6QDL_PAD_CSI0_MCLK__IPU1_CSI0_HSYNC		0x80000000
 				MX6QDL_PAD_CSI0_VSYNC__IPU1_CSI0_VSYNC		0x80000000
+				MX6QDL_PAD_EIM_DA4__GPIO3_IO04			0x80000000
+				MX6QDL_PAD_EIM_DA5__GPIO3_IO05			0x80000000
 			>;
 		};
 
 		pinctrl_pwm2_1: pwm2grp {
 			fsl,pins = <
 				MX6QDL_PAD_DISP0_DAT9__PWM2_OUT		0x1b0b1
 			>;
 		};
 		/* Linux Console */
 		pinctrl_uart1_1: uart1grp-1 { /* RX/TX only */
 			fsl,pins = <
 				MX6QDL_PAD_CSI0_DAT10__UART1_TX_DATA	0x1b0b1
 				MX6QDL_PAD_CSI0_DAT11__UART1_RX_DATA	0x1b0b1
 			>;
 		};

Note: This is just an example. You should set everything according to your camera model and hardware design, and make sure the pins are not conflicting with other devices in the device tree.


In addition, you need to patch arch/arm/mach-imx/mach-imx6q.c to set the appropriate GPR bits for connecting the desired CSI to the parallel interface.
For example:

diff --git a/arch/arm/mach-imx/mach-imx6q.c b/arch/arm/mach-imx/mach-imx6q.c
index ae5c3cd29d04..c4def47886e5 100644
--- a/arch/arm/mach-imx/mach-imx6q.c
+++ b/arch/arm/mach-imx/mach-imx6q.c
@@ -229,31 +229,39 @@ put_node:
 	of_node_put(np);
 }
 
 static void __init imx6q_csi_mux_init(void)
 {
 	/*
-	 * MX6Q SabreSD board:
+	 * DART-MX6 board:
+	 * IPU2 CSI1 connects to parallel interface.
+	 * Set GPR1 bit 20 to 0x1.
+	 *
+	 * MX6Q SabreSD/VAR-SOM-MX6 boards:
 	 * IPU1 CSI0 connects to parallel interface.
 	 * Set GPR1 bit 19 to 0x1.
 	 *
-	 * MX6DL SabreSD board:
+	 * MX6DL SabreSD/VAR-SOM-MX6 boards:
 	 * IPU1 CSI0 connects to parallel interface.
 	 * Set GPR13 bit 0-2 to 0x4.
 	 * IPU1 CSI1 connects to MIPI CSI2 virtual channel 1.
 	 * Set GPR13 bit 3-5 to 0x1.
 	 */
 	struct regmap *gpr;
 
 	gpr = syscon_regmap_lookup_by_compatible("fsl,imx6q-iomuxc-gpr");
 	if (!IS_ERR(gpr)) {
-		if (of_machine_is_compatible("fsl,imx6q-sabresd") ||
-			of_machine_is_compatible("fsl,imx6q-sabreauto"))
+		if (of_machine_is_compatible("fsl,imx6q-var-dart"))
+			regmap_update_bits(gpr, IOMUXC_GPR1, 1 << 20, 1 << 20);
+		else if (of_machine_is_compatible("fsl,imx6q-sabresd") ||
+			of_machine_is_compatible("fsl,imx6q-sabreauto") ||
+			of_machine_is_compatible("fsl,imx6q-var-som"))
 			regmap_update_bits(gpr, IOMUXC_GPR1, 1 << 19, 1 << 19);
 		else if (of_machine_is_compatible("fsl,imx6dl-sabresd") ||
-			 of_machine_is_compatible("fsl,imx6dl-sabreauto"))
+			 of_machine_is_compatible("fsl,imx6dl-sabreauto") ||
+			 of_machine_is_compatible("fsl,imx6dl-var-som"))
 			regmap_update_bits(gpr, IOMUXC_GPR13, 0x3F, 0x0C);
 	} else {
 		pr_err("%s(): failed to find fsl,imx6q-iomux-gpr regmap\n",
 		       __func__);
 	}
 }

Refer to: Customizing the Linux kernel for detailed steps about kernel customization.

UVC Camera Setup

Change android_build/device/variscite/var_mx6/init.rc

setprop back_camera_name uvc

Change android_build/device/variscite/var_mx6/BoardConfig.mk

BOARD_HAVE_USB_CAMERA := true

Make sure your kernel have

$ CONFIG_USB_VIDEO_CLASS=y

For that add CONFIG_USB_VIDEO_CLASS=y to arch/arm/configs/ and make sure you have UVC / USB WebCam driver enabled.
Refer to: Customizing the Linux kernel for detailed steps about kernel customization.
You may be able to find the UVC class USB drivers here:https://github.com/varigit//tree//drivers/media/usb

Build your new boot-<setup-name>.img

Refer to: Customizing the Linux kernel for detailed steps about building kernel.

Flash your new boot-<setup-name>.img via fastboot and reboot device

Connect the target with host PC at fastboot mode:

1. Connect a USB OTG cable from the target board OTG port to a your host machine USB HOST port.
2. Power up the board and hit return/space to stop the boot at U-Boot.
3. type fastboot usb0 in the U-Boot command line.

On the Host PC:

$ sudo `which fastboot` flash boot out/target/product/var_mx6/boot-SETUP_NAME.img
$ sudo `which fastboot` flash recovery out/target/product/var_mx6/recovery-SETUP_NAME.img
$ sudo `which fastboot` flash system out/target/product/var_mx6/system.img
$ sudo `which fastboot` reboot

Replace SETUP_NAME with the actual desired setup name accrding to the second table in the "images created" section.

Testing Camera

If the camera is connected/configured correctly. Once you click the Android Camera Application mentioned blow.

The camera preview should start, like below

Click on the Camera icon, it will take JPEG snapshot.

If you wish to record/video, click on the leftmost side of the preview screen you will see moving UI.