Sub-device configuration models

A sub-device configuration model specifies in detail what the user space can expect from a sub-device in terms of V4L2 sub-device interface support, including but not limited to IOCTLs, selection rectangles and controls.

A sub-device may implement more than one configuration model at the same time. The implemented configuration models can be obtained from the sub-device’s V4L2_CID_CONFIG_MODEL control.

Common raw camera sensor model

The common raw camera sensor model defines the configurability of a superset that covers the vast majority of raw camera sensors. Not all of the configuration and enumeration interfaces are offered by all drivers.

A sub-device complies with the common raw sensor model if the V4L2_CONFIG_MODEL_COMMON_RAW bit is set in the V4L2_CID_CONFIG_MODEL control of the sub-device.

The common raw camera sensor model is aligned with Using camera sensor drivers. Please refer to that regarding aspects not specified here.

Each camera sensor implementing the common raw sensor model exposes a single V4L2 sub-device. The sub-device contains a single source pad (0) and two or more internal pads: an image data internal pad (1) and optionally an embedded data pad (2). Additionally, further internal pads may be supported for other features, in which case they are documented separately for the given device.

This is show in Common raw sensor sub-device.

Common raw sensor sub-device

Routes

A sub-device conforming to common raw camera sensor model implements the following routes.

Routes

Sink pad/stream	Source pad/stream	Static (X/M(aybe)/-)	Mandatory (X/-)	Synopsis
1/0	0/0	X	X	Image data
2/0	0/1	M		Embedded data

Some devices may support enabling and disabling the embedded data stream. Others may not support it at all, in which case the embedded data route does not exist.

Sensor pixel array size, cropping and binning

The sensor’s pixel array is divided into one or more areas. The areas around the edge of the pixel array, usually one one or more sides, may contain optical black pixels, dummy pixels and other non-image pixels.

A rectangle within the pixel area contains the visible pixels. Capturing the non-visible pixels may be supported by the sensor.

The sensor can perform three operations that affect the output image size. First comes analogue crop. This configuration limits parts of the pixel array which the sensor will read, affecting sensor timing as well. The granularity of the analogue crop configuration varies greatly across sensors: some sensors support a few different analogue crop configurations whereas others may support anything divisible by a given number of pixels.

The default analogue crop rectangle corresponds to the visible pixel area if supported by the hardware.

In the next step, binning is performed on the image data read from camera sensor’s pixel array. This will effectively result in an image smaller than the original by given proportions horizontally and vertically. Typical values are 1/2 and 1/3 but others may well be supported by the hardware as well.

The combination of the analogue crop and binning operations may result in an image size that may be larger than desirable. For this purpose, a digital crop operation may be performed on the binned image. The resulting image size is further outputted by the sensor.

Selection targets on pads

Pad/Stream	Selection target/format	Mandatory (X/-)	Synopsis
1/0	Format	X	Image data format. The width and height fields of this format are the same than those for the V4L2_SEL_TGT_CROP_BOUNDS rectangle. The media bus code of this format reflects the native pixel depth of the sensor.
1/0	V4L2_SEL_TGT_NATIVE_SIZE	X	The full size of the pixel array, including all pixels in the pixel array, even if they cannot be captured. This rectangle is relative to the format on the same (pad, stream).
1/0	V4L2_SEL_TGT_CROP_BOUNDS	X	The crop rectangle bounds. No pixels outside this area can be captured. This rectangle is relative to the V4L2_SEL_TGT_NATIVE_SIZE rectangle on the same (pad, stream).
1/0	V4L2_SEL_TGT_CROP_DEFAULT	X	The visible pixel area. This rectangle is relative to the V4L2_SEL_TGT_NATIVE_SIZE rectangle on the same (pad, stream).
1/0	V4L2_SEL_TGT_CROP	-	Analogue crop. Analogue crop typically has a coarse granularity. This rectangle is relative to the V4L2_SEL_TGT_NATIVE_SIZE rectangle on the same (pad, stream).
1/0	V4L2_SEL_TGT_COMPOSE	-	Binning. This rectangle is relative to the V4L2_SEL_TGT_CROP rectangle on the same (pad, stream).
2/0	Format	X	Embedded data format.
0/0	V4L2_SEL_TGT_CROP	-	Digital crop. This rectangle is relative to the V4L2_SEL_TGT_COMPOSE rectangle on (pad, stream) pair 1/0.
0/0	Format	X	Image data source format. The width and height fields of the format are the same than for the V4L2_SEL_TGT crop rectangle on (pad, stream) pair 0/0 where as the media bus code reflects the pixel data output of the sensor.
0/1	Format	X	Embedded data source format.

Embedded data

The embedded data stream is produced by the sensor when the corresponding route is enabled. The embedded data route may also be immutable or not exist at all, in case the sensor (or the driver) does not support it.

Generally the sensor embedded data width is determined by the width of the image data whereas the number of lines are constant for the embedded data. The user space may obtain the size of the embedded data once the image data size on the source pad has been configured.

Also see Embedded data.