.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later .. _media_subdev_config_model: Sub-device configuration models =============================== A sub-device configuration model specifies in detail what the user space can expect from a sub-device in terms of V4L2 sub-device interface support, including IOCTL (including selection targets and controls) semantics. A sub-device may implement more than one configuration model at the same time. The implemented configuration models can be obtained from the sub-device's ``V4L2_CID_CONFIG_MODEL`` control. .. _media_subdev_config_model_common_raw_sensor: Common raw camera sensor model ------------------------------ The common raw camera sensor model defines a set of enumeration and configuration interfaces (formats, selections etc.) that cover the vast majority of funcitionality of raw camera sensors. Not all of the interfaces are necessarily offered by all drivers. A sub-device complies with the common raw sensor model if the ``V4L2_CONFIG_MODEL_COMMON_RAW`` bit is set in the ``V4L2_CID_CONFIG_MODEL`` control of the sub-device. The common raw camera sensor model is aligned with :ref:`media_using_camera_sensor_drivers`. Please refer to that regarding aspects not specified here. Each camera sensor implementing the common raw sensor model exposes a single V4L2 sub-device. The sub-device contains a single source pad (0) and two or more internal pads: an image data internal pad (1) and optionally an embedded data pad (2). Additionally, further internal pads may be supported for other features, in which case they are documented separately for the given device. This is shown in :ref:`media_subdev_config_model_common_raw_sensor_subdev`. .. _media_subdev_config_model_common_raw_sensor_subdev: .. kernel-figure:: common-raw-sensor.svg :alt: common-raw-sensor.svg :align: center **Common raw sensor sub-device** Routes ^^^^^^ A sub-device conforming to common raw camera sensor model implements the following routes. .. flat-table:: Routes :header-rows: 1 * - Sink pad/stream - Source pad/stream - Static (X/M(aybe)/-) - Mandatory (X/-) - Synopsis * - 1/0 - 0/0 - X - X - Image data * - 2/0 - 0/1 - M - \- - Embedded data Some devices do not support the embedded data stream, others do support it and in some of the latter, it can be turned on and off before streaming is started. Sensor pixel array size, cropping and binning ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The sensor's pixel array is divided into one or more areas. The areas around the edge of the pixel array, usually one or more sides, may contain optical black pixels, dummy pixels and other non-image pixels. The entire pixel array size is conveyed by the format on (pad, stream) pair 1/0. A rectangle within the pixel array contains the visible pixels. Capturing the non-visible pixels outside the visible pixel area may be supported by the sensor. The visible pixel area corresponds to the ``V4L2_SEL_TGT_CROP_DEFAULT`` selection target on (pad, stream) pair 1/0. Sensors can perform multiple operations that affect the output image size. First of these is the analogue crop. Analogue crop limits the area of the pixel array which the sensor will read, affecting sensor timing as well. The granularity of the analogue crop configuration varies greatly across sensors: some sensors support only a few different analogue crop configurations whereas others may support anything divisible by a given number of pixels. The analogue crop configuration corresponds the ``V4L2_SEL_TGT_CROP`` selection target on (pad, stream) pair 1/0. The default analogue crop rectangle corresponds to the visible pixel area. In the next step, binning is performed on the image data read from camera sensor's pixel array, as determined by the analogue crop configuration. Enabling binning will effectively result in an image smaller than the original by given binning factors horizontally and vertically. Typical values are 1/2 and 1/3 but others may well be supported by the hardware as well. Sub-sampling follows binning. Sub-sampling, like binning, reduces the size of the image by including only a subset of samples read from the sensor's pixel matrix, typically every n'th pixel horizontally and vertically, taking the sensor's colour pattern into account. Sub-sampling is generally configurable separately horizontally and vertically. Binning and sub-sampling are configured using the ``V4L2_SEL_TGT_COMPOSE`` rectangle, relative to the analogue crop rectangle, on (pad, stream) pair 1/0. It depends on the driver which of these operations are being used to achieve the resulting size. Binning and sub-sampling are also directly configured using :ref:`V4L2_CID_BINNING ` and :ref:`V4L2_CID_SUBSAMPLING_HORIZONTAL and V4L2_CID_SUBSAMPLING_VERTICAL ` controls on drivers that support them. The digital crop operation takes place after binning and sub-sampling. It is configured by setting the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair 0/0. The resulting image size is further output by the sensor. The digital scaling operation is performed after the digital crop. It is configured by setting the ``V4L2_SEL_TGT_COMPOSE`` rectangle on (pad, stream) pair 0/0, relative to the digital crop. The sensor's output mbus code is configured by setting the format on the (pad, stream) pair 0/0. The width and height fields are used to configure post-scaler digital crop, affecting the right side and the bottom of the frame. Drivers may only support some of even none of these configurations, in which case they do not expose the corresponding selection rectangles. If any selection targets are omitted, the further selection rectangle or format is instead related to the previous implemented selection rectangle. For instance, if the sensor supports binning but not analogue crop, then the binning configuration (``V4L2_SEL_TGT_COMPOSE`` selection target) is done in relation to the visible pixel area (``V4L2_SEL_TGT_CROP_DEFAULT`` selection target). Also refer to :ref:`Selection targets `. .. flat-table:: Selection targets on pads :header-rows: 1 * - Pad/Stream - Selection target/format - Mandatory (X/-) - Modifiable (X/-) - Synopsis * - 1/0 - Format - X - \- - Image data format. The width and the height fields indicates the full size of the pixel array, including non-visible pixels. The media bus code of this format reflects the native pixel depth of the sensor. * - 1/0 - ``V4L2_SEL_TGT_CROP_DEFAULT`` - X - \ - The visible pixel area. This rectangle is relative to the format on the same (pad, stream). * - 1/0 - ``V4L2_SEL_TGT_CROP`` - \- - X - Analogue crop. Analogue crop typically has a coarse granularity. This rectangle is relative to the format on the same (pad, stream). * - 1/0 - ``V4L2_SEL_TGT_COMPOSE`` - \- - X - Binning and sub-sampling. This rectangle is relative to the ``V4L2_SEL_TGT_CROP`` rectangle on the same (pad, stream). Binning is configured using the :ref:`V4L2_CID_BINNING ` control and sub-sampling is configured using the :ref:`V4L2_CID_SUBSAMPLING_HORIZONTAL and V4L2_CID_SUBSAMPLING_VERTICAL ` controls on drivers that support these controls. To configure binning and sub-sampling on drivers that do not support these controls, the selection rectangle may be changed directly to configure the combined effect on the image size. * - 2/0 - Format - X - \- - Embedded data format. * - 0/0 - ``V4L2_SEL_TGT_CROP`` - \- - X - Digital crop. This rectangle is relative to the ``V4L2_SEL_TGT_COMPOSE`` rectangle on (pad, stream) pair 1/0. * - 0/0 - ``V4L2_SEL_TGT_COMPOSE`` - \- - X - Scaling. This rectangle is relative to the ``V4L2_SEL_TGT_CROP`` rectangle on (pad, stream) pair 0/0. * - 0/0 - Format - X - X - Image data source format and post-scaler crop. The width and height fields of the format, used to configure post-scaler crop on the right and bottom edges of the image, are related to the ``V4L2_SEL_TGT_COMPOSE`` rectangle on (pad, stream) pair 0/0. The media bus code reflects the pixel data output of the sensor. * - 0/1 - Format - X - \- - Embedded data source format. Embedded data ^^^^^^^^^^^^^ The embedded data stream is produced by the sensor when the corresponding route is enabled. The embedded data route may also be immutable or not exist at all, in case the sensor (or the driver) does not support it. Generally the sensor embedded data width is determined by the width of the image data whereas the number of lines are constant for the embedded data. The user space may obtain the size of the embedded data once the image data size on the source pad has been configured. Also see :ref:`media_using_camera_sensor_drivers_embedded_data`.