Camera API Technical Report:
A Camera Model

A Camera Model

A digital camera samples light intensity over a physical area and produces an array of numbers. The physical area is discretised into a 2-D rectangular array which is mapped onto the output array by windowing and scaling. The digital values corresponding to the light intensity are modified by binning (summation of adjacent values, a transfer function (LUT), by the exposure which is an integer multiple of some time quantum and possibly by shade correction and a calibration function. The output from the camera is a series of numbers with a number of bits-per-pixel determined by the original bits-per-pixel and the transfer LUT. The data-rate may be settable.

To the program the camera appears as an input device with settable and read-only attributes. Typically the read-only attributes correspond to limits and constraints on the settable values. Some constraints may only be available as functions with the general model: request a value (a window) get back the nearest allowed window. The attributes can be considered as read-only (RO) or read-write (RW), my suggested minimum set is below. Many of these are encoded within the camera functions rather than the device driver.

I can think of two types of camera distinguished by whether the image is stored or not. We can refer to these as DMA or Frame-Buffer cameras. In some sense the frame-buffer camera is more general because it can mimic the DMA type - by setting ``autocapture''. All our cameras and frame-grabbers fit this model, the Sun VideoPix board is something of a hybrid but must be thought of as a DMA type - data is read from a single address which is internally incremented after each short word is read.

The videopix raises the problem of colour, the model proposed here is a purely monochrome camera. Extension to colour shouldn't be difficult but I'll leave it for another day.

Identity:

each camera will need some form of ID:

Attribute	I/O	Comments
name	RO	camera name
type	RO	camera type if name is insufficient
init_str	RO	string passed to the camera initialisation procedure
file descriptor	RO	device file descriptor for cases when access beyond the camera model is required
frame buffer	RO	the frame buffer pointer for direct access

Spatial Transform:

the sampled intensity values are mapped onto a rectangular array. Information under this category should be sufficient to understand that transform. Unfortunantly the pixel value transform gets linked to the spatial transform via binning. This just has to be accepted. I am assuming that users will want to read and write either the capture domain or the image domain, before or after scaling respectively. The user will have to be aware that the window and image are linked via the scaling and that constraints on allowed values are likely. A model for the constraints is too hard so I propose that we go allong the same lines as geometry management in Xt, which is to make a geometry request and be given a nearest available option.

The Spatial transform attributes are then:

Attribute	I/O	Comments
horizontal size	RO	Horizontal sampling point separation
vertical size	RO	Vertical sampling point separation
size units	RO	units (microns) of the pixel size
window limits	RO	maximum size of the capture window in pixels, note a window comprises four numbers, a top-left corner with either a bottom-right or width-height pair
binning options	RO	a list of permitted binning values
sampling options	RO	a list of permitted sampling values
bin value	RW	the current binning value
sampling value	RW	the current sampling value
window	RW	the current window
image domain	RW	the current output image domain

I suggest that we adopt a fixed and simple relationship between the window and image domain coordinates namely:

$$image\:coordinate = \frac{window\:coordinate}{scaling},$$

where $scaling = sampling \times binning$. Note in general we need to allow different binning and scaling in the two directions. The formula above has the problem of a many to one mapping from window to image, furthermore the digital round-off is not as you might expect. On the Xillix controller the image horizontal coordinates are rounded UP and in the vertical direction the rounding depends on the line parity as well as the sampling factor. By way of complication the camera binning always bins the same pixel values independently of the window. To avoid these complications I suggest that windows should be set by choosing the image domain and the sampling and binning factors and treat the window as read-only.

Value Transform:

the sampled intensity values are mapped into output values by binning and a LUT transfer function. In the future we may have hardware shade correction and optical density or transmission value calibration and would therefore expect these to be part of the camera library function. In our applications the shade images will vary even within colours (fluourescence) therefore I suggest we leave these as external functions.

Attribute	I/O	Comments
pixel depth	RO	input pixel depth: bits-per-pixel
gain options	RO	list of input gains available - this is gain in the amplifier before digitisation
input gain	RW	current input gain
image depths	RO	list of output pixel depth options
output depth	RW	bits-per-pixel of the output image
transfer LUT	RW	transfer LUT of length $2^{pixel\_depth}$ and providing appropriate values for the output depth. This will typically be byte or short.

Exposure:

the exposure time in all cameras I have come across is quantised. In the Xillix case the quantisation is with respect to the line frequency and therefore will vary with the data-rate. For this reason a requested exposure time will not be honoured exactly.

Attribute	I/O	Comments
exposure units	RO	time units ms
exposure limits	RO	limits on the settable exposure time (if any)
exposure	RW	current exposure time

Data Rate:

the output data rate can be set on a number of cameras. Typically the user will want this set at the maximum if the computing system can stand it.

Attribute	I/O	Comments
data rates	RO	data-rate options list
data rate	RW	current data rate

Extensions:

there will always be a need to extend the camera model. There should be a way of querying for extensions to find out what is available. An obvious starter is the live video option or simultaneous low and high resolution. In the first instance I want to be able to treat the Pulnix/Virtuoso within the standard model and so include the video extension here.

Attribute	I/O	Comments
video option	RO	is live video available
video overlay	RO	is a video display overlay available
video LUT size	RO	video output LUT - non zero if settable
live flag	RW	video live toggle
overlay flag	RW	overlay visibility toggle
overlay plane	RW	bitmask of overlay plane
overlay colour	RW	index of overlay colour
video LUT	RW	video output LUT