| How many bits are really needed in the image pixels? |
by Andrey Filippov (Jan 23., 2009)
How many bits are really needed in the image pixels?
Modern
CMOS image sensors provide high resolution digital output of 10 to 12
bits. When CCD sensors are used, the same (or higher) resolution is
provided by separate ADC or integrated CCD signal processors. This
resolution is significantly higher than that used in popular image and
video formats. Most are limited to only 8 bits per pixel, or 8 bits per
color channel depending on the image format. To catch up with the
sensor and provide higher dynamic range, many cameras switch to formats
that support higher number of bits per pixels, and some of them use
uncompressed, full (sensor) dynamic range raw data. Such formats use
significantly more storage space, and consume more bandwidth for
transmission.
Do these formats always preserve more of the
information registered by the sensors? If the sensor has 12-bit digital
output, does that mean that when using 8-bit JPEG, the four least
significant bits are just wasted, and therefore a raw format is
required to preserve them?
In most cases the answer is "no".
With sensor technology advances, sensor pixels get smaller and smaller,
approaching the natural limit of the light wavelength. It is now common
to have pixels of less than 2x2 microns, as in the sensors used in
mobile phone cameras. One consequence is reduced Full Well Capacity (FWC) -- the maximal number of electrons that each pixel can accommodate without spilling them out. This is important because of the Shot Noise
-- variations of the number of electrons. (There is always an integer
number of them, as there can be no ½ electron). Shot noise is caused by
the quantum nature of electric charge itself. There is no way to
eliminate or reduce it for the particular pixel measurement. This noise
is proportional to the square root of the total number of electrons in
a pixel, so it is highest when the pixel is almost full.
Pixel output uncertainty caused by the shot noise
If
we try to keep track of the pixel value in ideal conditions, with the
same illumination, the same camera settings that measured pixel value
will change from frame to frame. The image above shows what happens
with a hypothetical sensor with an FWC of just 100. In real sensors,
the uncertainty is smaller, but it can still significantly reduce the
amount of received information. Our measurements show that a typical
Micron/Aptina 5 Megapixel sensor MT9P031 with 2.2x2.2 micron pixels
have FWC of ≈8500 electrons. As a result, when the sensor is almost
full, the pixel value would fluctuate as 8500±92 or more than±1%. Such
fluctuation corresponds to 44 counts of the 12 bit sensor ADC.
So
is there any real need for such high resolution ADC when some 5-6 LSBs
don't carry any real information? Yes, because modern sensors have a
very low readout noise, and in the dark, a single ADC count is
meaningful. The square root of zero is zero. (Actually, even in the
pitch black, there are some thermal electrons that reach the pixels,
even with no light at all.) With ADC counts having different
information payload for small and large signals. it is possible to
recode the pixel output equalizing the information values of the output
counts.
Non-linear conversion of the sensor output
This
non-linear conversion assigns incremental numbers to each pixel level
that can be distinguished from the previous one in a single
measurement, so for small signals in the dark. each next ADC output
value receives the next value, increasing to more than 40 ADC counts
per number for large signals. Such conversion significantly reduces the
number of output values, and therefore the number of bits required to
encode them, without sacrificing much of the pixel values. The form
below calculates the effective number of bits for different sensor
parameters and ratio of the encoder step to the noise value for that
output level.
Optimal encoding and gamma correction
Luckily nothing has to be done to utilize the non-linear encoding described above,
the encoding optimal for maintaining constant noise to output count ratio.
All cameras incorporate some kind of a gamma correction
in the signal path. Historically it was needed to compensate for the
non-linear transfer function of the electron guns used in CRT monitors.
Cameras had to apply a non-linear function so the two functions applied
in series (camera and display) were providing an image that was
perceived to have contrast close to that of the original. With LCD
displays, this is no longer required, but gamma correction (or gamma
compression) also does a nice job of transferring higher dynamic range
signal, even when the signal itself is converted into digital format.
Different standards use slightly different values for gamma, usually in
the range of 0.45-0.55 on the camera side. Compression with gamma=0.5
is exactly the same as the square root function shown above, optimal
for encoding in the presence of the shot noise. With this kind of gamma
encoding, a full well capacity of several hundred thousands electrons
is needed to have 12 bits of meaningful data per pixel in the image
file. Such high FWC values are available only in the CCD image sensors
with very large pixels.
Measuring the sensor full well capacity
Earlier,
I noted that our Aptina sensors have FWC≈8500e-. Such data was not
provided by the manufacturer; we measured it ourselves. The FWC value
is important because it influences the camera performance, so by
measuring the camera performance it is possible to calculate the FWC.
The same method may be applied to most other cameras, as well. First,
one needs to control the ISO settings (gain) of the camera and acquire
a long series of completely out of focus images. (Remove the lens if
possible; if not, completely open the iris and use a uniform target.)
ISO (gain) should be set to minimum, and exposure adjusted so that the
area to be analyzed has a pixel level close to maximal. Use natural
lighting or DC-powered lamps/LEDs to minimize flicker caused by AC
power. If it is a color sensor, use green pixels only (or with
incandescent lamps, red). This will provide the minimal gain. Then
measure the differences between the same pairs of pixels in multiple
frames. (We used 100 pairs in 100 frames.) Then find the root mean
square for the differences in each pair, and divide by the square root
of 2 to compensate for using differences in pairs, not the pixel
values. (Pairs make this method more tolerant to fluctuation of the
total light intensity and to some uncontrolled parameter changes in the
camera). If it is possible to turn off gamma correction, the ratio of
the pixel value to the measured root mean square of the variation will
be equal to the square root of the number of electrons. If the
gamma-correction cannot be controlled, one may assume it is around 0.5.
(Click here for further information)
|
|
|
FUEL Database on MontaVista Linux
Whether building a mobile handset, a car navigation system, a package
tracking device, or a home entertainment console, developers need
capable software systems, including an operating system, development
tools, and supporting libraries, to gain maximum benefit from their
hardware platform and to meet aggressive time-to-market goals.
Breaking New Ground: The Evolution of Linux Clustering
With
a platform comprising a complete Linux distribution, enhanced for
clustering, and tailored for HPC, Penguin Computing�s Scyld Software
provides the building blocks for organizations from enterprises to
workgroups to deploy, manage, and maintain Linux clusters, regardless
of their size.
Data Monitoring with NightStar LX
Unlike
ordinary debuggers, NightStar LX doesn�t leave you stranded in the
dark. It�s more than just a debugger, it�s a whole suite of integrated
diagnostic tools designed for time-critical Linux applications to
reduce test time, increase productivity and lower costs. You can debug,
monitor, analyze and tune with minimal intrusion, so you see real
execution behavior. And that�s positively illuminating.
Virtualizing Service Provider Networks with Vyatta
This paper highlights Vyatta's unique ability to virtualize networking
functions using Vyatta's secure routing software in service provider
environments.
High Availability Messaging Solution Using AXIGEN, Heartbeat and DRBD
This
white paper discusses a high-availability messaging solution relying on
the AXIGEN Mail Server, Heartbeat and DRBD. Solution architecture and
implementation, as well as benefits of using AXIGEN for this setup are
all presented in detail.
Understanding the Financial Benefits of Open Source
Will
open source pay off? Open source is becoming standard within
enterprises, often because of cost savings. Find out how much of a
financial impact it can have on your organization. Get this methodology
and calculator now, compliments of JBoss.
Embedded Hardware and OS Technology Empower PC-Based Platforms
The modern embedded computer is the jack of all trades appearing in many forms.
Data Management for Real-Time Distributed Systems
This
paper provides an overview of the network-centric computing model, data
distribution services, and distributed data management. It then
describes how the SkyBoard integration and synchronization service,
coupled with an implementation of the OMG�s Data Distribution Service
(DDS) standard, can be used to create an efficient data distribution,
storage, and retrieval system.
7 Advantages of D2D Backup
For
decades, tape has been the backup medium of choice. But, now,
disk-to-disk (D2D) backup is gaining in favor. Learn why you should
make the move in this whitepaper. |
|
|
|
|
news feed