CamCyclopedia
What is multi-frame processing?
Multi-frame processing technology creates a single image by combining multiple images when the user hits the shutter button.
Smartphone cameras have smaller lenses and sensors compared to regular cameras (mirrorless). The sensors have a small size per pixel. As a result, lesser light reaches the aperture, noise is relatively higher, and the dynamic range to indicate light and shadow is narrower.
High-end smartphones are equipped with relatively large sensors, but portability is a huge factor in the design, so the sensor size is smaller than that of a regular camera.
To overcome the limitations caused by lens and sensor size in smartphone cameras, technology is applied to combine multiple images to create a single image. This is called multi-frame processing.
Effects of multi-frame processing
You can remove noise, get HDR effects and create super-resolution images with multi-frame processing.
Noise Cancellation
The noise produced by sensors is removed in image processing before the image file is saved. Image processing methods such as averaging surrounding pixels or selecting an intermediate value remove noise but simultaneously degrade the quality of the image.
Multi-frame processing technology both removes noise and maintains the image quality.
In the figure above, if you look at the pictures taken in succession, noise is constantly changing. Observe that the image on the right has a lot less noise. It is a result of combining multiple images.
This is how multi-frame processing combines multiple images to create a single image with less noise.
HDR (High Dynamic Range) Effect
Two or more images with different brightness can be combined to produce highlights and shadows with greater detail. (HDR: High Dynamic Range).
The feature compares the brightness determined by the camera's exposure control system (AE) for two or more images by capturing a half-dark, a darker, or a brighter image in succession.
The contrast in brightness may depend on the intensity of light or the photography setting.
Taking photos at night with outdoor backlight or any other light makes the bright areas saturated, which makes them appear white. But if you stack them with the dark images, the saturated areas are nullified and the photo looks good. Dark areas of an image, such as shadows can also be brightened by combining them with a brighter shot.
You can shoot an optimal HDR photo with this, depending on the combination of lights you use in different scenes.
You get better HDR photos, with more detailed highlights and shadows with the combination of multiple images that have different exposure values.
Creation of High-Resolution Images
You can also use multi-frame processing to create photos with high resolution.
‘Super resolution’ allows you to transform low-resolution photos into high resolution by using numerous images.
The effect of enlarging the photo (digital zoom) is on the bottom left in the above figure. But with multi-frame processing entering the scenario, it looks like the one on the bottom right.
It fixes the decline in image quality and removes noise from the photo when the user zooms in. It also sharpens the image definition and increases the image detail.
Photos with Movement
Multiple frame input requires the camera to shoot multiple images consecutively. There is some time difference between photographing one image and the next. If the camera or subject moves in this gap, it becomes difficult to combine the multiple frames.
OIS (Optimal Image Stabilization) technology and motion detection technology counter this problem by reducing the effect of handshaking.
Combining motion photos may result in the angle of the image narrowing or the sides becoming blurry. In the above case, the image with the most stacked shots can be cropped and the part with no image overlap can be blurred.
Multi-Frame Processing Technology in Galaxy
The Galaxy camera automatically applies multi-frame processing technology to remove noise and implement the HDR effect according to the scenery shot on the camera.
You can press the 1x in the preview frame or the below 30x button to use the high-power zoom shot.
The Galaxy S22 Ultra uses 20x zoom and above, and the S22 uses 10x zoom and above to create high-resolution photos
Multi-frame processing also applies to the ‘Nightography’ mode, used to shoot in the dark.
You can take a ‘Nightography’ shot when you spot the moon icon in photo mode or the timer in night mode.
The night mode shows you the time left to shoot, in the night mode. The shot frame goes through multi-frame processing where the noise and brightness are optimized before the image is saved.
The Expert RAW app available in the Galaxy Store also uses multi-frame processing. Samsung is also collaborating to make multi-frame processing available on third-party camera apps.
AI based on deep learning improves image quality (Deep-Learning driven image quality-enhancing AI Te... - Samsung Members) and is used together with multi-frame processing when an image is processed using the later.
CamCyclopedia Index - Samsung Members
You can also access CamCyclopedia anytime by going to Community -> Category (app) -> CamCyclopedia -> “CamCyclopedia Index”.
Reference
[1] Night Solution Youtube (https://youtu.be/Sjm6CcBFnp4?t=49)
[2] Some of the above examples are virtual images with graphic effects applied for illustrative purposes.
3 Comments
