Consumer Digital Still HD video shoutout.
If you can get a consumer camera that shoots HD for just a couple hundred bucks, why not load up on the cameras and get multiple angles of an event for next to no cost. Plus, you can move them around easily, perch them in unusual places and you don’t need a half-dozen video camera operators. Sounds great, doesn’t it?
Well, the reality is that the rolling shutter CMOS image distortion in video cameras is just as prevalent in digital still cameras. You can easily see it when you bounce the camera up and down lightly, or pan the camera side to side. Things that you naturally do when you are recording video with the camera in your hands instead of on a tripod. These motions distort the image from what really exists in reality. Camera flashes are partially bad- partially illuminating multiple frames. When you play that back, it looks completely unnatural.
To quantify these CMOS distortions, I secured two brand new digital still cameras that shoot HD video and pitted them side by side in some critical tests and the results clearly demonstrate the difference between CMOS and CCD when it comes to capturing video that faithfully represents what happened.
The two contestants in this competition are Canon’s brand-spankin’ new, 10 MP, CMOS-based, SX1 IS. Previously only available outside of the USA, it has garnered rave reviews for the image quality of its full 1920x1080p30 HD video.
This camera is nearly identical to the 10 MP, CCD-based, SX10 IS. However, even though there are more than enough pixels, the CCD model does not record any flavor of HD; only 640x480p30 SD. (Note: Since this article was written for EventDV several months prior to publishing, Canon has released a CCD-based SX20 which does shoot 720p30 video.)
The competitor in this test is Canon’s even newer, 12 MP, CCD-based, SX200 IS. The “IS” in these camera model names means “image stabilized.” Since all the cameras are IS, I will omit that part of the model name from the rest of the article.
The SX200 has a completely revamped menu system and records 1280x720p30 HD video. It is not as high resolution as the “full HD” SX1, but you can decide if accurate video is more important than the number of pixels per frame.
Camcorders that use CMOS image sensors often use a technology called a “rolling shutter.” This exposes different parts of the frame at different times until the entire frame is fully exposed.
If the camcorder is moved before the entire frame is fully exposed, the resulting image may appear distorted. Sometimes, video stabilization [in editing software] may make this distortion more apparent. Videos that show signs of distortion due to your camcorder’s rolling shutter may not be suitable for use with video stabilization [in editing software].
So there’s image stabilization in the cameras, and there’s image stabilization in the editing programs, but neither can fix the distorted images that CMOS chips record. (NOTE: in the interim from when this article was written, The Foundry has developed software specifically for fixing CMOS image bending due to rolling shutter. It does not fix partial frame exposures from a flash strobe.) How bad is it? What does it look like? There are individual examples of the distortion out on the web, but I haven’t seen head-to-head tests with two similar cameras with different chips. So this is what I tested.
I wanted to test three things… panning distortion, tilting distortion, and flash distortion. And by distortion, I mean a distortion from reality- not that the image gets messed up in and of itself, although I have previously reported examples of CMOS chips exhibiting that on the web. I wanted to test these distortions head-to-head, CCD to CMOS. Same panning speed. Same tilting speed. Same motion in front of the camera with the flash going off.
To do this, I mounted both cameras to one tripod head with several Monfrotto clamps. This way, I moved the handle of the tripod and both cameras had the same exact panning speed and motion. I could not favor one camera over the other in any way. They would have the same tilting speed. They would be inches from each other as a flash illuminates my little daughter as she walks around the house.
Almost all the images you see here are frame grabs from the video. They are all full resolution and have only been converted to JPEG for the web and not retouched in any way. The first image of the vertical stripes is a digital still from the SX1, not video. Images of the cameras themselves were taken by another camera- a Canon S2 IS, 5 MP, CCD.
I set up two tall white strips on a brown wall. This would clearly illustrate if the motion blur was horizontal, or diagonal. I panned in both directions, at multiple speeds and I found that the distortion is evident at every speed.
The rolling shutter of the CMOS chip does not “snap” a frame, it gathers the image a row of pixels at a time until it has gathered the entire chip. There is no problem if here is no movement in front of the camera. However, if what is in front of the camera moves, or the camera moves, whatever you are trying to record with video is in a different place by the time the camera gets around to recording pixel row 2, and different again when it records pixel row three, etc. Compile those errors row after row and you get images that bear no resemblance to reality.
While some may assert that it’s motion and people aren’t paying attention to it, the reality is that we to notice when something leans forward and back, bends left and right, no matter if the camera is panning or not. Moreover, this distortion is also evident on things that move within the frame- a car driving by, cyclists, running, golf swings, kids playing soccer, animals’ legs moving, etc. Anything that moves is distorted by some amount. The amount of movement determines the amount of distortion.
In the images, you can see that panning the CCD across the white vertical panels is sharp on the top and bottom edge, and the sides are blurry, but vertical. This shows motion, but properly represents the vertical panels. The CMOS takes the vertical panels and turns them into different diagonals depending on the panning direction of the camera. So the mind sees this moment as white panels that are leaning, which is not what is really there.
So you can clearly see that the CMOS chip is “slowly” gathering the image from top to bottom. As the camera pans to the right, the vertical white strip moves to the left. Over the course of scanning the CMOS chip for the frame, the white vertical strip almost moved one complete width sideways. Move back and forth a couple times and you’ll really question whether those strips are honestly vertical.
Another issue is that the CMOS rolling shutter distortion is not limited to horizontal panning. Vertical tilting is also affected. To demonstrate this, I tilted the cameras up and down on the same strips. For this section, I did do a bit of image editing: I combined an image from when the cameras were not tilting with the cameras tilting up and down. This way you can more easily see if there was any sort of distortion from reality.
You can see the CCD camera has blurry edges top and bottom, but the size of the wall remains the same as reality. Interestingly, the white band seems to have become longer, while the dark brown wall has become shorter. This is because the white band throws more light into the camera than the darker wall. I spit the image down the middle and you can compare the left and right bands on the first image to see their comparative similar length.
The CMOS comparison is much more interesting. When the camera tilts down, the wall is moving up, while the CMOS chip is scanning down rows. As it descends a row, the wall is now further up, till it prematurely reaches the bottom of the wall. This makes the wall look several feet shorter than it is.
Conversely, by tilting up, the wall moves down the image, and as the CMOS chip scans down the rows, it finds the same piece of wall over and over again. This seemingly extends the wall several feet taller than it is. The center image is the same wall, from the same position, the same zoom. I compiled this image by taking the left white band and sliding it to the left. The right band and sliding it to the right, revealing both bands in the static image for comparison. This can be verified by the amount of tree evident over each white band.
No matter tilting or panning, CMOS chips clearly distort the reality they are being counted on to capture.
For this test I enlisted the assistance of my 1 year old daughter to walk around and show me the things she likes to play with while I carried the dual camera mount in one hand, and fired a separate camera flash, to simulate video you might record while other people are taking pictures. The flash was not connected or related to either of the two digital cameras in any way.
Here I will show you the frame before the flash, and then the frame(s) of the flash. This is just one moment of time represented in still images. There is a bit of horizontal offset between the two cameras because the subject is only three feet from the two cameras. This does not affect the test.
The CCD image represents the flash as a full-on blinding blanket of light.
As many of us have experienced, that is a very accurate representation of how a flash feels and looks in reality, including the temporary blindness that you get.
In this case, two separate CMOS frames show the strobe. It is the bottom part of the first frame, then the top part of the second frame. Does this mean that the strobe fired twice, once for the first 1/30th of a second, and then a little bit more another 1/30th a second later? That’s what the frames of video say. If you look closely, you will also see that there is a portion of the subject that is never illuminated by the strobe (her top three fingers holding the bowl). How does that happen in reality?
Moreover, this particular CMOS division of the image is not repeated. Each time, it is random. This abberation is also not limited to digital still cameras. I’ve reported here previously how the CMOS-based Sony EX1 series has the same issue with flashes. If it has a CMOS imager, without internal correction, it has a problem. Even RED. Top, bottom, a single or two frames, partial, etc. It does do “bottom-top” several times which looks very weird on video, like it is backwards, and that there were two flashes.
This footage would be especially troublesome to watch if you slowed it down- a typical effect used heighten the emotional effect of video. Imagine your subjects coming in to a room, arms up, cheering the crowd. You want to slow down this shot for a video montage. Well, those partial flashes will now last much longer, and their random nature will become very visible.
The video clip here shows more clearly how the CMOS chip misrepresented the various flashes. For event video where there will be flashes going off, this is a serious concern. If your CMOS camera will be handheld, following motion (like dancing) while pictures are being taken, you can understand how the images you capture will not accurately represent what was seen.
In comparison, a CCD chip faithfully captures motion- both fast and slow. It also properly represents flashes of light to be the single, momentary, blowouts that they are.
When selecting a camera to capture precious moments for posterity, the images you record will be all that are left, years / decades from now. Do you want them to faithfully represent the people and action that were in front of you? Or are you looking to creatively add something to your video. CMOS distortions can be used creatively- provided you know what will happen and it is your desire to deliberately make images like that.
As CMOS chips are embedded in more and more consumer and pro gear year after year, I sincerely hope the manufacturers figure out how to correct the chip’s inerrant rolling shutter distortions so we can record undistorted images. Camera manufacturers already use software to correct for chromatic aberrations from inexpensive lenses focusing light onto ever tinier chips. Software can do amazing things. Perhaps engineers will figure out how to gather the data from a CMOS chip all at once, like a CCD. Let us keep our fingers crossed.
9/9/09, updates to this article will be made next week to provide video samples.