MSU Video Codecs Comparison. Frequently Asked Questions

Frequently Asked Questions

• Q: Why do you use exactly PSNR metric in the comparison?

  A: There are several important reasons for it:

  • First, PSNR is THE ONLY GENERALLY ACCEPTED metric nowadays used for codecs comparison. To become certain of that you may look through any scientific articles or drafts of the new standards (H.264, for example). So using PSNR measures provides clear and unique interpretation by any professional in the codecs scope. Actually we use special non-standard metrics in our researches. Moreover we create metrics ourselves. But if we presented all the measures in some "MSU metric-3" it would far not improve clearness and value of the comparison.
  • Second, along with having several drawbacks PSNR can be calculated rather quickly. This is very essential, because all other metrics that more or less reflect visual perception need much more time for calculation (up to dozens of times more). We've spent 11 days to perform all the calculations for the first test using PSNR and we are not ready yet to spend something like 110 days on it :) Of course it is possible to reduce the number of the testing video sequences but it would make the adequacy of the estimation worse.
  But considering the fact that computers are becoming more and more powerful we are planning to add new non-standard metrics to our comparison. So have patience!

• Q: Why don't you consider some certain types of video in the comparison?

  A: We haven't had time to do it yet. Till this moment we've covered with tests only different types of DVD ripped movies and have left animation and video received from tuner without attention. We'll try to increase the variety of tested videos if we find time and free computers to perform calculations.

• Q: You've mistakes in the comparison!

  A: Yes, this could have happened. Please be so kind and report about them to ! We'll appreciate it a lot.

• Q: You should have performed the comparison in another way!

  A: Your approach might also make sense, but we don't have a possibility now to implement it. Try doing it yourself! We can even place your results on our site if they prove to be good.

• Q: There are some graphs for some sequences in the comparison but there are no same graphs for another sequence. Can I somehow get to see them?

  A: Alas, if we had placed graphs for ALL the measures that we had done for the comparison, we would have got a difficult-to-read 1000-paged document. That's why we used only a subset of all the graphs in the comparison. But we have all of them in our laboratory and we can share them and other additional information that hasn't been included in the comparison with those people who in their turn help us. These are, for example, the companies that had provided their codecs for the testing or people who have sent a lot of remarks. We won't be sending anything for nothing, we have no time for it! We are busy working on the new projects and performing new tests.

• Q: Why being dated September 2004 did the first test appear much later?

  A: The first test contained too much information and it took us long to study the results, and to choose what to include in the final document. However we're going to release further results with some delay as well. But this time the reason will be different: we want to give companies that have provided us with their codecs a chance to fix bugs that we find in the codecs during the testing. (We find bugs rather frequently during the testing and report about them to the codec's producer.)

• Q: It's all clear to me, you've described all the codecs in details, thanks a lot! But which codec is the best after all?

  A: Thanks for your warm words :) But the question "what codec is the best" is not correct and the comparison clearly illustrates this fact. Different codecs with different settings provide best results for different type of video. There are no any universal codecs that would outperform all other codecs for any given video sequence. We propose just a detailed report of the testing which other companies usually provide only for money. All the conclusions you should make yourself.

Other materials

Video resources:

Public MSU video filters Here are available VirtualDub and AviSynth filters. Commonly we develop a whole family of some kind of a filter. Generally there are also versions optimized for PC and hardware implementations (ASIC/FPGA/DSP). These optimized versions can be licensed to companies. Please contact us for details via video(at)graphics.cs.msu_ru. MSU Cartoon Restore (New!) MSU Noise Estimation MSU Frame Rate Conversion MSU Image Restoration MSU Denoising (Top!) (New!) MSU Old Cinema MSU Deblocking (Top!) MSU Smart Brightness and Contrast (Top!) MSU Smart Sharpen (Top!) MSU Noise generation MSU Noise estimation MSU Motion Estimation Information MSU Subtitles removal MSU Logo removal (Top!) MSU Deflicker (Top!) MSU Field Shift Fixer AviSynth plug-in MSU StegoVideo MSU Cartoonizer (Top!) MSU SmartDeblocking (Top!) MSU Color Enhancement MSU Old Color Restoration MSU filters FAQ (Read me!) MSU filters statistics	MSU filters for companies We are working with Intel, Samsung, RealNetworks and other companies on adapting our filters other video processing algorithms for specific video streams, applications and hardware like TV-sets, graphics cards, etc. Some of such projects are non-exclusive. Also we have internal researches. Please let us know via video(at)graphics.cs.msu_ru if you are interested in acquiring a license for such filters or making a custom R&D project on video processing, compression, computer vision. Automatic Objects Segmentation Semiautomatic Objects Segmentation New Frame Rate Conversion New Deinterlacer MSU-Samsung Deinterlacing Project Digital TV Signal Enhancement Old Film Recovery Tuner TV Restore Panorama Video2Photo SuperResolution SuperPrecision MSU-Samsung image and video resampling MSU-Samsung Frame Rate Conversion Motion Phase filter Deshaker (video stabilization) Film Grain/Degrain filter Deblurring filter Video Content Search
Codecs comparisons Objective and subjective quality evaluation tests for video and image codecs Options Analysis of Codec x264 (New!) 4-th MPEG4-AVC/H.264 Comparison (Top!) Lossless Video Codecs Comparison 2007 3-rd MPEG4-AVC/H.264 Comparison HD Photo and JPEG 2000 Comparison MPEG-2 Decoders Crash Test (Top!) Subjective Compar. of 4 Modern Codecs (Top!) 2-nd MPEG-4 AVC/H.264 Comparison (Top!) FAQ of 2-nd H.264 Comparison 1-st MPEG-4 AVC/H.264 Codecs Comparison FAQ of 1-st H.264 Comparison JPEG 2000 Image Codecs Comparison (Top!) Lossless Video Codecs Comparison 2004 MPEG-4 SP/ASP Codecs Comparison (Top!) Video codecs comparison (old) Ext. link: x264 parameters efficiency comparison	Video quality metrics Programs with different objective and subjective video quality metrics implementation MSU Video Quality Metric - objective metrics for codecs and filters comparison (Top!) Implemented metrics short info: PSNR, Delta, MSAD, MSE, SSIM, VQM, MSU Blurring, MSU Blocking MSU Brightness Flicking Metric MSU Brightness Independent PSNR MSU Drop Frame Metric MSU Noise Estimation Metric MSU VQM Metric Tool FAQ MSU Human Perceptual Quality Metric - several metrics for exact visual tests (Top!) Implemented metrics short info: ITU-R BT.500-11: DSIS, DSCQS I & II, SCACJ; EBU: SAMVIQ; MSUCQE MSU PVQ Metric Tool FAQ
Video codecs projects Different research and development projects on video codecs MSU Lossless Video Codec (Top!) MSU Screen Capture Lossless Codec (Top!) MSU MPEG-2 Video Codec x264 Codec Improvement	Other Other information Crazy gallery (filters screams :) License for commercial usage of MSU VideoGroup Public Software (please be careful: some soft like metrics has another license!)

Bookmark this page:   Add to Del.icio.us     Digg It     reddit
 

Last updated: 12-October-2007

---

Search (Russian): Server size: 7629 files, 938Mb (Server statistics)

Project updated by
Server Team and MSU Video Group

Project sponsored by YUVsoft Corp.

Project supported by MSU Graphics & Media Lab