Compression project >> Video Area Home

Algorithm, ideas: Dr. Dmitriy Vatolin
Algorithm, implementation: Sergey Grishin

FRC (Frame Rate Conversion) algorithms are used in compression, video format conversion, quality enhancement, stereo vision, etc. The most popular application is format conversion. This is the case when FRC is used in order to convert the frame rate of video stream. It is needed for example in order to playback 50Hz video sequence using TV set with 100Hz frame rate. FRC makes the motion of objects smoother and therefore more pleasant for eyes. It allows to slow down the playback speed thus making the objects' movements more evident.

Pic.1 Basic scheme of FRC

FRC algorithm increases the total number of frames in the video sequence. This is performed by inserting new frames (interpolated frames) between each pair of neighbor frames of original video sequence (see pic.1). The number of interpolated frames between each pair of original frames is defined by the interpolation factor. Interpolation factor is a user defined parameter and can be equal to any positive integer number.

Main advantage of developed algorithm is using of several quality enhancement techniques such as adaptive artifact masking, black stripe processing and occlusion tracking:

• adaptive artifact masking technique allows to make artifacts less noticeable for eyes thus increasing the integral quality of processed video;
• black stripe processing allows to avoid artifacts which are commonly appeared in interpolated frames in case of black stripe presented near frame edges;
• occlusion tracking performs high quality restoration of interpolated frames near edges in case of presence of motion with direction to/from the frame edge.

Examples

This section contains performance results of developed algorithm and its comparison with methods of other companies.

First example (pic. 2-4) demonstrates result obtained using 'schumacher' test video sequence. Interpolated frame (see pic. 4) is calculated by developed algorithm using two reference frames (pic. 2,3). Presented interpolated frame located in the centre position in time domain between reference frames.

Pic.2 Previous reference frame

Pic.3 Next reference frame

Pic.4 Interpolated frame

Pic.5 Previous reference frame

Pic.6 Next reference frame

Pic.7 Retimer result	Pic.8 Motion Perfect result
Pic.9 Twixtor result	Pic.10 AFRC result

Next example shows performance result for test video sequence 'foreman'. Interpolated frames are obtained during x1 conversion (sequence is firstly decimated with factor 2 and decimated frames are then interpolated) of input video stream. The number of interpolated frame in output video sequence is 171.

Pic.11 Previous reference frame

Pic.12 Next reference frame

Pic.13 Retimer result	Pic.14 Motion Perfect result
Pic.15 Twixtor result	Pic.16 AFRC result

Next diagram (see pic. 17) demonstrates the results of objective comparison. The objective quality of processed sequences for various methods was measured using Y-PSNR. During PSNR calculation only interpolated frames had been used. In order to do that original video sequences are first decimated with factor 2 and then decimated frames are recovered using FRC. After that interpolated frames are compared with frames from original video sequences using Y-PSNR metric.
Vertical axis is marked with average Y-PSNR values for each sequence, horizontal one - by test sequences' names. As it can be clearly seen the developed algorithm (AFRC) shows the best objective quality result.

Pic. 17 Objective comparison result

Publications

• D.Vatolin, S.Grishin, "N-times Video Frame-rate Up-conversion Algorithm based on Pixel Motion Compensation with Occlusions Processing", "Graphicon", International Conference on Computer Graphics & Vision, conference, July 2006, pp. 112-119. (Russian)
• D.Vatolin, S.Grishin, "Video Frame Rate Conversion Method Based on Compensated Frames Interpolation", "New Information Technologies in Automated Systems", Seminar at M.V.Keldysh Institute of Applied Mathematics, March 2006, pp. 32-46. (Russian)

Download

For commercial license of this filter please contact us via

• Previous FRC project: MSU FRC Project
• Public FRC filter: MSU Frame Rate Conversion

Another resources

Video resources:

Public MSU video filters Here are available VirtualDub and AviSynth filters. Commonly we develop a whole family of some kind of a filter. Generally there are also versions optimized for PC and hardware implementations (ASIC/FPGA/DSP). These optimized versions can be licensed to companies. Please contact us for details via video(at)graphics.cs.msu_ru. MSU Cartoon Restore (New!) MSU Noise Estimation MSU Frame Rate Conversion MSU Image Restoration MSU Denoising (Top!) (New!) MSU Old Cinema MSU Deblocking (Top!) MSU Smart Brightness and Contrast (Top!) MSU Smart Sharpen (Top!) MSU Noise generation MSU Noise estimation MSU Motion Estimation Information MSU Subtitles removal MSU Logo removal (Top!) MSU Deflicker (Top!) MSU Field Shift Fixer AviSynth plug-in MSU StegoVideo MSU Cartoonizer (Top!) MSU SmartDeblocking (Top!) MSU Color Enhancement MSU Old Color Restoration MSU filters FAQ (Read me!) MSU filters statistics	MSU filters for companies We are working with Intel, Samsung, RealNetworks and other companies on adapting our filters other video processing algorithms for specific video streams, applications and hardware like TV-sets, graphics cards, etc. Some of such projects are non-exclusive. Also we have internal researches. Please let us know via video(at)graphics.cs.msu_ru if you are interested in acquiring a license for such filters or making a custom R&D project on video processing, compression, computer vision. Automatic Objects Segmentation Semiautomatic Objects Segmentation New Frame Rate Conversion New Deinterlacer MSU-Samsung Deinterlacing Project Digital TV Signal Enhancement Old Film Recovery Tuner TV Restore Panorama Video2Photo SuperResolution SuperPrecision MSU-Samsung image and video resampling MSU-Samsung Frame Rate Conversion Motion Phase filter Deshaker (video stabilization) Film Grain/Degrain filter Deblurring filter Video Content Search
Codecs comparisons Objective and subjective quality evaluation tests for video and image codecs Options Analysis of Codec x264 (New!) 4-th MPEG4-AVC/H.264 Comparison (Top!) Lossless Video Codecs Comparison 2007 3-rd MPEG4-AVC/H.264 Comparison HD Photo and JPEG 2000 Comparison MPEG-2 Decoders Crash Test (Top!) Subjective Compar. of 4 Modern Codecs (Top!) 2-nd MPEG-4 AVC/H.264 Comparison (Top!) FAQ of 2-nd H.264 Comparison 1-st MPEG-4 AVC/H.264 Codecs Comparison FAQ of 1-st H.264 Comparison JPEG 2000 Image Codecs Comparison (Top!) Lossless Video Codecs Comparison 2004 MPEG-4 SP/ASP Codecs Comparison (Top!) Video codecs comparison (old) Ext. link: x264 parameters efficiency comparison	Video quality metrics Programs with different objective and subjective video quality metrics implementation MSU Video Quality Metric - objective metrics for codecs and filters comparison (Top!) Implemented metrics short info: PSNR, Delta, MSAD, MSE, SSIM, VQM, MSU Blurring, MSU Blocking MSU Brightness Flicking Metric MSU Brightness Independent PSNR MSU Drop Frame Metric MSU Noise Estimation Metric MSU VQM Metric Tool FAQ MSU Human Perceptual Quality Metric - several metrics for exact visual tests (Top!) Implemented metrics short info: ITU-R BT.500-11: DSIS, DSCQS I & II, SCACJ; EBU: SAMVIQ; MSUCQE MSU PVQ Metric Tool FAQ
Video codecs projects Different research and development projects on video codecs MSU Lossless Video Codec (Top!) MSU Screen Capture Lossless Codec (Top!) MSU MPEG-2 Video Codec x264 Codec Improvement	Other Other information Crazy gallery (filters screams :) License for commercial usage of MSU VideoGroup Public Software (please be careful: some soft like metrics has another license!)

Bookmark this page:   Add to Del.icio.us     Digg It     reddit
 

Last updated: 12-October-2007

---

Search (Russian): Server size: 7629 files, 938Mb (Server statistics)

Project updated by
Server Team and MSU Video Group

Project sponsored by YUVsoft Corp.

Project supported by MSU Graphics & Media Lab