Menu
Foreword by Founder Mark Rejhon
William Sokol Erhard writes about multiple methods of blur busting, including flicker-based methods (BFI, CRT, strobing) and flickerless methods (framerate-based, framegen-based, etc) of display motion blur reduction. He built an app on Steam called Vint, a motion blur-reducing video processor with BFI/CRT and framegen options. Here is his article in its entirety. We encourage you to support his blur-reducing software!
Introduction
With moving images, there is simple math. The more images you show in a given time, the higher the framerate, results in less time that each frame persists and less motion blur you get as well as a clearer and smoother resulting image.
This is the fundamental principle of Blur Buster’s Law, also covered in other articles at Blur Busters:
When you can increase the framerate of the media you consume and the games you play, you get to enjoy all the benefits of this higher fidelity motion. Unfortunately, some games are locked to lower framerates, other games simply run poorly, and you don’t generally get to choose the framerate of movies or TV that you watch.
Fortunately, when you are limited to a given source framerate, whatever the cause, you can still improve the motion clarity and overall temporal fidelity of the image in two distinct ways.
I am William Sokol Erhard, an experienced AR/VR engineer and the creator of the Vint temporal upscaling software. about.willse.me This article will discuss when and how to use frame generation or interpolation as well as flicker or strobing methods to reduce image persistence and improve motion clarity.
First, let’s look at a bit of light theory.
Theory
Strobing is an effective method of motion blur reduction that has been used for many decades and is even an intrinsic effect of certain displays like CRTs. Strobing is known by many names and comes in many forms. You may know it as black frame insertion or BFI, ‘Lightboost’, ‘ULMB’, ‘ELMB’, ‘DyAc’, ‘OLED Motion’, ‘Gsync Pulsar’, or a variety of other terms.
While strobing very effectively brute force reduces persistence and therefore motion blur, it has several significant drawbacks.
First, strobing often causes flicker and lots of it. In order to avoid flicker you have to run above the flicker fusion threshold which for most people and situations is between 90Hz and 120Hz. Any strobing below that can cause significant discomfort and fatigue. 24FPS or 30FPS video for instance will flicker intensely with normal black frame insertion (BFI) strobing.
You also need to run your content with a Vsync lock. Unlike regular Vsync, you can not, for example, run at 60FPS on a 120Hz strobed display without a resulting double projection artifact. This multiple projection artifact essentially looks like two separate overlaid and slightly offset images. If your content drops ever below the strobing rate you get even more significant stutter than normal Vsync drops.
Strobing also requires near perfect strobe pacing. Any interference in the output of even a 120Hz strobe will result in unpleasant changes in brightness that are essentially flickers themselves.
Theoretically, strobing can be done with very little additional latency but most implementations require Vsync and often incur other significant added latency on the display.
Ultimately, if you are able to avoid each of these pitfalls, you can get a great experience with little computational overhead or image alteration.
Frame generation or interpolation is an alternative approach that seeks to replicate a high framerate output by filling in the gaps of a low framerate source with educated guesses.
This approach can rely on motion vector information from a specific video game like AMD’s FSR 3.0 frame generation (FG), Nvidia’s DLSS 3.0 FG, or Intel’s XeSS-FG. Alternatively, it can rely solely on the RGB image like very low quality built in TV interpolation, low-intermediate quality GPU video encoder accelerated optical flow interpolation, or high quality depth aware machine learning interpolation like RIFE or cinema industry video editing solutions.
Motion vectors help inform interpolation and increase output quality but when rendering video games you are limited in available computation power and latency requirements. Motion vector based framegen also requires support to be explicitly built into the game by the developer.
RGB image interpolation is far more flexible. It can be used with any content as long as the images can be processed by the interpolation tool. For realtime content, interpolation can focus on latency and low overhead. Alternatively, for movies and TV or streamed video, interpolation can still run in realtime but at a far higher level of quality using more system resources. Finally, interpolation can be run offline ahead of time to get the highest possible quality that can be widely distributed
Ideally, to minimize latency and artifacts in the image which could manifest as minor flicker, interpolation works best with video sources above the flicker fusion threshold. That said, more computationally demanding interpolation can produce good results from low framerate sources.
Interpolation necessarily adds some latency but it can be quite low in optimal circumstances.
Augmented and virtual reality (AR and VR) headsets implement both strobing and frame generation (often in the form of reprojection) to great effect. Comment below if you are interested in further writing about how those systems work.
Now let’s look at how we can process video for either strobing or interpolation:
Acceleration
There are four types of normal implementations for either strobing or interpolation. In order, we start with more flexibility and less acceleration, and we end with less flexibility and more acceleration.
Image processing can occur:
- In software on the CPU
- In software with acceleration on the GPU
- Uncommonly with FPGA devices in specialized devices
- In hardware on the display or TV using ASICs.
CPU software can run on any existing system and offers incredible settings and customization. Unfortunately this approach is not fast enough for realtime processing and is essentially incompatible with games because it conflicts with system resource usage.
Video editing tools use the CPU for image processing when GPU acceleration isn’t practical.
GPU accelerated software is most common. The GPU is very well suited to image processing and offers accelerators to scale and manipulate images, create optical flow maps useful for interpolation from the video encoding engine, and has immense bandwidth for high resolution processing in addition to programmability that allows for customized processing behavior.
Video game framegen as well as hardware accelerated interpolation and scaling software will run on the GPU. The GPU is complex and has many different levels of acceleration.
https://commons.wikimedia.org/wiki/File:Altera_StratixIVGX_FPGA.jpg
FPGA (field-programmable gate array) devices are very expensive reprogrammable processors. They are very specialized and low sales volume devices that can often cost between $700 and $1000 that can offer low latency and limited custom behavior for specialized purposes.
Finally, ASICs (application-specific integrated circuits) are hardcoded processors that can do a single predetermined task quickly, efficiently, and cheaply. They are only cost effective at massive scale so they are only made for mass market devices like TVs or monitors. Any display based strobing system or TV based very low quality interpolation will be running on an ASIC.
These devices can not offer significantly customizable behavior or specialized functions.
Now let’s look at flicker and framegen in praxis.
In Practice
Here we’ll focus on what options you have to strobe and interpolate/framegen and how best to use them.
Strobing:
Strobing is a very old process and can be an integral and inherent aspect to the base functionality of certain display types. Most modern displays however like LCDs and OLEDs operate without any strobing. A subset of these do offer a strobing mode, generally for gaming purposes like you’d find on a gaming monitor. Virtual Reality displays for example, both OLED and LCD, nearly universally have low persistence strobed displays in part because of the work of Chief Blur Buster Mark Rejhon.
Finally, strobing can be added in a video stream using ‘Black Frame Insertion’ where the display acts as though those inserted black frames are regular images of video. This can be done with lower latency using an FPGA solution like the RetroTink or on existing hardware with more control and customizability with a software solution like Retroarch for emulation or Vint for video.
https://commons.wikimedia.org/wiki/File:PWM_duty_cycle_with_label.gif
The term ‘Duty Cycle’ is key to understanding the effect of strobing. The term refers to the percentage of the time the image is displayed as a portion of the display strobing frequency. For example, if the image persistence on a 250Hz display is 1ms, you have a 25% duty cycle. The lower the duty cycle percentage, the lower the brightness of the image and the lower the motion blur of movement in the image. A 100% duty cycle is simply a ‘sample and hold’ full persistence image without strobing.
First lets start with the two disparate analogue formats of old, film and video. The former usually for movies and the latter for live TV or TV/video tape recordings. These formats largely dictated the method of display.
Film projection:
Old film projectors operate much the same way that film cameras do. They use a single 180° rolling shutter to hide the movement of the film itself in addition to reducing image persistence. At low framerates like 24Hz, this causes immense flicker so most film projectors instead use multiple rotating shutters to increase the strobing rate while the base image only changes 24 times per second.
Engineer Guy explains how film projectors strobe.
This multiple projection is an effective way to minimize flicker but it causes a multiple projection artifact on any motion being displayed.
This film type of strobing is broadly similar to modern black frame insertion techniques and is easiest to replicate with software. Modern TVs like LG OLEDs or OLED gaming monitors often offer BFI display options with limited controls. These displays add a moderate amount of latency and often allow a single video rate equal to half of the maximum refresh rate and outputting a 50% duty cycle strobed image.
Alternatively, FPGA solutions like the RetroTink offer more but still limited Black Frame Insertion (BFI) options with similar latency to display solutions.
https://www.retrotink.com/shop/retrotink-4k
Finally, software solutions like RetroArch for emulation or Vint for video offer more customizability for BFI without the need for additional or special hardware but they do add some latency. Vint even offers support for variable refresh rate on freesync/gsync displays and custom BFI ratios so you have more control over the duty cycle you want. (Brightness vs clarity) Vint also offers multi-projection options to replicate the film projector shutter strobing and reduce flicker.
Black Frame Insertion (BFI):
Black frame insertion is a feature in some displays where individual refresh cycles are blacked out. A great visual demonstration of this is at testufo.com/blackframes.
CRT Beam Scanout:
The other analogue display format is video displayed on a CRT. A beam would scan across a raster pattern where only a single point would be hit by the electron beam and those phosphors would light up instantly. A trail following the beam would decay in brightness over the course of about five milliseconds.
The Slo-mo Guys recorded great footage of CRT scan-out.
This would generally result in at least some part of the screen being illuminated at any given time but any specific point on that screen being illuminated for less than 5ms every 16.7ms or a roughly 30% duty cycle. This change based on the brightness and pattern of content displayed but you could expect very roughly a 70% reduction in motion blur for an image displayed on a CRT at 60FPS vs an instantly refreshing full persistence OLED at equivalent brightness.
The strobing effect of CRTs of course can also result in multi-projection artifacts just like film projection if the content is strobed/scanned out multiple times per image.
While some retro purists still use CRTs, using actual hardware has become quite impractical, especially in recent years. Lack of new production, rising prices of old stock, hardware/brightness degradation over time, and new hardware making analogue outputs far more difficult have all made physical CRT displays less viable than ever.
The only alternative to replicate it then is to use software emulation. Fortunately, Blur Busters is working on the cutting edge of CRT emulation and has created new shaders that replicate CRT scanout and phosphor decay on suitably high refresh rate displays. blurbusters.com/crt
The Blur Buster CRT beam simulator is currently implemented in RetroArch for emulation and in Vint for video playback. Vint provides all CRT beam simulation features including simulated phosphor decay and even VRR support so, for example, a 50FPS video can be played back perfectly on a 240Hz gsync display at 4x (200Hz output).
Backlight Strobing:
The most common strobing technique you’ll find in gaming displays is backlight strobing, also covered in the Strobe Backlight FAQ.
This technique is especially important for LCD monitors to mitigate their very slow pixel response times. Turning off the backlight during a refresh cycle can help hide that pixel change process. LCDs, despite their numerous drawbacks, also often have higher brightness ceilings than OLEDs. This higher brightness allows them to strobe at a shorter persistence/lower duty cycle than OLEDs. Backlight strobing is key to the low persistence required for VR displays. While OLEDs don’t have a backlight, some OLED displays and many VR headsets operate similarly with lower persistence and higher refresh rates than normal BFI implementations.
Backlight strobing fundamentally requires hardware support from the display. Backlight strobing is limited to a fixed refresh rate and is therefore entirely incompatible with variable refresh rate technologies (freesync/gsync). Nvidia gsync pulsar is intended to support both strobing and VRR simultaneously but does not appear any closer to launch than it was when it was announced well over a year ago. VR headsets use fallback reprojection to prevent egregious stutter from the rendering rate falling below the refresh rate.
The features and performance of backlight strobing vary enormously depending on the exact implementation so expert reviews from the likes of RTings, TFTCentral, and precious few others are critical before purchase. Some implementations offer adjustable persistence which allows you to balance brightness/flicker intensity, and motion clarity. Some also offer strobing at different configurable refresh rates, though not with variable refresh rates.
Some displays include a strobe utility to fine-tune the strobe backlight, such as BenQ Strobe Utility and ViewSonic Strobe Utility.
Currently backlight strobing implementations still offer the lowest persistence display possible though the mass availability of OLED gaming monitors has quickly started to contest that crown.
If you’ll allow me to editorialize, with the availability of 500Hz OLED monitors offering 2ms persistence, LCDs make little sense for the modern motion clarity aficionado. OLEDs offer near instant pixel response times more than an order of magnitude faster than the fastest LCDs which eliminates ghosting in addition to offering perfect contrast, viewing angles, and colors.
More Frame Rate:
Doubling frame rate will halve motion blur, and quadrupling frame rate will eliminate 75% of display motion blur, assuming you have enough refresh rate & you have instant pixel response (GtG=0). This is covered in Mark’s article, 120 Hz vs 480 Hz OLED is more visible than 60 Hz vs 120 Hz LCD.
It shows that geometric frame rate differences are easy to achieve for fast 2D graphics motion on OLED, much like 1/120sec camera shutter versus 1/480sec camera shutter. This even applies to mudane non-gaming situations such as browser smooth-scrolling/panning.
Unfortunately, achieving such high frame rates for advanced 3D gaming graphics is very challenging for GPUs. This is where alternative methods of generating high frame rates, such as framegen, for those who may prefer this approach to eliminating display motion blur.
Frame Generation:
Frame generation technologies offer the promise of matching the low persistence of strobed displays without any of the nasty side effects of flicker, crosstalk, or multiple image artifacts. In theory, perfect framegen can replicate the motion characteristics of a native high framerate.
In practice however, framegen can often come with vexing artifacts of its own. Framegen necessarily requires processing power to work, limiting available performance for video game rendering. Unlike strobing techniques, framegen also alters the image presented so generated frames can sometimes diverge from the original content in unpleasant ways. One of the biggest benefits of framegen is that they can often improve motion fidelity in situations where other solutions aren’t available. For example, in games that are CPU limited, framegen uses GPU resources to smooth the image that would have otherwise gone unused. Framegen can create a smoother experience in games with an FPS cap and video interpolation can make low framerate content far more palatable, especially in fast motion.
blurbusters.com/framegen
Before we dive into what framegen options are available, if you are interested in how framegen works in principle check out the Blur Busters article I wrote on the subject previously.
Framegen options are generally delineated by low latency video game solutions and higher latency video options. Universally, these solutions are interpolation based as opposed to extrapolation. This means they use information from both preceding and succeeding frames which adds some latency.
Motion Vector Framegen:
Video game framegen in the form of AMD’s FSR 3.0 FG, Nvidia’s DLSS 3.0 FG, or Intel’s XeSS-FG use motion vector information provided by the game itself. This requires that the game explicitly implement framegen support which means users or modders can’t add it to unsupported games after the fact. It does however allow the interpolation to use that motion vector information to minimize distracting artifacts. These solutions combine motion vector information with optical flow information provided by the GPU hardware video encoding unit or similar acceleration. This approach minimizes added latency but results in a low quality interpolated result which has noticeable artifacts in fast motion where there are large differences between sequential frames. FSR 3 FG for example actually stops interpolating altogether during moments of high image movement resulting in an even larger perceived drop in fluidity.
DLSS 4.0 MFG is interesting because unlike previous iterations, it uses more general purpose tensor core acceleration to create optical flow information rather than video encoding acceleration hardware. Tensor cores are more flexible than the video encoder hardware so results can be better but require more processing time. Because latency and performance overhead are still at a premium, even tensor accelerated optical flow can only be expected to perform marginally better than video encoder optical flow based solutions like DLSS 3.0 FG.
Digital Foundry made an excellent video analysis of DLSS FG.
Framegen Without Motion Vectors:
If your game doesn’t offer framegen support with motion vectors through one of these proprietary solutions, you still have options. AMD offers AMD Fluid Motion Frames (AFMF) which is a driver level framegen solution that does not use motion vectors and instead relies solely on optical flow from the hardware video encoder. AFMF works on Radeon 6000 series GPUs and newer. This should work on nearly every game or application but has even more artifacts than motion vector based framegen. Nvidia recently announced NVIDIA Smooth Motion which, like DLSS 4.0 MFG, is accelerated using tensor hardware instead of the video encoding optical flow accelerator.
Nvidia Smooth Motion is currently limited exclusively to 5000 series RTX GPUs. Finally, if you don’t want a hardware limited solution you can use Lossless Scaling’s framegen feature (LSFG). LSFG relies solely on generic video encoder optical flow acceleration hardware which means it generally has even more artifacts and worse performance than hardware exclusive or motion vector based solutions. That said, it works on nearly every piece of hardware, supports nearly every software or game, and even offers adaptive interpolated upscaling to a set target rather than a multiple of the rendered framerate. Unfortunately, Intel does not currently have a driver level framegen option that doesn’t require game integration. Hopefully that will change in the near future.
https://store.steampowered.com/app/993090/Lossless_Scaling/
For either motion vector assisted or driver level game framegen I would highly recommend starting from the highest framerate you can. The best experience requires starting from a base framerate at or above the flicker fusion threshold. If you can get 100FPS before framegen, you’re probably going to get a good experience with it enabled. That means you’re also going to want a 240Hz or higher refresh rate display to get the best experience.
Reprojection (Warping):
While not explicitly framegen, reprojection is a very relevant technology in this context. AR and VR headsets use reprojection both to reduce latency from head movement and to mitigate stutter from frame drops. On desktop, Nvidia recently announced Reflex 2.0 which unlike Reflex 1.0 does not simply reduce rendering latency.
Reflex 2.0 actually warps the frame after it is rendered based on mouse or controller input just like VR reprojection. This allows the reprojection to offset any added latency from framegen in addition to some of the base rendering latency. It seems quite likely that AMD and Intel will offer alternative reprojection implementations in the future. Reprojection, like motion vector information, necessarily requires game specific support and cannot practically be added at the driver level.
High Frame Rate Video:
Unfortunately, video interpolation and high framerate video in general has gotten a bad rep. Cheap soap operas filmed at 60 fields per second interlaced as well as poor quality TV based interpolation with bad latency and horrible artifacts has soured their perception for many.
Fortunately, there are some positive developments in the area of high framerate video. High framerate (HFR) has been championed by some major cinema players. James Cameron, Peter Jackson, Ang Lee, Andy Serkis, and the late Douglas Trumbull have all done substantial work in high framerate film. Trumbull, known for his work on special effects in 2001: A Space Odyssey and the original Blade Runner, developed the showscan process in the late 70’s which recorded films on 70mm film at 60FPS. HFR has also become critical for VR video content with 3D180 content generally being 60FPS and Apple’s very well received Immersive Video format using 90FPS 3D180 video.
High framerates are critical now more than ever because many developments in the consumption of film and video have allowed us to see images and therefore their motion more clearly than ever. Back in the early 1900’s a small, blurry, noisy, and dim projected image filmed on a stationary camera can be played back without flicker and motion blur destroying the experience.
In the intervening century, film and video has gotten much sharper with higher resolutions and better lenses, cameras are light and films have lots of panning and movement, projectors and TVs have gotten much brighter especially with HDR, movie theaters and TVs have gotten massive and take up a large part of our FOV, and VR pushes all of that much further. All of those factors require higher framerates to compensate just to keep the same relative motion experience as old films. That says nothing of the fact that we should be pushing film motion fidelity technology forward just as hard as we have with resolution, visual effects, and everything else.
High framerates are functionally the same development as standard definition 480p TV to high definition widescreen. There was a substantial pushback at the time by purists as well. Many decried the fact that you could make out cheap and unconvincing props and effects. The recording process evolved and TV has become a premiere storytelling platform. Hopefully we can soon see the same evolution with motion fidelity as we have with static spatial fidelity.
In the meantime, what can we do?
Video Interpolation:
Yes, some video interpolation is straight up terrible. I can never recommend the poor quality built in TV interpolation or even some of the simpler MEMC software like MVTools that uses CPU or limited GPU acceleration. There are some basic optical flow based options that are not significantly better but they do run more efficiently.
There are however some advanced depth-aware machine-learning based interpolation algorithms that can provide high quality high framerate output with minimal artifacts. The following GIFs show the depth map and the generated output based on just two source frames made by one of these ML interpolation algorithms.
First I need to highlight the importance of the fact that video sources like movies or TVs start with a 30FPS or even 24FPS framerate which is far far below the framerate that video games tend to run at on PC. This lower source framerate results in far larger differences in content between successive frames and the interpolation engine must be much more effective to provide reasonable results. Video sources also lack any motion vector information to help reduce artifacts.
This is a pretty tall order so traditionally, high quality interpolation generally required a high performance compute server and it would take many times the length of the video to process. This would be used to restore video for films like Peter Jackson has done with old WWI footage and to create slo-mo footage from regular 24FPS recordings. Consumer options have become available and are integrated into video editing software like DaVinci Resolve, Adobe Premiere, and Final Cut Pro among others. These options, however, are often expensive and slow.
Fortunately there are better options available today. While reconstructing missing information, in this case temporal resolution, can never be as good as capturing more information at the source, modern depth-aware machine-learning based interpolation provides surprisingly good results. These ML interpolation techniques benefit directly from the rapid development of tensor processing performance on modern GPUs.
One option, often used by multi-deca-millionaires for their six or seven figure home theater setups, is the $16,000 MadVR Envy Extreme MK2 video processing box. These devices offer a range of video upscaling and processing features including ‘MotionAI’ which allows for high quality ML based interpolation in realtime. Interestingly, MadVR recommends interpolating 24FPS movies to 30FPS or 36FPS to get less flicker but avoid smooth (non-cinematic) playback.
Maybe you’re looking for a software solution you can use on your own devices instead. Let’s look at some other tools. Some of these solutions are focused quality at the cost of processing time like Google’s Frame Interpolation for Large Motion.
Others, like DAIN (Depth-Aware Video Frame Interpolation), are faster but still far too slow for realtime on the fly processing. Only one solution provides this level of high quality interpolation but can be accelerated using tensor acceleration to run in realtime, RIFE (Real-Time Intermediate Flow Estimation). Keep in mind, even using the best hardware tensor acceleration available with the TensorRT API, RIFE still requires substantial GPU power and even an RTX 3080 can realistically only upscale 1080p video from 24FPS by a three times multiplier to 72FPS (or 2.5x to 60FPS) in realtime, though the quality is exceptional.
The Nvidia 4000 and 5000 series GPUs have made massive strides in tensor performance far outpacing their raster or raytracing gains. The same RTX 3080 level of tensor performance can be found in the RTX 4060 mobile laptop GPU. The RTX 5070 desktop can be expected to interpolate 4k footage up to 60FPS or 1080p up to 240FPS in realtime. The RTX 5090 can do about 4k up to 240FPS.
The only software solution available that offers BFI and realtime interpolation using this high quality depth-aware ML interpolation is Vint.
Vint:
In this context, I developed the video processing software ‘Vint’ to offer both strobing options like BFI and CRT beam scanout simulation, and super high quality RIFE depth-aware ML interpolation in realtime using GPU tensor acceleration. Vint supports playback from all variety of local video files and streams as well as online sources like YouTube and even from UVC capture cards.
Vint offers offline transcoding for super high resolution videos or later playback on mobile devices. Vint is on Steam and has no DRM or online licensing requirements and is entirely user customizable. Vint requires no admin privileges and is fully modular without external dependencies. Vint supports freesync/gsync, RTX video and RTX HDR upscaling, as well as realtime HDR video interpolation.
Vint just released the version 1.1.0 update with many quality of life improvements. Feel free to contact me with any questions or support requests at [email protected]
Thanks for reading. I hope you got some useful information from this article. Please let me know what you’re interested in learning about next.
Tell me which of these motion busting tools you use, how they’re working for you, and which new ones you want to try out.
-William Sokol Erhard (Available for hire)
NOTE: Blur Busters receives no commission from the sales of this software. We are supporting the endeavours of an indie software developer that has released excellent blur busting software.