Ultra HFR: 240fps Real Time Video Now Possible Today. 1000fps Tomorrow. [UPDATED]

ORIGINALLY WRITTEN FEB 2018. Updated June 2020 with new information.

Ever since the Blur Busters Holiday Special article about the Amazing Journey to 1000Hz Displays and the earlier true 480Hz monitor tests, Blur Busters has been paying much more attention to HFR video developments.

We are the only website in the world to exclusively talk about “Better Than 60Hz”. This universe also includes 120fps HFR. However, there are also experiments with Ultra HFR — the art of displaying video at ultra high frame rates (HFR 240fps, 480fps and 1000fps) in real-time on ultra high refresh rate displays. Not slow motion!

Longtime readers are familiar with our world’s first embedded 120fps HFR web browser videos and world’s first 120fps gameplay video five years ago (Year 2013)

Now, we are testing video beyond 120 frames per second!

Ultra HFR: True 240 fps Real Time

Here is a method of creating 240 frame per second HFR video, for play back on today’s 240 Hz eSports monitor.

  • Get any favourite slo-mo camera capable of 240fps.
    …From smartphones to GoPros to Phantom Flex cameras, many cameras now can do 240fps.
  • Get a GPU capable of 4K60 playback.
    …GPUs capable of 4K 60fps playback are also able to do 1080p 240fps playback.
  • Get a true-240Hz gaming monitor. See List of Best Gaming Monitors.
  • Download and install ffmpeg.
  • Record video in normal 240fps slo-mo. Resulting file is 30fps.
  • If you need to use a video editor, edit while video is still slo-mo. Then export an .MP4 video.
  • Finally, run this ffmpeg command line to speed up the video back to real-time playback.
    This speeds up video to play at 8 times the frame rate, converting 30fps to 240fps.

    ffmpeg -i slowmo240.mp4 -r 240 -vf "setpts=(1/8)*PTS" -an realtime240.mp4
  • Play the video in your player (VLC, MPC-HC, Windows Media, etc) on your 240 Hz display.
    …You may need to experiment with different video players to find the best 240fps experience.

High speed cameras, designed for slo-mo, often do not record audio well. At the moment, the best way to add audio is to record the audio separately, and use a command line utility to dub the audio onto the sped-up file.

The same general instructions apply to 480fps and 1000fps HFR, with different speedup factors (replace number “8” with the appropriate speedup factor).

Share your results in the new Blur Busters Forums High Frame Rate (HFR) Video discussion area!

Future 1000fps HFR: 240fps is not the final frontier

In the newly created Blur Busters HFR Forum, there is a thread worth reading about Ultra HFR.

The art of playing true 1000fps real time on true 1000Hz displays: Cinematography of 2030s: Ultra HFR (1000fps at 1000Hz!)

Experimental 1000 Hz displays are already available, and will eventually be cheap within one human generation. Now even some smartphones are also gaining ultra-high frame rate video recording capabilities too. This presents opportunities for cheap Ultra HFR in the coming years!

By doing 1000fps HFR on a 1000Hz display, you can simultaneously avoid camera motion blur and avoid display motion blur and avoid stroboscopic effects. Strobless low-persistence (blurless sample-and-hold) without flicker is successfully achieved with 1000 Hz experimental laboratory displays, and is also useful with Ultra HFR video.

But why do UltraHFR?

Source Camera Blur and Destination Display Blur is Additive

Today, modern displays are sample-and-hold, for ergonomic flicker-free reasons, which creates a Hz bottleneck. No CRT flicker, but far more motion blur than a CRT tube.

A camera shutter 1/60sec combined with common display 1/60sec persistence = creates 2/60sec worth of motion blur. That’s 1/30sec of motion blurring seen by human eyes when playing 60fps video taken with 1/60sec shutter, played onto a 60Hz sample-and-hold display! This is even assuming instant pixel response GtG 0ms, otherwise, there’s worse motion blur than 1/30sec!

To reduce camera-side motion blur massively, a content creator can use a fast sports shutter 1/1000sec on the camera. But this will always be bottlenecked by destination display blur, from the sample-and-hold effect.

This diagram below is from our Pixel Response FAQ, GtG versus MPRT (the two different pixel response benchmarks). Flickerfree displays (sample-and-hold displays such as LCDs, OLEDs, MicroLEDs) produces a mandatory guaranteed added display-side motion blur:

To see this for yourself, we have TestUFO motion animations that demonstrates enforced motion blur caused by display persistence: TestUFO animation demo #1, TestUFO animation demo #2. As a general rule of thumb, the refresh rate of a flickerfree display is typically the guaranteed minimum display motion blur: 120Hz = 1/120sec = 8.3ms of motion blur = 8.3 pixels of blurring per 1000 pixels/sec motion.

To eliminate all motion blur weak links, both source shutter (1/1000sec) and display persistence (1ms MPRT) should be fast. There are two ways to reduce display motion blur:

  1. Shorten display’s frame visibility time via impulsing (CRT, plasma, black frames, etc)
  2. Shorten display’s frame visibility time via higher frame rate

The problem with approach (1) is you can produce stroboscopic stepping effects (phantom arrays) for fast-camera-shutter material such as sports, see The Stroboscopic Effect of Finite Frame Rate Displays. Besides, real life doesn’t flicker or strobe, so impulsing is a humankind band-aid for simulating reality.

Solving motion blur and stroboscopic effect simultaneously requires ultra high frame rates: 1000fps at 1000Hz! One can do a 360-degree camera shutter blurlessly. A full-open shutter is only 1 millisecond during 1000fps video. A 1000Hz display is blurless sample-and-hold via sheer brute Hz.

This makes video look closer to real life, closer to a Holodeck — with no framerate-induced limitations like forced motionblur or forced stroboscopic effects. This can make 1000fps UltraHFR less nauseating than 120fps HFR.

Diminishing Curve: Next Step After 120fps HFR should be 1000fps UltraHFR

Time-wise, the difference between 120fps HFR versus 1000fps HFR is roughly as big as the difference between 60fps and 120fps.  This is because 1/60sec and 1/120sec is an 8.3ms difference, while the difference between 1/120sec and 1/1000sec is a 7.3ms difference.  There is a curve of diminishing points of returns, but the massive jump up to 1000fps HFR greatly compensates.

  • 1/60sec = 16.7ms blur
  • 1/120sec = 8.33ms blur — observe 8.3ms improvement over 1/60sec
  • 1/1000sec = 1ms blur — observe 7.3ms improvement over 1/120sec

As a result, a large jump up the diminishing curve of returns is required to get as dramatic improvement as the jump between 60fps and 120fps HFR.  Therefore we believe that the next step after 120fps HFR should now be 1000fps HFR.

Also, UltraHFR benefits higher resolution more. 1000fps UltraHFR would not be nearly as useful for standard-definition video material (less than 1 pixel of motion blur for one-screen-width-per-second motion) but would be hugely beneficial to 8K material (8 pixels of motion blur for one-screen-width-per-second panning motion)

Very Important UltraHFR Tips For Max Frame Rate Video

When trying to force a display to go maximum motion clarity, there are some best practices.

In the world of experiments with 240fps, 480fps and 1000fps real-time video playback on ultra-high Hz displays, these are very important considerations from recent research:

  1. Display persistence is additive to camera persistence (shutter). A 1/60sec persistence display combined with a 1/60sec shutter video, generates a combined 2/60sec of human-perceived motion blur.  As a rule of thumb, your common  perceived blur is often at least twice the camera shutter, and often much worse (e.g. 1/1000sec shutter combined with 1/60sec display persistence). This means that on most displays, a 1/1000sec shutter at 60fps only has barely half the motion blur of 1/60sec shutter, simply because it is bottlenecked by 1/60sec display persistence! Also, a display that has a pixel response (GtG) that is a significant percentage of a refresh cycle at its max Hz — will also limit UltraHFR benefits even further.
    You must be aware of how both source (Camera) and destination (Display) persistence combines in motion blur in order to understand limitations to UltraHFR benefits
  2. Resolution reduction is a big problem at higher frame rates on current cameras.  Recommended is to oversample by recording at about 2x resolution (e.g. true 4K) and then downconvert (e.g. to 1080p).  For this reason, it is hugely preferable to get a superior high speed camera (e.g. Phantom Flex 4K) instead of smartphone camera (e.g. iPhone in Slo-mo) when generating source footage.
    You must minimize resolution loss to preserve UltraHFR benefits
  3. Compression is a big blur problem.  Use less compression
    You must minimize compression to preserve UltraHFR benefits
  4. Lens autofocus is sometimes difficult during filming moving imagery.  It is especially amplified when your motion clarity is extremely sharp, to the point where any camera focus issues begin to become visible.
    You must make sure focus is sharp during fast motion to preserve UltraHFR benefits
  5. Be aware of the vicious cycle effect where higher resolutions demands higher refresh rates to eliminate resolution limitations. The difference between static images and moving images become much more pronounced at 8K than it is for 1080p, since motion blur is time-based (e.g. same amount of blur per inch, affects more pixels on higher DPI displays).  This is especially hard because current camera technology often goes lower resolution at higher frame rates.
    You must use a higher resolution to preserve UltraHFR benefits
  6. High frame rates benefits high motion material.  A great example is sportsfield panning, downhill skiing, BMX racing, car racing, etc.  When the first HDTVs arrived, some camera operators reported they had to intentionally slow down panning and operate slightly differently because LCD HDTV blur (of the first flat panel HDTVs) was much worse than old CRT motion clarity.
    UltraHFR benefits fast motion material very significantly.

Many historical frame rate tests about frame rate benefits have often neglected to consider (1) (2) (3) (4) (5) (6) simultaneously.

Research indicates that any one of the above can be a weak link in eliminating the benefits of UltraHFR video, and one must carefully control the variables to give truly human-visible benefits of UltraHFR. If you are planning any experiments in UltraHFR at maximum frame rates for the best possible Holodeck-league motion clarity, it is very important to know and understand the above variables.

Four Simple Fixes for Temporally Accurate Reality Simulation

To eliminate all weak links, so that remaining limitations is human-vision based. No camera-based or display-based limitations above and beyond natural human vision/brain limitations.

  1. Fix source stroboscopic effect (camera): Must use a 360-degree camera shutter;
  2. Fix destination stroboscopic effect (display): Must use a sample-and-hold display;
  3. Fix source motion blur (camera): Must use a short camera exposure per frame;
  4. Fix destination motion blur (display): Must use a short persistence per refresh cycle.

Therefore;

  • Ultra high frame rate with 360-degree camera shutter is also short camera exposure per frame;
  • Ultra high refresh rate with sample-and-hold display is also short persistence per refresh cycle,

Therefor, to solve (1), (2), (3), (4) simultaneously requires ultra high frame rates at ultra high refresh rates.

We Love Hollywood Movie Maker Mode Too: Full Control Of Entire Blur Chain

1000fps UltraHFR gives the director full control of motion blur, too.  1000fps UltraHFR is a huge venn digram capable of embedding the entire range of framerates in humankind.

With such fine granularities on an ultra-high-Hz displays, refresh rates no longer need to be evenly divisible by frame rate, in order to look smooth!

When 18fps, 23.97fps, 24fps, 25fps, 50fps, 59.94fps, 48fps, 60fps, are all played to a 1000Hz display, all framerates have their original look and motion blur. They play judderlessly without noticable pulldown judder (i.e. 3:2 pulldown) because of the unnoticeable ultrafine refresh rate granularity.

In other words, 24fps video embedded inside a UltraHFR-capable video file, played on a 1000Hz display, looks exactly like a 24Hz display — all with original camera motion blur and persistence blur.

The ability to go as blurry as you want (preserve camera blur / old frame rates) — and the ability to go virtually zero-blur as you want — is an amazing property of UltraHFR video files in the quadruple-digit framerate range.

Conclusion

At Blur Busters, we are among the few people in the world to have witnessed Ultra HFR video!

Being the founder of Blur Busters (Mark Rejhon), I used to work with video processor companies (RUNCO, Key Digital, and a PCI card containing a Faroudja chip). I was the author of the world’s first open source 3:2 pulldown deinterlacer algorithm twenty years ago in dScaler (Year 2000, Internet Archive).

We believe this is very useful for many applications in the coming decade, from speciality theatre, virtual reality, amusement park rides, advanced cinema, truly immersive virtual vacations, “Holodeck” video, and other applications once more 1000 Hz displays are commercially available beginning sometime within the next decade and beyond.

Related Reading