Frame Interpolation on Generated Image-to-Video Footage: How It Works and What Breaks

The common assumption is that frame interpolation fixes the jerkiness of AI generated video the same way it fixes camera footage. It does not. Frame interpolation artefacts on generated video are a different category of problem because the source footage already contains motion that the generation model invented.

Filmed footage has physical boundaries. A walking person in a real clip has legs that stay attached. An optical flow interpolation tool like RIFE or FFmpeg minterpolate can track those real edges and insert intermediate frames. Generated footage from an image-to-video model such as Runway Gen-3 Alpha or Kling 1.6 does not have persistent boundaries. The model that produced the clip predicted pixel motion without building a 3D skeleton. What looks like motion is statistical guesswork about what pixels usually do in that context.

When you apply frame interpolation to that clip, the interpolation tool treats every pixel shift as legitimate motion. Drift becomes amplified motion. Limb mutation becomes a gap that the interpolator tries to fill with more invented pixels. The result is not smoother video. It is smoother artefacting.

Optical flow based interpolation tools such as RIFE v4.6 (MIT licence, 2023), Google FILM (Apache 2.0, 2022), and FFmpeg minterpolate all work on the same principle. They estimate where each pixel moved between two input frames, then generate a new frame halfway along that motion vector. The FILM model from Google Research handles non-rigid motion better than older methods but still assumes the scene is physically consistent between frames.

An image-to-video generation clip from Luma Dream Machine or Stable Video Diffusion does not meet that assumption. A person's face may shift ethnicity across the clip duration because the model interpolated toward its training distribution rather than anchoring to the source image. A shirt collar can morph into a different cut because the model lacks persistent garment representation. Optical flow treats these as motion and draws lines from pixel positions in frame A to frame B. The interpolated frame then shows a face that is halfway between two different faces.

The RIFE documentation discusses occlusion artefacts and non-rigid motion as known failure cases. The readme for the RIFE repository (hzwer/arXiv2020-RIFE) warns that large motion and occlusion areas produce blurring and artefacting. Generated footage presents exactly those conditions in every clip: background sliding separates foreground from background, creating synthetic occlusion boundaries that the flow model treats as real edges.

RIFE generated footage interpolation artefacts fall into three categories that compound each other when applied to AI video.

Warping Artefacts From Subject Drift

Subject drift is the gradual transformation of the main figure over the clip duration. A generated clip from Kling 1.6 produced at 24 fps over five seconds may show a woman whose jawline narrows by the final frame. RIFE interprets that narrowing as a forward motion vector. The interpolated frame pushes the jawline further inward than any source frame, creating a visible warping artefact that snaps back on the next source frame.

Limb Mutation Amplification

Limb mutation is where arms, legs or fingers warp or multiply during motion. The ComfyUI frame interpolation node running RIFE v4.6 on a Wan 2.1 clip at 720p showed a hand with six fingers by the third interpolated frame. The source clip had five fingers that shifted position. RIFE interpreted the blur of the shifting hand as a motion vector field and generated a new hand shape that combined both positions with an extra digit.

Background Sliding And Independent Layer Motion

Background sliding happens when the background moves independently of the foreground figure. LTX-Video clips at 768x512 often show this because the model does not infer a consistent 3D scene. FFmpeg minterpolate with the default motion search radius treats the sliding background as camera motion and attempts to interpolate it. The result is a background that shimmers independently from the figure, which the interpolation smooths into motion blur that does not look like camera blur.

FFmpeg minterpolate AI video smoothing uses the minterpolate filter which applies motion compensated interpolation. The FFmpeg documentation for version 7.1 (2025 release) notes that the filter works best on footage with consistent motion vectors and fails on scenes with occlusion, scene cuts, and non-rigid motion. Generated clips from Hailuo Minimax or Seedance 1.0 contain all three.

The minterpolate filter accepts a motion_block_size parameter and a search_radius parameter. Default values assume filmed footage where motion blocks correspond to real objects. On AI generated video the blocks often straddle a morphing boundary connecting the figure and the background. The filter then interpolates across that boundary, creating a visible seam that the source clip did not have.

The documentation advises using scene detection before interpolation to avoid blending across scene cuts. Generated video does not have scene cuts but it does have identity shifts where a face changes age or gender. The minterpolate filter has no parameter to detect identity shifts. It treats the transition as motion and blends the two identities together across interpolated frames.

Tool	Licence	Release Year	Documented Failure Mode Relevant To Generated Video
RIFE v4.6	MIT	2023	Occlusion artefacts, large motion blur, non-rigid motion degradation
Google FILM	Apache 2.0	2022	Motion exceeding 50 pixel displacement degrades quality
FFmpeg minterpolate 7.1	GPL	2025	Scene cuts and occlusion produce blending artefacts
EMA-VFI	Apache 2.0	2024	Temporal consistency loss on fast motion
AMT	Apache 2.0	2023	All-pairs matching fails on non-rigid deformation

Optical flow interpolation generated video warping becomes most visible when converting a 24 fps generated clip to 60 fps. The conversion inserts 1.5 frames for every source frame pair. Each interpolated frame is a synthetic guess at what happened between two frames that were themselves synthetic.

At 24 fps the temporal gap is approximately 42 milliseconds. Generated clips from Pika 2.0 or Runway Gen-3 Alpha at 24 fps often hide subject drift because the movement is small enough per frame that the viewer does not notice the morph. At 60 fps the viewer sees 60 individual frames per second instead of 24. The interpolated frames reveal the drift trajectory that was previously invisible.

Stable Video Diffusion outputs at 24 fps by default. The model has no native frame interpolation capability as of its November 2023 release. Running the output through FFmpeg minterpolate with a 2.5x frame rate multiplier pushes the motion vector field beyond what the model's temporal attention layer was trained to handle. The warping artefacts appear as a visible shudder on the figure outline that does not exist in the source 24 fps clip.

Motion Blur Synthesis And The Jutter Problem

Filmed footage at 24 fps uses motion blur during capture to smooth motion. Generated video lacks realistic motion blur because the model predicts pixel positions rather than simulating a shutter open time. Frame interpolation tools do not synthesise motion blur unless specifically configured to. FILM from Google Research includes a motion blur handling parameter but the 2022 paper notes it degrades with motion exceeding 50 pixel displacement.

The result is jutter: a rapid oscillation of the figure edge across interpolated frames. The source clip looked acceptable at 24 fps because the viewer's persistence of vision smoothed the motion. At 60 fps the interpolated frames create a stroboscopic effect on edges, especially on high contrast boundaries like a dark coat against a light background. The effect is worse on clips from Seedance 1.0 at 720p because the lower effective resolution means the pixel displacement is a higher proportion of the figure size.

Temporal consistency is the absence of flicker, jitter, and frame to frame lighting shifts that are not part of the intended motion. Generated video from every image-to-video tool examined here, including Kling 1.6 with its start and end frame support and Luma Dream Machine with its camera controls, shows measurable temporal inconsistency even at native frame rate. The US Copyright Office AI report Part 2 published January 2025 noted that AI generated video lacks the deterministic relationship between frames that film footage has.

Converting a 30 fps generated clip to 60 fps through frame interpolation exposes this temporal inconsistency as a visible pulsing. The interpolator cannot distinguish between intended motion flicker and generation flicker. It treats both as motion vectors and creates intermediate frames that show the flicker halfway between two states. The viewer sees a light source that dims and brightens at 30 Hz because the source frames had inconsistent brightness and the interpolated frames reproduced the inconsistency at a higher temporal resolution.

ComfyUI frame interpolation node workflows chain a VFI node after the video generation node. The default configuration applies interpolation to every frame pair. On a Wan 2.1 clip with visible temporal flicker, the interpolated output showed a pattern of alternating frame brightness that the source 30 fps clip did not display because the flicker rate was below the viewer's conscious detection threshold.

When To Use Interpolation

Frame interpolation of generated video suits the open source tinkerer running Wan 2.1 or LTX-Video locally who needs to slow motion for effect without re prompting. The interpolation gives a 24 fps clip a 48 fps or 60 fps output that can be slowed to 50% speed in the edit without dropping frames. Use interpolation only on clips where the figure stays recognisable across the full duration. The artefacts are structural; you cannot fix them later.

When To Skip It Entirely

The social media content creator who needs high volume short form video at 60 fps should skip frame interpolation of generated clips. The artefacts are visible on phone screens. Platform detection systems may flag the output. Generate at the platform's target frame rate directly. TikTok and Instagram accept 30 fps. YouTube accepts 30 fps and 60 fps. The visual quality of a native 30 fps clip without interpolation is better than a 24 fps clip interpolated to 60 fps. The interpolation artefacts add a synthetic look that viewers associate with poor AI output.

For Client Work: Test One Clip First

The solo filmmaker who needs b-roll for client work should generate at the model's highest native frame rate and test one interpolation before committing to a workflow. Higher frame rate does not always look better for generated video. The smoother motion comes with amplified subject drift that may be worse than the jutter the interpolation was meant to fix.

The thing that most often goes wrong is assuming that because the interpolation tool works on real footage it will work on generated footage. The optical flow algorithm sees motion everywhere in a generated clip because the clip has motion everywhere. It is not the right tool for the job unless you have inspected every frame pair for subject drift and limb mutation first.