Motion-Aware Regularization via Soft Mask Gating for 4D Gaussian Splatting

Motivation & Overview

4D Gaussian Splatting (4DGS) models dynamic scenes by coupling canonical Gaussians with a time-dependent deformation network — but applies deformation uniformly to every Gaussian, including those representing static regions.

In real scenes, only a small fraction is dynamic. Treating all Gaussians identically wastes representational capacity and prevents identifying which parts of the scene are actually moving.

Our solution: a lightweight motion-mask head that predicts a per-Gaussian soft gate m_i(t) ∈ (0,1), controlling how much each Gaussian deforms — learned fully without supervision.

Abstract

This project studies motion-aware dynamic scene reconstruction built on top of the 4D Gaussian Splatting (4DGS) framework. The system takes a set of images with associated camera poses and timestamps as input and outputs a 4D Gaussian representation enabling novel-view rendering over time. While the baseline 4DGS model represents dynamic scenes by predicting time-dependent deformations for all Gaussian primitives in a canonical space, it lacks an explicit mechanism to distinguish between static and moving elements.

To address this, we introduce a lightweight motion-mask branch within the 4DGS deformation network. This mask acts as a learned scalar gate that modulates deformation strength and is optimized jointly with image reconstruction. Our method is fully unsupervised: it requires no ground-truth motion labels or external segmentation. Instead, it guides the learning process through a combination of the standard reconstruction objective and two novel regularizers: a binarization loss and a static-deformation loss.

Experiments on D-NeRF scenes show that the proposed method can modestly improve reconstruction on strongly dynamic synthetic scenes, with up to +0.28 dB PSNR on BOUNCINGBALLS and +0.22 dB on JUMPINGJACKS. However, the method does not produce universal reconstruction gains: on the real-world HyperNeRF CHICKCHICKEN scene, baseline 4DGS remains slightly better on the main image metrics. The results also show that reconstruction quality and motion-mask separability are related but not identical objectives. A near-binary mask is not always required for better reconstruction, while stronger regularization can improve mask diagnostics without improving image quality.

Method

In the standard 4DGS pipeline, deformation offsets (Δμ_i, Δs_i, Δr_i) are applied to every Gaussian regardless of whether the corresponding region actually moves. We address this by adding a single linear layer g_φ that maps the shared deformation hidden state h_i(t) to a scalar pre-activation and applies a sigmoid:

The mask gates the positional update: μ_i(t) = μ⁰_i + m_i(t) · Δμ_i(t). When --motion-gate-rot-scale is enabled, scale and rotation are gated analogously. Opacity is intentionally left ungated, as it is managed by the 3DGS densification and pruning logic.

Because the mask is predicted from the shared feature rather than stored as a persistent per-Gaussian parameter, it requires no modification to Gaussian cloning, pruning, optimizer-state management, or checkpoint serialization — it is a drop-in addition of a single MLP head.

Architecture: Motion Mask Integration

The motion mask head is appended to the deformation decoder. The shared hidden feature feeds four heads: position, rotation, scale, and motion mask.

Training Objective

The full training loss adds two motion-specific regularizers on top of the baseline 4DGS objective: ℒ = ℒ_img + λ_gridℒ_grid + λ_binℒ_bin + λ_staticℒ_static

Identifiability Limitation

The motion-mask formulation introduces an inherent ambiguity. The renderer does not observe the mask and deformation independently, but only their product:

As a result, multiple configurations can produce similar rendered images. For example, a small mask with a large deformation can be equivalent to a large mask with a small deformation. This creates an identifiability issue: image reconstruction alone cannot uniquely determine whether motion should be attributed to the mask or to the deformation magnitude.

The binarization and static-deformation losses are introduced to partially resolve this ambiguity by encouraging the mask to take on interpretable values and discouraging deformation in low-mask regions. However, this ambiguity is not fully eliminated and contributes to the observed gap between reconstruction quality and mask separability.

Notation Reference

Diagnostics & Tooling

Since the system operates without ground-truth motion masks, we implemented a suite of diagnostic tools to monitor the emergence of static-dynamic separation. The training pipeline logs per-iteration mask statistics to motion_mask_stats.jsonl, tracking variables such as:

• Mean & Std Dev: Overall distribution of the learned gate values.
• Dynamic Fraction: The percentage of the scene identified as moving, specifically the fraction where m > 0.4.
• Loss Components: Real-time values for the static-deformation and binarization regularizers.

To analyze spatial behavior, the method exports specialized color-coded PLY point clouds. In these files, the RGB channels are mapped to mask intensity: Red indicates high motion-mask values (predicted dynamic regions), while Blue indicates low values (predicted static regions).

Results

We evaluated our method across two synthetic D-NeRF scenes (BouncingBalls, JumpingJacks) and one real-world HyperNeRF scene (ChickChicken). Our findings indicate that while motion-aware gating improves metrics in controlled synthetic environments, real-world data presents unique challenges for unsupervised decomposition.

Regularization Settings

• Reg-A: λ_static = 1e-3, λ_bin = 1e-3 — mild regularization
• Reg-B: λ_static = 2e-3, λ_bin = 1e-3 — stronger static penalty
• Reg-C: λ_static = 2e-3, λ_bin = 2e-3 — stronger on both
• Over-reg: λ_static = 1e-2 — collapse sanity check

Baseline vs. Reg-B Comparison

Motion Mask Visualizations

Red/blue PLY exports: red = high mask (dynamic), blue = low mask (static).

Reg-A: λ_s=1e-3, λ_b=1e-3

Reg-B: λ_s=2e-3, λ_b=1e-3

Reg-C: λ_s=2e-3, λ_b=2e-3

Reg-A: λ_s=1e-3, λ_b=1e-3

Reg-B: λ_s=2e-3, λ_b=1e-3

Reg-C: λ_s=2e-3, λ_b=2e-3

@article{ren2025motionaware4dgs,
  author    = {Ren, Yuan and Taylor, Ben and Swierad, Lucas},
  title     = {Motion-Aware Regularization via Soft Mask Gating for 4D Gaussian Splatting},
  year      = {2026},
  note      = {ROB 430 Final Project, University of Michigan},
  url       = {https://github.com/yuanren114514/ROB_430_Final_Project},
}