SplatRenderer: 3D/4D Gaussian Splatting for Unreal Engine
An open-source UE5 plugin that renders 3D and 4D Gaussian Splats in real-time using a custom compute shader pipeline. 2M+ Gaussians at 100+ FPS.
Overview
SplatRenderer is a free, open-source Unreal Engine plugin that brings both static 3DGS (.ply) and animated 4DGS sequences (.gsd) into the engine. I built the entire rendering pipeline from scratch using UE5’s RenderGraph API and custom compute shaders.
The demo runs 2M+ Gaussians at 100+ FPS on an RTX 4080.
Demo Reel
Every 4DGS scene in this video was generated from a single video capture. Multiple splat scenes are cropped, duplicated, and arranged together in one UE level.
SplatRenderer demo reel: 3DGS + 4DGS scenes running in Unreal Engine 5 | Watch on YouTube
v1.1 Demo Reel
SplatRenderer v1.1: Level Sequencer integration | Watch on YouTube
From Video to 4DGS
Every 4DGS scene in the demo was generated from a single video shot on iPhone. One recording becomes a fully animatable Gaussian Splat sequence running in Unreal Engine.
The pipeline: capture video, train with Apple SHARP, convert to .gsd with the 4DGS Converter, drag into Unreal Engine.
Features
Drag-and-Drop Blueprints
Drag BP_3DGS or BP_4DGS into your level. Set the file path, adjust parameters in the Details panel, done.
Crop Volume
Use the 3D crop box with a draggable editor widget to isolate regions of interest. Works with both 3DGS and 4DGS.
Level Sequencer Integration
v1.1 adds full Level Sequencer support. Keyframe Brightness, SplatScale, ScrubFrame, PlaybackSpeed, AudioVolume, and Crop parameters directly on the UE Sequencer timeline. Set Playing to off and drive playback entirely through Scrub Frame keyframes for precise cinematic control.
Multi-Scene Composition
Crop, duplicate, and arrange multiple splat scenes in one level. Each instance runs independently with its own playback controls.
Scene Gallery
Technical Deep Dive
Rendering Pipeline
The plugin implements a fully custom GPU rendering pipeline using UE5’s RenderGraph API and custom compute shaders written in HLSL. The entire pipeline runs on the GPU with no CPU readback, keeping the render thread free.
At a high level, each frame goes through: frustum culling, covariance projection, depth sorting, and the final draw pass with Gaussian alpha blending. The pipeline is designed to handle millions of splats while maintaining real-time frame rates.
Architecture
The plugin follows UE5’s Scene Proxy pattern, cleanly separating game thread logic (actor setup, file loading, parameter control) from render thread execution (GPU dispatch, sorting, drawing). This ensures thread safety and integrates naturally with UE5’s rendering architecture.
For 4DGS, a custom sequence player handles frame preloading on background threads with double-buffered textures, so the GPU never stalls waiting for data.
3DGS vs. 4DGS
The plugin supports two rendering paths:
- 3DGS (
.ply): Gaussian data is loaded once and stored as GPU textures. The covariance is pre-computed at load time, so per-frame cost is minimal. - 4DGS (
.gsd): Per-frame Gaussian data is streamed from a custom compressed format. The GPU reconstructs each frame’s covariance on the fly, enabling animated sequences from a single file.
.gsd Format (Open Source)
I designed .gsd (Gaussian Sequence Data) as a compact container for 4DGS sequences. Unlike raw PLY sequences that can be hundreds of GBs, .gsd compresses per-frame Gaussian data into a single streamable file.
A 480-frame PLY sequence weighs around 30 GB. The same sequence in .gsd compresses to ~13 GB with generic LZ4, or ~5 GB when optimized for SHARP’s PLY layout.
The format supports:
- LZ4 compression: chosen for decompression speed over ratio. 4DGS needs to decode frames in real-time during playback, so fast decompression is critical.
- VQ (Vector Quantization) codebooks: rotation, scale, and SH data are quantized to 256-entry codebooks per frame, reducing entropy before LZ4 kicks in. This pushes compression from ~3-5x (LZ4 alone) to ~5-8x.
- Precision control: FP32 or FP16 per component (position, rotation, scale, SH coefficients)
- Frame-level random access: seek to any frame without decoding the entire file
The converter and format spec are fully open source:
GitHub: 4DGS ConverterPerformance
Key design decisions that enable 2M+ Gaussians at 100+ FPS:
- GPU-only pipeline: No CPU-GPU sync points in the render path
- Aggressive culling: Splats outside the view frustum or crop volume are discarded before sorting
- Smart caching: The sort pass is skipped when the camera hasn’t moved
- Async streaming: 4DGS frames are decompressed ahead of time on background threads
Particle FX
An optional compute shader system applies forces (noise, wind, gravity, vortex) to individual Gaussians in real-time, enabling procedural effects on top of captured scenes.
Future Work
- Relighting: dynamic lighting interaction with Gaussian Splat scenes
- FX system: expanded particle effects and material-level control
- Temporal optimization: smarter frame interpolation and predictive streaming
Community Reception
The plugin was featured on Radiance Fields alongside tools from NVIDIA and other industry players within its first week of release.
Links
- GitHub: SplatRenderer Plugin
- GitHub: 4DGS Converter
- YouTube: Demo Reel
- YouTube: Sequencer Demo
- Radiance Fields Feature
Related Articles
Get in Touch
Have questions or want to collaborate? Feel free to reach out!