Object-level understanding of reconstructed 3D scenes remains challenging without costly 3D annotations. Recent mask-lifting methods use 2D foundation-model masks to supervise 3D segmentation, while structured Gaussian representations such as Scaffold-GS provide a useful prior for balancing compact semantics and detailed appearance. However, existing Scaffold-GS-based semantic methods often rely on anchor- or region-level shared features, which improve compactness but can be too coarse for object boundaries and fine-grained parts. We propose ChildSplat, an anchor-child semantic Gaussian representation for 3D object segmentation with downstream editing handles. Starting from a pretrained Scaffold-GS scene, we freeze the geometry and appearance fields and learn a lightweight semantic branch on its anchor-child structure. Anchors encode compact semantic context, while child Gaussians obtain geometry-aware residual semantic embeddings and are rendered with the frozen splatting weights. We further derive child-level ownership supervision from multi-view mask contributions, explicitly linking 2D mask evidence to individual child Gaussians to improve fine-grained semantic ownership and reduce cross-object leakage in child-level selection. When hierarchical annotations are available, residualized leaf supervision and lightweight hierarchy-aware constraints enable the same semantic field to support both object-level and part-level queries. Experiments show that our method learns compact, view-consistent 3D semantic groups, which can serve as semantic handles for downstream local editing.
Abstract
Framework
ChildSplat builds a lightweight semantic branch on top of a pretrained and frozen Scaffold-GS representation. The semantic branch follows the anchor-child structure and is trained in two stages, sharing a single linear classifier that maps all semantic embeddings to class logits.
Stage 1 — Anchor-level semantic scaffold: Starting from the pretrained and frozen Scaffold-GS scene, each anchor is augmented with a learnable semantic feature that is decoupled from the appearance feature used for RGB reconstruction. The anchor position acts only as a fixed geometric condition. An anchor semantic MLP produces a compact anchor embedding, and this anchor-level semantic scaffold is learned from rendered 2D mask supervision. It stores compact semantic context while leaving the pretrained geometry and appearance fields unchanged.
Stage 2 — Child semantic refinement: The anchor scaffold is frozen, and each child Gaussian obtains a geometry-aware residual semantic embedding from the anchor context and its own child geometry. Child Gaussians are rendered with the frozen splatting weights. We further derive child-level ownership supervision from multi-view mask contributions, explicitly linking 2D mask evidence to individual child Gaussians to improve fine-grained semantic ownership and reduce cross-object leakage during child-level selection.
When hierarchical annotations are available, residualized leaf supervision and lightweight hierarchy-aware constraints extend the same semantic field to support both object-level and part-level queries. Both anchor and child predictions are classified by the shared linear head, aligning rendered 2D predictions and 3D child-level semantics in a common semantic space.
Results
