Fishbone: From One 3D Asset to a Million Controllable Edits

Anonymous

Fishbone extracts a coupled rib–spine control structure from an input mesh and defines a compact deformation space over rib- and spine-driven primitives. From a single freeform asset, this unified handle system enables diverse, geometrically plausible shape variations, spanning local cross-sectional edits as well as global bending, twisting, and stretching, all driven by intuitive low-dimensional parameters. The rib–spine representation also supports real-time reduced-space simulation and keyframe animation.

✨ Abstract ✨

We present Fishbone, a fully automatic mesh deformation framework that generates diverse, controllable variations of any input triangle mesh. The control structure is a tuple of ribs (cross-sectional iso-contours extracted at successive geodesic levels on the surface) and a smooth spine curve threaded through their centers, derived directly from the mesh via the heat method and a robust center-finding procedure with no manual annotation, template fitting, or per-shape rigging. On top of it we define six rib-driven and three spine-driven deformation primitives, all driven by low-dimensional parameters and propagated to mesh vertices through a Gaussian-weighted skinning scheme that preserves local detail. The same handle system is simulation-ready, real-time, and interactive: lumped mass and damping are inherited from the precomputed skinning weights, the integrator runs at interactive frame rates, and user input projects back to the reduced spine through the same matrix that lifts spine motion to the mesh. Experiments show greater deformation diversity and geometric plausibility than cage-based, skeleton-driven, and neural baselines, while supporting batch processing of large asset libraries, sim-to-real diversification in robotics, and procedural content creation.

🎯 Pipeline 🎯

From left to right, our geometry-driven pipeline takes a raw input mesh, runs a lightweight cleaning + watertightness-repair pass to produce a clean mesh, extracts ribs as iso-contours of a heat-method geodesic field rooted at an automatically selected face of the part's AABB, kept as closed loops on solid meshes and as open polylines on thin-shell parts, evaluates spine points per rib by maximizing a flatness-+-centroid-+-parent score, generates the spine by chaining the per-rib spine points into a branching tree, and computes per-vertex skinning weights that bind the mesh to the surrounding ribs via soft-cutoff Gaussians of the projected rib distance.

The same control structure ℱ = (ℛ, 𝒮, W) then drives three downstream stages, all sharing the cached weights: a parameterized static deformation algebra of rib- and spine-driven primitives, a reduced-coordinate dynamics solver with mesh lift, and a physics-free keyframe animation pipeline.

🔧 Deformation 🔧

Rib-driven Deformation

Uniform Scale

Anisotropic Scale

Translation

Rotation

Local Deform

Cross Section

Spine-driven Deformation

Stretch

Bend

Twist

⚙️ Dynamics ⚙️

Mushroom

Plant

Tuna

🎞️ Animation 🎞️

Cat

Rose

Teletubbies

🎬 More Demo 🎬


🖥️ Editing GUI 🖥️

An interactive editor lets users sculpt deformations directly on the Fishbone representation.

Editing GUI

🧊 3D Shape Generation 🧊

Generated 3D shapes compared against ground truth and PartCrafter. Each row shows the input, ground-truth mesh / rib / spine, the PartCrafter output, and our generated mesh / rib / spine.

♠️ Editing Comparison ♠️

Editing comparison across Ours, WIR3D, and ARAP. For each method we show its representation and the resulting deformation; the leftmost column is the input mesh.

🤖 Robot Learning via Data Diversification 🤖

Policy is trained from Fishbone-generated deformation variants of single objects. We present single object scene and multi object scene for three objects.

Croissant
Tuna
Cup
Single Object
Multi Objects

💬 Agent 💬

A VLM agent drives the editing GUI through tool calls — listing ribs, selecting them, and applying deformations on behalf of the user.

user
make the four legs thicker
active part
{"type":"active_part","part":"geometry_0"}
tool call
list_ribs({})
tool result
list_ribs: Available ribs on part 'geometry_0': ['r1', 'r2', 'r3', 'r4-1', 'r4-2', 'r4-3', 'r5-1', 'r5-2', 'r5-3', 'r6', 'r7', 'r8-1', 'r8-2', 'r8-3', 'r9-1', 'r9-2', 'r9-3', 'r10']
(Names are labelled in the screenshot. Lower indices are usually closer to the part's starting end.)
list_ribs screenshot
tool call
select_ribs({"ribs":["r4-2","r5-2","r4-3","r5-3","r8-2","r9-2","r8-3","r9-3"]})
tool result
select_ribs: Selected 8 rib(s): ['r4-2', 'r5-2', 'r4-3', 'r5-3', 'r8-2', 'r9-2', 'r8-3', 'r9-3'] select_ribs screenshot
tool call
deform({"mode":1,"params":{"uniform_scale":1.35}})
tool result
deform: Applied mode 1 (uniform_scale) with params {'uniform_scale': 1.35} to 8 rib(s). deform screenshot
evaluation
✓ APPROVED — The four legs are visibly thicker with smooth geometry and a reasonable magnitude change.
tool call
done({"message":"Made all four legs thicker by uniformly scaling the leg ribs outward."})
tool result
done: Made all four legs thicker by uniformly scaling the leg ribs outward.
done
Made all four legs thicker by uniformly scaling the leg ribs outward.

♣️ Ablation: Rib Extraction Design Choices ♣️

Each panel ablates one component of our rib extraction. Ours uses heat-method geodesic ribs with automatic root-axis selection and adaptive level count K = clip(round(10 Lp / Lo), 3, 10). Removing the heat method (grid ribs) or auto root-axis selection misaligns the rib stack with the object's principal axis. Fixed-K variants instead over- or under-sample multi-part objects: K=5 under-resolves large parts (e.g., the plant stem in row 4), while K=10 and 15 over-resolve small parts (leaves, buds).

♣️ Ablation: Score-maximization vs. Midplane Center ♣️

Comparison between selecting spine points by score maximization and our midplane center criterion.

♣️ Ablation: Branching Tree Handling ♣️

Effect of the branching design on the spine structure. Each row compares the spine produced without branching against our full method with branching.

♣️ Ablation: Mesh Lifting ♣️

Comparison of mesh lifting strategies on the same input.