Orienting Fish in 3D

Where this boids simulation diverges from the textbook algorithm — and why.

The Gap Between Particles and 3D Models

Craig Reynolds' original rules produce a velocity vector for each agent — a direction and a speed. For a point particle or an arrow, that's everything you need to draw it. But a 3D fish model has a nose, a tail, a dorsal fin. Positioning it correctly in space requires knowing not just where it's pointing but which way is up for that particular fish. The boids algorithm doesn't tell you that.

Think of each fish as living in two coordinate systems at once. There's the velocity frame — the direction the fish is swimming, which is exactly what the boids rules produce. And there's the body frame — the fish's own local axes: nose forward, spine upward, fins out to the sides. This is what the renderer needs to orient the 3D mesh. Bridging the two is the core engineering challenge, and it's mostly straightforward — until a fish tries to swim straight up.

The Vertical Singularity

The standard way to align a 3D mesh with a direction is a lookAt operation: given the fish's velocity vector and a "world up" reference — straight up in world space — compute the three axes of the body frame using cross products. This works perfectly for horizontal swimming. But when a fish swims nearly straight up, its velocity vector and the world-up reference become nearly parallel. A cross product of two nearly-parallel vectors produces a result that's close to zero and can point in any direction depending on floating-point rounding. The body frame becomes undefined, and the fish's orientation starts flickering or spinning wildly.

This is the "boids spiraling" bug. A fish drifts into a steep upward angle, the orientation math breaks down, the fish starts spinning, and the chaotic rotation feeds back into velocity — which sends the fish spiraling out of control. Fixing it cleanly turns out to require three mechanisms working together.

The Three-Part Fix

1. Blended up-vector

Instead of always using straight-up as the lookAt reference, detect how vertical the fish is swimming and smoothly blend the reference toward a horizontal vector as the fish approaches vertical. The transition is gradual — there's no hard cutoff — so the fish's orientation never jumps. When it matters most (near-vertical), the two input vectors are never nearly parallel, and the cross product stays numerically stable.

2. Turn-rate damping near vertical

Even with a stable reference vector, the computed target orientation can still flicker near vertical, because small changes in velocity direction map to large changes in the body frame in that zone. The fix: scale the turn rate down — to as little as 10% of normal — as a fish approaches vertical. The fish can still reorient; it just does so slowly, giving its velocity time to curve back toward horizontal before the orientation can accumulate visible error.

3. Orientation smoothing

Each fish maintains a target orientation and blends toward it over time rather than snapping to it instantly. The blend rate is itself adaptive: faster when the fish swims horizontally (stable zone, snappy response is fine), slower when near-vertical (resist sudden changes). Any momentary flicker in the computed target gets absorbed by the smoothing before it ever reaches the rendered mesh.

These three fixes work at different timescales. The blended up-vector prevents bad geometry from being computed in the first place. Turn-rate damping slows down how fast bad geometry can propagate into orientation. Orientation smoothing absorbs whatever brief instability slips through. Together they make the singularity practically invisible — but any one of them alone isn't enough.

The Preventive Measure: DAMP_Y

The cleanest fix for a singularity is to not go there in the first place. The Vertical damping slider (DAMP_Y) reduces the vertical component of each boid's acceleration every frame. It doesn't prevent vertical motion entirely, but it biases the school toward horizontal swimming — where the orientation math is straightforward and the three-part fix above rarely needs to engage.

Think of DAMP_Y as the first line of defense and the three-part fix as the safety net. In practice, a little vertical damping goes a long way: with the default setting, fish still bank and dive naturally, but steep sustained climbs are rare enough that the singularity zone becomes an edge case rather than a regular occurrence.

Try dragging DAMP_Y all the way to zero in the playground — the school becomes much more three-dimensional, fish occasionally spiral, and you can see the fix working (imperfectly) in real time.

For a clearer view, uncheck Show fish to replace the 3D models with center-point spheres, or turn on Radius spheres & distances to see the separation, alignment, and cohesion zones in real time.

Open the Playground →