Constructing an Countless Interactive Glass Xylophone with Three.js

Editor’s Observe: The primary Three.js Convention is coming to Paris this September, and we’re stepping into the spirit of it! Over the approaching weeks, we’ll be bringing you plenty of Three.js gems from fantastic folks in the neighborhood, generously sharing their experiments, methods, and artistic concepts with us. We’re kicking issues off with this lovely interactive xylophone by Sujen! Get pleasure from!

This can be a xylophone you play along with your cursor. Sweep throughout the bars and every one rings its be aware, swings like a struck chime, and floods with color because the wake passes over it.

The column by no means ends, both. Scroll so long as you want and there are all the time extra bars, sixty-four of them biking eternally up a helix of frosted glass.

I’ve all the time been drawn to supplies which might be virtually translucent however nonetheless maintain a little bit of sheen, sufficient to catch gentle and keep distinct. Fancy, however minimal. That is the place that concept landed.

Right here’s what we’ll cowl:

• An limitless scroll the place nothing strikes. The entire helix is laid out from a single quantity, so bars recycle from high to backside and the composition by no means shifts.
• A fluid simulation used as a masks. Each bar already owns its color, completely, the way in which an actual xylophone key owns its be aware. The simulation decides how a lot of that color you’re at present allowed to see.
• Convincing frosted glass. Why MeshPhysicalMaterial doesn’t work on instanced geometry, and tips on how to pretend each optical cue as an alternative with an offscreen blur, screen-space refraction, Fresnel, iridescence, and ambient occlusion standing in for contact shadows.
• Swinging animation with no CPU state. Sixty-four bars swinging independently, pushed by one timestamp every and some strains of vertex shader.

It’s constructed with Three.js on plain Vite, with postprocessing working the cross pipeline. It assumes you’re comfy with Three.js and have written a shader earlier than.

1. The Idea

This one began with doomscrolling. I’ve a love-hate relationship with Twitter, the place I get impressed by everybody’s nice concepts after which really feel barely buried by everybody’s nice concepts. Anyone else?

I got here throughout a submit I couldn’t cease : the interplay, the colors, the smoothness of the animation. So I got down to work out how. And to make it extra fascinating, I made a decision to fold in a conventional xylophone too.

2. The Implementation

We begin by establishing the form and the helix, then add the scroll and the interplay, then the fabric that sells the xylophone’s glassiness.

The Form and The Helix

I modelled the bar as a capsule harking back to a xylophone key, however rounded, so it has some depth to catch gentle with. It exports as a single bar in a .glb, and we occasion it 64 occasions from one InstancedBufferGeometry.

I modelled the bar as a capsule harking back to a xylophone key, however rounded, so it has some depth to catch gentle with. It exports as a single bar in a .glb, and we occasion it 64 occasions from one InstancedBufferGeometry.

const geometry = new InstancedBufferGeometry()

geometry.index = mannequin.index

geometry.setAttribute("place", mannequin.getAttribute("place"))

geometry.setAttribute("regular", mannequin.getAttribute("regular"))

geometry.setAttribute("uv", mannequin.getAttribute("uv"))

geometry.setAttribute("aPos", new InstancedBufferAttribute(transforms.positions, 3))

geometry.setAttribute("aRot", new InstancedBufferAttribute(transforms.rotations, 4))

geometry.setAttribute("aTintOffset", new InstancedBufferAttribute(transforms.tintOffsets, 1))

Laying these 64 out right into a helix is usually what you’d anticipate. Stroll an angle round a circle, step up in Y, flip every bar to face outward. The half that isn’t apparent is the third rotation:

const tiltX = Math.atan2(geometryHeight, cfg.radius * cfg.tiltFalloff)

for (let i = 0; i < cfg.rely; i++) {

  const s = wrap(i + section, cfg.rely)

  const theta = s * cfg.thetaStep + cfg.thetaOffset

  positions[i * 3] = cfg.radius * Math.cos(theta)

  positions[i * 3 + 1] = s * geometryHeight - halfHeight

  positions[i * 3 + 2] = cfg.radius * Math.sin(theta)

  e.set(tiltX, -theta, 0) // Euler order "YXZ"

  q.setFromEuler(e)

}

tiltX is what lets every bar lean into the climb as an alternative of sitting flat. Consider a spiral staircase: with out the lean you get the steps, stage plates stacked one above the subsequent. However with it, you get the handrail, working easily alongside the slope. The angle itself is simply rise over run. The rise is one bar top, as a result of that’s how far we go up between one bar and the subsequent. The run is how far we journey sideways in that very same step, which scales with the radius: a large helix climbs gently, a slender one climbs steeply. Math.atan2 turns these two lengths into the angle, and tiltFalloff is a configuration on high to flatten or elevate it.

The Euler order issues as properly. "YXZ" applies the worldwide rotation earlier than including the customized tilt onto the bar’s native X rotation.

Vertical spacing is precisely one bounding-box top, so the bars stack flush with no hole. Angular spacing works out to a bit underneath two full turns throughout the 64. The entire group is then tilted and rolled so the helix enters top-left and leaves bottom-right, and scaled down to suit.

The Scroll

Scrolling doesn’t transfer the group, or the digicam, or anything. Scroll accumulates right into a single section worth measured in bar-index models, which will get eased and fed into the wrap() from that loop above:

operate wrap(a: quantity, n: quantity): quantity {

  return ((a % n) + n) % n

}

As a result of section feeds the structure fairly than an index, every bar slides repeatedly up the spiral, and the second one passes the highest it reappears on the backside. The envelope (how tall it’s, how extensive, the way it sits in body) is mounted. You possibly can scroll for ten minutes and the composition stays precisely the place I put it.

Two small issues maintain it low-cost. The easing makes use of MathUtils.damp fairly than a lerp, so it settles on the identical velocity on a 60Hz display and a 120Hz one. And the occasion buffers are solely rewritten when the eased section has really moved.

Observe: 64 bars at this angular step will trigger the recycling to leap rotation, although it’s invisible as a result of it occurs out of body. If you’d like the recycling to be genuinely seamless, decide a step that divides evenly into 2π.

The interplay

I spent an embarrassing variety of hours on the color and the velocity of the motion, then many extra simply taking part in with it.

The color was the onerous half. I knew I needed a fluid simulation, however I additionally needed each bar to maintain its personal color, like an actual xylophone. My first intuition was to gentle the entire bar up on hover. It labored, but it surely was boring.

Right here’s the association I landed on. Each bar carries one quantity: its place within the row, from 0 on the backside to 1 on the high. This quantity is its everlasting slot on a gradient texture.

The gradient is constructed at runtime on a canvas: a prism ramp working pink, magenta, violet, blue, cyan.

const ctx = canvas.getContext("2nd")!
const grad = ctx.createLinearGradient(0, 0, width, 0)
grad.addColorStop(0.0, "#ff0033")
grad.addColorStop(0.3, "#ff00d4")
grad.addColorStop(0.5, "#6a00ff")
grad.addColorStop(0.8, "#0090ff")
grad.addColorStop(1.0, "#00ffe1")
ctx.fillStyle = grad
ctx.fillRect(0, 0, width, 1)

The fluid’s velocity subject then decides how a lot of it you’ll be able to see:

float velocity = smoothstep(0.0, 0.2, size(texture2D(u_tFluid, vScreenUv).xy));
float reveal = clamp(velocity * u_fluidStrength, 0.0, 1.0);
vec3 tint = texture2D(u_tGradient, vec2(fract(vTintOffset * u_tintWrap), 0.5)).rgb;

One thing value noticing is that every fragment samples the fluid’s velocity at its personal place on display, so the reveal follows the form of the swirl because it drifts and decays. At relaxation the bars sit near-white. Sweep throughout and the wake uncovers every bar’s personal color, by no means the cursor’s.

We add a config u_tintWrap, so the ramp cycles ten occasions up the 64 bars as an alternative of as soon as. That is the quantity to play with for those who construct one thing related.

The fluid simulation itself is a reasonably normal Secure Fluids solver however closely simplified. It runs at 128×128. It does a single strain iteration the place an actual solver would do dozens. There’s barely any vorticity.

The one factor I didn’t skimp on is splatting alongside a section fairly than at a degree:

vec2 ab = b - a;
float t = clamp(dot(uv - a, ab) / max(dot(ab, ab), 1e-6), 0.0, 1.0);
vec2 p = uv - (a + t * ab);
vec3 splat = exp(-dot(p, p) / (u_splatRadius / 50.0)) * u_splatColor;

Splatting alongside the road from the earlier place to the present one creates a clean brush stroke.

Understanding which bar you’re hovering is intentionally unsophisticated. We loop over all 64 and raycast towards a bounding field. The trick is pushing the ray into every bar’s native house fairly than reworking the field, which is less expensive. Each short-term object is allotted as soon as up entrance, so a body of selecting allocates nothing.

The strike

While you sweep throughout a bar, it swings. The apparent method to construct that could be a tween per bar, ticked on the CPU each body. I went with the cheaper route as an alternative and animated it within the shader.

float dt  = u_time - aStrikeTime;
float env = step(0.0, dt) * exp(-dt * SWING_DECAY);
float ang = env * SWING_AMP * sin(dt * SWING_FREQ) * u_swingScale;

vec4 swing = vec4(normalize(u_swingAxis) * sin(ang * 0.5), cos(ang * 0.5));
rot = qmul(rot, swing);

The method is a damped pendulum, written out in a single line as an alternative of stepped by body by body. Give it the time for the reason that strike and it fingers again an angle, extensive at first and dying out after a few second. All of the CPU ever does is be aware when every bar was hit, so sixty-four bars can swing independently and it by no means is aware of.

There are two small particulars to notice right here:

• aStrikeTime is initially set to a time very far prior to now, so decay collapses to 0.
• The order of qmul is necessary. Proper now, the bar pivots round its finish. If flipped, the bars will orbit across the centre of the helix.

The sound

Then, after all, a xylophone must make sound. I downloaded a single be aware pattern and pitch-shifted it throughout a pentatonic scale, so sweeping throughout bars in a rush doesn’t sound horrifying.

We used three octaves up from C5, which supplies fifteen notes throughout sixty-four bars, so roughly 4 neighbouring bars share a pitch. That’s deliberate. Spreading sixty-four distinct notes throughout the vary would push the extremes removed from the pattern’s authentic velocity, and a pattern stretched that arduous begins to sound like a cartoon.

Observe: Notes hearth when the cursor enters a bar fairly than whereas it sits there, which is nearer to how an actual xylophone behaves.

The fabric

Now the half that began all of this: the glassy, matte, faintly shiny floor. This can be a moodboard of the best materials I needed.

The apparent strategy is MeshPhysicalMaterial with transmission turned on, and I wish to be clear I didn’t purposefully keep away from it. It genuinely can’t work right here. All 64 bars are a single instanced draw name, and Three.js can’t depth-sort situations towards one another. So the bars keep utterly opaque.

We run a number of passes to get the best materials. The passes run on this order:

1. render the background by itself, into its personal goal
2. blur that background right into a second goal
3. render the scene, with the bars studying that blurred background as they draw
4. render a standard buffer for the bars
5. ambient occlusion, then antialiasing

The blurred backdrop

Step 1 makes use of a layer masks fairly than a second scene. The background quad lives by itself layer; the cross narrows the digicam’s masks to that layer, renders, then places the masks again.

Step 2 fingers the sharp background to postprocessing’s GaussianBlurPass, which blurs one goal into one other and manages its personal scratch buffers. It really works at 1 / 4 of the display decision and runs a number of passes over it, which supplies a large, milky frost with out paying for a large kernel at full dimension. A single frost worth sits on high of it, so the energy is tunable at runtime.

Transmission + Refraction

After which, in spite of everything that setup, transmission is a single texture learn:

vec2 buv = vScreenUv + N.xy * u_refractStrength; // screen-space refraction
vec3 trans = texture2D(u_tBackdrop, buv).rgb;
trans = combine(trans, tint, reveal);
vec3 frosted = combine(physique, trans, u_transmission * (1.0 - 0.6 * reveal));

Nudging that lookup by the floor regular is what bends the background on the edges and sells the thickness.

The (1.0 - 0.6 * reveal) is a small factor I like. Wherever the wake is revealing color, we let much less milky backdrop by, so the tint stays vivid as an alternative of being washed out by frost.

The blur is genuinely onerous to see towards a plain background, so there’s a patterned-background toggle sitting behind the dev-only tuning panel. It by no means ships, but it surely’s the very first thing I attain for when the frost seems improper.

Fresnel + Iridescence

Fresnel comes subsequent, mixing the bar towards a sky color at grazing angles. That’s what provides the perimeters definition and lifts every bar off the background. There’s no setting map behind it, only a three-stop ramp from floor to horizon to sky, generated within the fragment shader. The sheen solely reads at grazing angles, the place an actual HDR is indistinguishable from a gradient, so it isn’t definitely worth the megabyte.

The bars nonetheless appeared boring, so I added iridescence, a cosine palette standing in for actual thin-film interference. There’s a time period within the section that varies with the floor regular, appearing as pretend thickness variation so the inside isn’t one flat hue. It’s low-cost, and it does the job.

Contact Shadows

Then contact shadows, besides there’s no flooring. The composition floats in house, so there’s nothing for a shadow to fall onto. As an alternative the bars occluding one another grow to be the shadow, through SSAO.

That wants one non-obvious piece. SSAO needs a buffer of floor normals, and Three’s built-in regular cross attracts with a generic materials that is aware of nothing about our per-bar spin and strike swing. So we render normals utilizing the identical vertex shader because the show materials, sharing the identical uniform objects by reference fairly than copying values throughout. Shared references can’t drift aside, so the conventional buffer bodily can not fall out of step with what you see.

We end with SMAA to scrub up the perimeters.

3. The Refinement

There are quite a lot of passes right here, which doesn’t sound like a recipe for efficiency. So fairly than chopping passes, I made every one low-cost.

The fluid clear up is idle-gated. After a few seconds with out enter, the complete clear up is skipped. And the scroll conveyor rewrites its buffers solely when the eased section has genuinely moved, so settled scroll means zero rewrites and nil uploads.

After that it’s a handful of small selections. The fluid simulation runs small and coarse. One background render feeds each the sharp copy for refraction and the blurred copy for transmission.

Telephones get their very own tier. Gadget pixel ratio is capped in every single place. On a coarse-pointer machine with a small viewport the cap drops additional, and the 2 full-resolution buffers (the floor normals and the occlusion cross) drop to half. These three collectively are what a cellphone GPU can’t maintain at 60fps; all the things else was already low-cost sufficient to go away alone.

Resizes are measured from a sizer aspect fairly than the window, and guarded so an identical sizes don’t churn render targets.

4. The Accessibility

The bars are the fascinating case right here, and the repair is one line:

this.uniforms.u_swingScale.worth = reduceMotion ? 0 : 1

Multiply the swing angle by zero and the movement is gone. Nothing else adjustments. The bar nonetheless lights up, the be aware nonetheless performs, the color nonetheless reveals.

The tempting transfer is to gate the entire interplay behind one boolean. However that doesn’t hand somebody a calmer model of the instrument; it takes the instrument away from them. Lowered movement ought to take away the vestibular drawback, not the function.

Observe: there’s no keyboard path to taking part in the instrument. Hanging a bar wants a pointer raycast, so a keyboard consumer will get the web page and the sound toggle however can’t play a be aware. The repair I’d attain for is arrow keys strolling a spotlight index by the bars and firing the identical strike path.

Abstract

The takeaway, which I refused to confess, is how a lot effort it takes to make one thing look easy.

Just a few issues value attempting:

• Decide an angular step that divides evenly into 2π for a genuinely seamless loop
• Swap the pentatonic for a scale you want higher
• Push u_tintWrap round and watch how a lot of this factor’s persona lives in that one quantity.

Credit