I make music below two names. LXSTNGHT is the darkish one, future storage and downtempo. F/MEMORY runs hardwave and chill. And I construct for the online. The music issues to this story due to what it pressured: connecting the various things I make (tracks, modeled geometry, real-time code) into single items. The connective tissue is a lab, an experiments gallery in-built Three.js and React Three Fiber, the place every bit exists to present the music someplace to dwell.
That constraint turned out to be the entire technique. A demo generally is a tech showcase; a launch has to carry a temper for 3 and a half minutes at 60 fps on no matter machine the listener owns. The purpose is at all times the identical: cinematic photos operating in a browser. Actual gentle, actual ambiance, and a body funds that survives contact with different folks’s {hardware}. So the lab turned a spot the place the renderer will get interrogated: what does this impact really price, what is that this body really product of, why did that one body die.
This text is a tour of three our bodies of labor and the Three.js classes they price me: FRONTIER, a 3D Area Invaders with my songs within the jukebox; the LXSTNGHT collection, seven fly-through worlds operating on one shared engine; and /fm, a wall of failing CRT displays that performs the F/MEMORY file. The geometry is modeled in Blender, that’s a home rule, nothing here’s a primitive carrying a intelligent shader. However this piece is in regards to the different half of the pipeline: what occurs after useGLTF resolves.
FRONTIER: the sport
Play it → lab.filipzrnzevic.com/frontier
FRONTIER is Area Invaders staged like a broadcast: a voxel fleet baked in Blender, an open deck dissolving into fog, one highlight rig, and a simulation that runs all the struggle outdoors React. The fleet ships as a single Draco GLB and renders as a stack of InstancedMesh swimming pools: six invader varieties in a two-frame stepped march, two materials slots apiece, plus bosses that erode voxel by voxel as you strip their hull.
The ground is a shader, as a result of each texture failed
The deck went by means of 4 full rejections (a canvas-textured board, a planar reflector, a black-mirror show), and each model learn fallacious for a special cause: fog lifting the midtones, surroundings sheen on the reflector, the important thing gentle’s specular lobe, a base colour that was by no means fairly black. The escape was to cease transport a floor in any respect, after which rebuild the bottom as one analytic ShaderMaterial: a hexagonal lattice computed from a hex-distance operate with fwidth() anti-aliasing. No texture, no mip chain, crisp at each distance, and nothing that may render the fallacious colour.
As soon as the ground was a shader, it might develop into an instrument. The lattice does its personal fog math so it melts into the scene fog as a substitute of combating it. It receives actual shadow maps, which a {custom} ShaderMaterial can do when you embody Three.js’s shadowmap chunks and declare the varyings the chunks anticipate. And it takes the struggle as enter: a small uniform array of emitters lets the ship’s thrust and each falling bomb drag a real pool of sunshine throughout the tiles. Occasion power displaces the lattice within the vertex shader, and a five-line JavaScript twin of that displacement (fieldLift) lets each invader, bunker, and drop experience the identical floor the GPU is bending.
The lesson that generalizes: a custom-shader floor is invisible to your lighting rig. Scene lights don’t contact a ShaderMaterial except you feed them in your self. The patrol searchlight swept that board for every week and lit nothing till it turned a uniform.
Menus turned issues within the lane
Between waves you decide an improve. The primary model was a modal, three playing cards, the sport paused behind a scrim, and it flattened the entire piece. The model that shipped: three hex gates standing within the lane, closing on you whereas the fleet is below manner. You are taking an improve by flying by means of one. Identical enter as the remainder of the sport, no pause, and the selection prices one thing, as a result of taking the left gate means giving up the precise.
The half that also makes me smile: RingGeometry(r, r + 0.34, 6) is a hex ring. The board’s personal cell, scaled right into a doorway. No modeling, no authoring, six segments.
The file is the explanation the sport exists
The jukebox performs F/MEMORY, the complete file, latest monitor first. The rule I’d give anybody constructing a music-led sport: the soundtrack is the product, the SFX are punctuation. Each sound impact in FRONTIER runs by means of one international bus that geese laborious whereas a monitor performs, so a twenty-minute run is twenty minutes of listening with a struggle on prime, not the opposite manner round.
What the struggle prices, measured
The receipts, from a headless bench on the true GPU (an M1 Max, Chrome on ANGLE/Metallic): mid-fight, at a 5.2-megapixel drawing buffer, all the body is 2 ms of GPU time on the ninety fifth percentile. Swarm, shadow move, HDR bloom, composite, all of it. The sport was by no means fill-bound. Three findings from getting it there are value stealing instantly:
Finances pixels, not device-pixel-ratio. dpr={2} on a big window quietly handed the sport a 9.7-megapixel drawing buffer. The identical launch config on a 5K show and a 14-inch laptop computer differ by greater than 2x in pixels on the similar DPR. FRONTIER caps the buffer at an absolute pixel funds (5.2 MP, with LOW/HIGH tiers within the settings) and derives DPR from it. Equivalent output, half the fill.
By no means put a mix-blend-mode over a WebGL canvas. The demise flash was a fullscreen DOM veil with mix-blend-mode: colour. Each body it was seen, the browser’s GPU course of paid 30–45 ms compositing the entire 5-megapixel canvas by means of Skia’s gradual path. Invisible to each in-page profiler: the draw passes measured harmless, the primary thread was idle, and the sport nonetheless hitched. The veil is now a fused composer impact, the identical CSS colour mix math in fifteen traces of GLSL. In the event you’re grading a canvas, grade it in the canvas.
Lights toggle depth, by no means visibility. Three.js bakes the scene’s gentle rely into each lit materials’s shader as a compile-time fixed, so the rely is a part of this system cache key. FRONTIER stored level lights inside teams that toggled seen (the ship, the boss, the beam weapon), and the primary body of each dramatic entrance modified the sunshine rely and recompiled ~20 packages mid-fight: a number of hundred milliseconds of useless cease, at all times “throughout an explosion”, for weeks.
{/* at all times seen */}
{/* the hull toggles */}
A light-weight at depth 0 nonetheless occupies its slot within the rely, so this system key by no means adjustments. That one structural change took mid-fight shader compiles from one or two per run to zero, and whole packages from 140 to 98. The total hunt, together with the methods three totally different profilers lied earlier than one advised the reality, is its personal detective story, and it’s coming as a separate write-up.
LXSTNGHT: one engine, seven worlds
The shelf → lab.filipzrnzevic.com/lxstnght
The LXSTNGHT collection is seven fly-throughs (longeron, flue, ambit, breach, hypostyle, marrow, derelict) constructed to advertise the file. Every one is a spot: a bored derelict hull, a biomech hall, a furnace shaft you ascend, an 84-meter processional corridor. You scroll, you fly, the music performs.
One factor to be upfront about: these are experiments, and so they sit at very totally different phases of polish and optimization. Some have had full efficiency passes, some are nonetheless paying for his or her first one. That’s what the shelf is for. The entire level is to maintain pushing every of them additional, extra cinematic and higher optimized for the browser, and the teachings beneath are what that push retains costing.
They keep shippable as a collection due to one contract: the runtime dispatches on materials identify. HullMetal will get triplanar weathered steel and static batching. Strip and HoleGlowMat are emissive practicals. Reactor* is a light-weight locator, hidden at runtime, that the volumetric phases from. Swap the GLB, hold the engine: digital camera rig, HUD, publish stack, audio map, all already wired. There’s even a spare piece on the bench (epigraph) constructed largely to show the swap works.
That’s the structure. The remainder of this part is the Three.js engineering the collection bumped into: the costly classes, with the piece that paid for every one.
Draw calls: 1,421 → 25, then un-learning half of it (longeron)
longeron is a 30-meter hall ported from a Blender scene of ~1,500 objects. First measurement: 1,421 draw requires 568k triangles. One name per bolt. Draw-call certain, which is why a 5K show crawled whereas the triangle rely appeared nice.
The repair is the traditional one: clone every geometry, bake its matrixWorld in, strip to a shared attribute set, group by materials identify, mergeGeometries. 25 draw calls, no visible change.
Then the correction, which is the extra helpful half. One 30-meter mesh per materials means its bounding sphere at all times intersects the frustum, so the merge had silently disabled frustum culling, and the entire hall was submitted each body. I’d dismissed that, reasoning a hall considered down its personal axis has nothing outdoors the view. True of what’s forward. Fully fallacious about what’s behind you. Re-merging per materials per 3-meter z-segment introduced again culling and reduce submitted triangles 21–37% relying on the place you stand.
And the trick that makes merged scenes animatable in any respect: the merge destroys per-object identification, so bake identification into vertex attributes when you nonetheless have it. Through the merge, every supply object writes its bounding-box middle and a hash of it into two {custom} attributes. That’s what lets a single merged wall carry per-panel gentle animation, with each panel flat-lit at any immediate as a substitute of smeared by a gradient.
HDR is a self-discipline, not a flag (longeron, marrow, derelict, breach)
The collection’ look is toneMapped: false emissives scorching sufficient for Bloom to halo, actual HDR values transferring by means of the composer. Three bugs taught me what that truly calls for, and all three put on disguises:
The composer buffer is 8-bit except you say in any other case. A hall stored “bleeding white” by means of six repair makes an attempt (metalness, gentle peak, bloom threshold, publicity), and the precise trigger was frameBufferType={THREE.HalfFloatType}. Any composer carrying HDR emissives wants it.
Bloom’s default mix is an LDR operator. The collection’ longest-running bug, black shapes carrying glowing rims throughout three items, got here all the way down to postprocessing‘s Bloom defaulting to BlendFunction.SCREEN: 1-(1-scene)(1-bloom). When scene and bloom each exceed 1.0, which is strictly a scorching emissive haloing, the 2 negatives multiply large-positive and the outcome flips unfavorable. (1-9)(1-3) = +16, so 1-16 = -15. Black. It even explains why elevating an emissive made issues worse: depth 4 to 9 crossed the flip threshold. One prop: blendFunction={BlendFunction.ADD}. Additive can by no means invert.
A grade can mint NaN. BrightnessContrast is linear. At distinction 0.12, black maps to -0.068. An 8-bit buffer would have clamped that; the HalfFloat buffer you simply put in fortunately shops it, and Three.js’s sRGB encode then evaluates pow(unfavorable, 0.41666). NaN, which shows as white. The repair is a five-line clamp impact sitting instantly after the grade. In the event you see white speckle that solely seems when a grade is on, test for this earlier than touching the rest.
Gentle that travels (flue, longeron)
The signature transfer of the collection is gentle transferring by means of structure. A cost operating down a 28-meter neon rail, a crest sweeping a wall of panels. Two guidelines made it learn:
A wave means the fixture is OFF. Each early model stored the run lit and rode a modulation on prime, and nothing learn as travelling, as a result of there was at all times one thing to see alongside the entire size. The model that works holds the strip black and runs one vibrant head alongside it: a sawtooth with a brief responsibility cycle, a tough smoothstep vanguard, an extended tail. If the remainder of the fixture remains to be glowing, you’ve constructed a modulation, not a wave.
Additive, previous white. The pinnacle provides a near-white push on prime of the fixture’s personal colour, intentionally overshooting 1.0 into the HalfFloat buffer so Bloom halos precisely the crest. Multiplying simply makes the strip a brighter model of itself. Including is what sells it as a cost.
And per-strip desync is non-negotiable. 4 completely synchronized rails learn as one pretend machine. With object identification gone within the merge, every run’s part comes from hashing its fixed lateral place.
The music is choreographed, not reactive
Each piece within the collection strikes with its monitor, and none of it’s a dwell FFT deciding issues at runtime. The belief that reframed it: the file doesn’t change, so a kick is just not a real-time query. An offline evaluation move bakes BPM, the beat grid, kick occasions, and per-frame band envelopes to JSON per monitor. The runtime is a lookup towards audio.currentTime.
The explanation isn’t purity. It’s that dwell detection is fallacious. Band degree says “there’s bass”, which is true of each body of each mastered monitor. A sustained bass notice is as loud as a kick and isn’t one. A kick is a transient, spectral flux, the optimistic frame-to-frame change. And offline, you possibly can afford to detect sensitively, then hold solely what lands on the beat grid.
Downstream, one doctrine: a beat causes a discontinuity. Spend it on a small steady brightness nudge and “the audio reactivity does nothing”. In longeron, a beat doesn’t brighten the sunshine heads. It lends the wave most of a pitch of journey, paid out over 200 ms, so the costs visibly sprint on the kick and coast between. Solely loudness-to-gain is steady.
Measure, don’t theorize (breach)
The most recent piece obtained the complete efficiency move, and the punchline is a warning about instinct. p50 mentioned a clear 60 fps. p95 was 49.6 ms, and p95 is the quantity you’re feeling. The probe (40 traces, accumulating renderer.information with autoReset = false, as a result of the composer resets it a number of occasions per body) discovered 1.88 million triangles per body towards 944k seen: one magnificence move plus the volumetric’s depth prepass.
The one greatest win wasn’t a Three.js change in any respect. A 13-mm two-segment bevel modifier on 1,600 greeble containers, sub-pixel at each distance the digital camera can attain, was carrying 27% of the scene’s triangles. Pure vertex price for a spotlight no one can resolve. After chopping it and yet another move: p95 17.9 ms, zero stalls, at extra decision.
And the repair that delivered nothing is the higher lesson. Flagging background meshes to skip shadows modified nothing, as a result of the runtime merges statics into one batch and a batch carries one castShadow flag. Batching erases per-object properties. Any per-mesh flag has to develop into a batch cut up, or it silently does nothing.
/fm: the file, taking part in on a wall of dying displays
Go to → lab.filipzrnzevic.com/fm
The F/MEMORY aspect obtained a special sort of piece: a monitor den. Twenty-four CRTs racked in a fogged room, every one a dwell canvas texture (spectrum readouts, clocks, static), and the file taking part in by means of it. There’s no transport UI. You drag the glass to clean, faucet a display screen to kill it to static.
Three issues below the hood earned their hold. The 24 panels repaint on a staggered 24 Hz schedule, every tube by itself slot, so twenty-four dwell textures price lower than the unique eleven did at 60. The monitor title is solid onto the again wall as gentle, tiled and scattered like a gobo throw; it began as an actual SpotLight.map till I measured that Three.js collects lights globally (a light-weight can’t be masked to particular objects), so it turned a world-space planar texture injected into the emissive time period of precisely two supplies: the wall, and the cables that occlude it. And one monitor is a CCTV feed of the room itself, rendered from a second digital camera within the nook. Bounded recursion, since that display screen hides throughout its personal feed render. It’s additionally intentionally the worst tube on the wall, 12 fps with a dragging roll bar, as a result of a clear second view reads as a bug and a degraded one reads as surveillance.
What retains proving true
Each undertaking above re-taught a number of the similar classes. The brief model, as guidelines I now construct by:
- Finances pixels, not DPR. A 5K show on the similar ratio prices greater than double a laptop computer. Cap the drawing buffer in megapixels.
- Decide the worst percentile, by no means the median. Vsync pins the median at 60 lengthy after your funds is gone.
- The sunshine rely is a part of each lit shader’s cache key. Lights toggle depth, by no means visibility.
- HDR by means of a composer is a sequence of custody: HalfFloat buffer, additive bloom, clamp after the grade. Break any hyperlink, and the failure reveals up someplace else, in disguise.
- Batching erases per-object flags. Break up the batch, or the flag is a no-op.
- Bisect by management, not by code. Each piece ships a swap panel, a dozen keyed toggles over lights, fog, grades, results. The collection’ worst bug was localized in seconds by flipping channels, after 4 rounds of shader concept did not.
- A kick is just not a degree, and it isn’t a real-time query. Bake the evaluation; choreograph, don’t react.
- Measure earlier than theorizing. One Blender bevel beat each decision lever within the toolbox, and no one suspected it.
The place to seek out it
The open doorways are the three above: FRONTIER, the LXSTNGHT shelf, and /fm. The remainder of the lab lives behind a gate, on objective: it’s a working sketchbook, and items graduate to a public handle after they’re prepared. Each piece right here is an experiment mid-push, towards extra cinematic and higher optimized. Subsequent from the lab: the complete story of the FRONTIER freeze. Three mendacity profilers, a shader compile hiding within the gentle rely, and the pipeline-state layer beneath all of it.
Constructed with Three.js, React Three Fiber, pmndrs/postprocessing, GSAP, and Blender. Due to the Three.js and pmndrs maintainers. Each behaviour described here’s a cheap engineering commerce; writing them down is the tax.
