Tom Boullay
134c0aecb7
🔍 Lint / 🪄 Check lint (pull_request) Has been cancelled
🔍 Lint / 🎨 Check format (pull_request) Has been cancelled
🔍 Lint / 🔎 Typecheck (pull_request) Has been cancelled
📊 Quality / 🔒 Security Audit (pull_request) Has been cancelled
📊 Quality / 📋 Dependency Freshness (pull_request) Has been cancelled
📊 Quality / 📦 Bundle Size (pull_request) Has been cancelled
🔍 Lint / 🏗 Build (pull_request) Has been cancelled
Shadows occasionally failed to render on initial load and the Fabrik doorway sometimes blocked the player. Both issues are tracked down to geometry that mounts after Lighting: - Shadows: GLTFs and the merged static map mount imperatively after Lighting, so materials get compiled against a renderer state that pre-dates the final scene and bake a 'no shadow map' permutation, silently dropping shadows. A WebGL context-restore cycle fixes it, but is too invasive. New 'useShadowMapWarmup' hook replays it cheaply: once the scene mesh count has been stable for ~1s, it disposes the directional shadow map (three.js reallocates it on the next render) and marks every material 'needsUpdate' so shaders rebind to the freshly created shadow sampler. - Doorway: the door slab + its Solidify-modifier frame (children of the 'Thicken' parent in the LaFabrik GLTF) sat inside the doorway AABB and prevented the player from walking through. Stripped from the collision octree alongside the existing 'porte' slab; visual rendering is unaffected. Also: extract sun-relative-to-camera placement into a small helper, remove the temporary diagnostic logs, and document the shadow warmup in three-debugging.md.
La-Fabrik
La-Fabrik is an interactive 3D web experience built with React, Vite, Three.js, React Three Fiber, Rapier, GSAP, MediaPipe, and Zustand.
The current prototype puts the player in a repair-oriented world where they progress through a short mission chain: intro, e-bike repair, power pylon repair, vertical farm repair, then outro. The project also includes a local editor for map, dialogue, subtitle, and cinematic data.
Current Scope
- Playable fullscreen 3D scene at
/ - Production map loaded from
public/map.json - Progressive map/model/collision/stage loading overlay
- Player controller with pointer lock,
ZQSDmovement, jump, octree collision, trigger input, and grab input - Reusable repair-game flow for
ebike,pylon, andfarm - Repair case animation, exploded model scan, broken-part markers, grabbable replacements, snap-to-placeholder placement, install validation, reassembly, and completion
- Shared interaction system for trigger and grab objects
- Rapier physics for gameplay objects while the player keeps a Three.js octree collision controller
- Hand tracking through either a local Python WebSocket backend or browser-side MediaPipe
- Category-based audio manager for music, SFX, and dialogue
- Dialogue manifest, SRT subtitles, subtitle overlay, and dialogue queueing
- Cinematic manifest with GSAP camera keyframes and optional dialogue cues
- In-game settings menu for volumes, subtitles, and subtitle language
- Debug mode with
?debug, lil-gui controls, game-state panel, hand-tracking panel, debug camera, physics playground, and R3F perf /editorroute for map transforms, SRT editing, dialogue manifest editing, cinematic manifest editing, preview, validation, export, and dev-server saves/docsroute that renders the repository documentation inside the app
Routes
| Route | Purpose |
|---|---|
/ |
Playable 3D experience |
/?debug |
Playable scene with debug GUI and overlays |
/editor |
Local map, dialogue, subtitle, and cinematic editor |
/gallery |
3D model gallery for browsing project assets |
/docs |
In-app documentation index |
Tech Stack
| Area | Packages |
|---|---|
| App | React 19, TypeScript, Vite, TanStack Router |
| 3D | Three.js, React Three Fiber, drei |
| Physics and animation | @react-three/rapier, GSAP, Three.js AnimationMixer |
| State | Zustand, custom singleton managers where imperative runtime objects are needed |
| Hand tracking | @mediapipe/tasks-vision, optional FastAPI backend |
| Docs | react-markdown, remark-gfm |
| Quality | ESLint, Prettier, TypeScript project build |
Project Structure
.
|-- backend/ # Optional Python hand-tracking backend
|-- docs/
| +-- technical/ # Architecture and implementation notes
| +-- user/ # Feature and user-facing guides
|-- public/
| +-- assets/ # UI videos, PDFs, logos, world videos
| +-- cinematics.json # Runtime cinematic manifest
| +-- map.json # Runtime/editor map data
| +-- models/ # GLTF/GLB assets resolved by model folder name
| +-- sounds/ # Music, SFX, dialogue audio, SRT subtitles
|-- src/
| +-- components/
| | +-- docs/ # In-app docs layout and renderer
| | +-- editor/ # Editor panels and editor scene
| | +-- three/ # R3F components by domain
| | +-- ui/ # HTML game/debug overlays
| +-- controls/ # Editor fly/player-style controls
| +-- data/ # Static tuning/config per domain
| +-- hooks/ # React hooks by domain
| +-- lib/ # Browser hand-tracking helpers
| +-- managers/ # Audio, interaction, and Zustand stores
| +-- pages/ # Route-level pages
| +-- providers/ # Docs and hand-tracking providers
| +-- routes/ # Lazy route wrappers
| +-- types/ # Shared domain types
| +-- utils/ # Core, map, editor, dialogue, subtitle, Three helpers
| +-- world/ # Production/debug world composition and player
`-- vite.config.ts # Vite config plus local editor save endpoints
Getting Started
Install and run the frontend:
npm install
npm run dev
Open:
http://localhost:5173
Useful local URLs:
http://localhost:5173/?debug
http://localhost:5173/editor
http://localhost:5173/gallery
http://localhost:5173/docs
Run checks:
npm run typecheck
npm run lint
npm run format:check
npm run build
Regenerate runtime map data after editing public/map_raw.json that came from the hierachy node of the model Blocking.gltf:
npm run map:transform
Optional Hand-Tracking Backend
The app can use the local Python backend, but the default debug source is browser-side MediaPipe.
python3.11 -m venv backend/.venv
source backend/.venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r backend/requirements.txt
python backend/download_model.py
python -m backend.main
Backend endpoints:
GET http://localhost:8000/health
WS ws://localhost:8000/ws
Documentation Index
| File | Purpose |
|---|---|
docs/technical/architecture.md |
Current runtime architecture |
docs/technical/scene-runtime.md |
Scene loading, world composition, and player spawn gates |
docs/technical/repair-game.md |
Repair-game implementation and state flow |
docs/technical/interaction.md |
Trigger, grab, focus, and hand-grab system |
docs/technical/target-architecture.md |
Intended medium-term architecture direction |
docs/technical/audio.md |
Music, SFX, dialogue, subtitles, and editor validation |
docs/technical/hand-tracking.md |
Webcam, backend/browser MediaPipe, glove, and gesture flow |
docs/technical/zustand.md |
Game, settings, and subtitle stores |
docs/technical/three-debugging.md |
DevTools workflow for stepping into Three.js internals |
docs/technical/map-performance.md |
Map draw-call bottlenecks and optimization notes |
docs/technical/map-lod.md |
Runtime map LOD presets, paths, and workflow |
docs/technical/editor.md |
Editor implementation details |
docs/technical/animation.md |
Animated, explodable, and reusable 3D model components |
docs/user/features.md |
Implemented feature inventory |
docs/user/main-feature.md |
User-facing repair-game walkthrough |
docs/user/editor.md |
Editor user guide |
docs/user/gallery.md |
Model gallery user guide |
docs/code-review-preparation.md |
French code-review preparation support |
Current Caveats
- This is still a prototype, not a complete game runtime.
useRepairMovementLocked()locks player movement during focused repair steps and drives the repair movement indicator.- Production editor persistence does not exist. Save endpoints in
vite.config.tsare local Vite dev-server helpers. - The player uses octree collision while gameplay objects use Rapier. Keep that boundary deliberate unless the whole player controller is migrated.
License
See LICENSE.