La-Fabrik

La-Fabrik is an interactive 3D web experience built with React, Vite, Three.js, React Three Fiber, Rapier, GSAP, MediaPipe, and Zustand.

The current prototype puts the player in a repair-oriented world where they progress through a short mission chain: intro, e-bike repair, power pylon repair, vertical farm repair, then outro. The project also includes a local editor for map, dialogue, subtitle, and cinematic data.

Current Scope

• Playable fullscreen 3D scene at /
• Production map loaded from public/map.json
• Progressive map/model/collision/stage loading overlay
• Player controller with pointer lock, ZQSD movement, jump, octree collision, trigger input, and grab input
• Reusable repair-game flow for ebike, pylon, and farm
• Repair case animation, exploded model scan, broken-part markers, grabbable replacements, snap-to-placeholder placement, install validation, reassembly, and completion
• Shared interaction system for trigger and grab objects
• Rapier physics for gameplay objects while the player keeps a Three.js octree collision controller
• Hand tracking through either a local Python WebSocket backend or browser-side MediaPipe
• Category-based audio manager for music, SFX, and dialogue
• Dialogue manifest, SRT subtitles, subtitle overlay, and dialogue queueing
• Cinematic manifest with GSAP camera keyframes and optional dialogue cues
• In-game settings menu for volumes, subtitles, and subtitle language
• Debug mode with ?debug, lil-gui controls, game-state panel, hand-tracking panel, debug camera, physics playground, and R3F perf
• /editor route for map transforms, SRT editing, dialogue manifest editing, cinematic manifest editing, preview, validation, export, and dev-server saves
• /docs route that renders the repository documentation inside the app

Routes

Route	Purpose
/	Playable 3D experience
/?debug	Playable scene with debug GUI and overlays
/editor	Local map, dialogue, subtitle, and cinematic editor
/gallery	3D model gallery for browsing project assets
/docs	In-app documentation index

Tech Stack

Area	Packages
App	React 19, TypeScript, Vite, TanStack Router
3D	Three.js, React Three Fiber, drei
Physics and animation	@react-three/rapier, GSAP, Three.js AnimationMixer
State	Zustand, custom singleton managers where imperative runtime objects are needed
Hand tracking	@mediapipe/tasks-vision, optional FastAPI backend
Docs	react-markdown, remark-gfm
Quality	ESLint, Prettier, TypeScript project build

Project Structure

.
|-- backend/                         # Optional Python hand-tracking backend
|-- docs/
|   +-- technical/                   # Architecture and implementation notes
|   +-- user/                        # Feature and user-facing guides
|-- public/
|   +-- assets/                      # UI videos, PDFs, logos, world videos
|   +-- cinematics.json              # Runtime cinematic manifest
|   +-- map.json                     # Runtime/editor map data
|   +-- models/                      # GLTF/GLB assets resolved by model folder name
|   +-- sounds/                      # Music, SFX, dialogue audio, SRT subtitles
|-- src/
|   +-- components/
|   |   +-- docs/                    # In-app docs layout and renderer
|   |   +-- editor/                  # Editor panels and editor scene
|   |   +-- three/                   # R3F components by domain
|   |   +-- ui/                      # HTML game/debug overlays
|   +-- controls/                    # Editor fly/player-style controls
|   +-- data/                        # Static tuning/config per domain
|   +-- hooks/                       # React hooks by domain
|   +-- lib/                         # Browser hand-tracking helpers
|   +-- managers/                    # Audio, interaction, and Zustand stores
|   +-- pages/                       # Route-level pages
|   +-- providers/                   # Docs and hand-tracking providers
|   +-- routes/                      # Lazy route wrappers
|   +-- types/                       # Shared domain types
|   +-- utils/                       # Core, map, editor, dialogue, subtitle, Three helpers
|   +-- world/                       # Production/debug world composition and player
`-- vite.config.ts                   # Vite config plus local editor save endpoints

Getting Started

Install and run the frontend:

npm install
npm run dev

Open:

http://localhost:5173

Useful local URLs:

http://localhost:5173/?debug
http://localhost:5173/editor
http://localhost:5173/gallery
http://localhost:5173/docs

Run checks:

npm run typecheck
npm run lint
npm run format:check
npm run build

Regenerate runtime map data after editing public/map_raw.json that came from the hierachy node of the model Blocking.gltf:

npm run map:transform

Optional Hand-Tracking Backend

The app can use the local Python backend, but the default debug source is browser-side MediaPipe.

python3.11 -m venv backend/.venv
source backend/.venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r backend/requirements.txt
python backend/download_model.py
python -m backend.main

Backend endpoints:

GET http://localhost:8000/health
WS  ws://localhost:8000/ws

Documentation Index

File	Purpose
docs/technical/architecture.md	Current runtime architecture
docs/technical/scene-runtime.md	Scene loading, world composition, and player spawn gates
docs/technical/repair-game.md	Repair-game implementation and state flow
docs/technical/interaction.md	Trigger, grab, focus, and hand-grab system
docs/technical/target-architecture.md	Intended medium-term architecture direction
docs/technical/audio.md	Music, SFX, dialogue, subtitles, and editor validation
docs/technical/hand-tracking.md	Webcam, backend/browser MediaPipe, glove, and gesture flow
docs/technical/zustand.md	Game, settings, and subtitle stores
docs/technical/three-debugging.md	DevTools workflow for stepping into Three.js internals
docs/technical/map-performance.md	Map draw-call bottlenecks and optimization notes
docs/technical/editor.md	Editor implementation details
docs/technical/animation.md	Animated, explodable, and reusable 3D model components
docs/user/features.md	Implemented feature inventory
docs/user/main-feature.md	User-facing repair-game walkthrough
docs/user/editor.md	Editor user guide
docs/user/gallery.md	Model gallery user guide
docs/code-review-preparation.md	French code-review preparation support

Current Caveats

• This is still a prototype, not a complete game runtime.
• useRepairMovementLocked() locks player movement during focused repair steps and drives the repair movement indicator.
• Production editor persistence does not exist. Save endpoints in vite.config.ts are local Vite dev-server helpers.
• The player uses octree collision while gameplay objects use Rapier. Keep that boundary deliberate unless the whole player controller is migrated.

License

See LICENSE.