Compare commits
97 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 08c10acd48 | |||
| 8d66391fa9 | |||
| 0ab5380b1e | |||
| 5a6596b755 | |||
| 9841b14388 | |||
| 317db48bcc | |||
| fe30596a5a | |||
| acdcb5515b | |||
| 5ad2e27a89 | |||
| 7bcbba4eb1 | |||
| 712fb851ad | |||
| d8b916d31f | |||
| e9808f8473 | |||
| 0ddecaa494 | |||
| 6c36440016 | |||
| f20c6b9961 | |||
| 47b69b01d2 | |||
| 8b0dd31014 | |||
| 171af683f5 | |||
| f820bee64f | |||
| 1538ef93a5 | |||
| 1325b7b2af | |||
| 96be49d358 | |||
| c2f55e3a2f | |||
| 63c2b294c1 | |||
| 62d0dcf531 | |||
| c75c4e0be6 | |||
| 10b0d4fc16 | |||
| 5f113cbba4 | |||
| b1037d5107 | |||
| 1cc3b0e47e | |||
| 00b1ff9e93 | |||
| 675a45f02b | |||
| bbae199105 | |||
| c4cad629c9 | |||
| 18fb5e39e9 | |||
| ff4ead1d24 | |||
| 974f340d33 | |||
| c6283d492c | |||
| 83194df14f | |||
| 918ee49d7c | |||
| c0e7567849 | |||
| 931308c92c | |||
| 4e1ca708b2 | |||
| ca6c8e00b6 | |||
| 220a661d6d | |||
| 0a3966a339 | |||
| be5d03a30c | |||
| ed0683d814 | |||
| d9a92e336c | |||
| 89050331df | |||
| 0f211cc169 | |||
| 6a0215d1a6 | |||
| 2a6a028e1d | |||
| a609314411 | |||
| d1665891f4 | |||
| eb5d4076d1 | |||
| 5177f43d96 | |||
| 7f37f9a747 | |||
| ff1ec56729 | |||
| 386abf06b6 | |||
| a73f9fb951 | |||
| d29b01e398 | |||
| 6edc5f7972 | |||
| ae35eb1dfb | |||
| 4de86f4e58 | |||
| 5b123f9704 | |||
| d1bf438465 | |||
| d2ce990165 | |||
| 7d2a257e84 | |||
| 58eb60292f | |||
| 73c6d7d50d | |||
| d9cacdad12 | |||
| ab88ab722f | |||
| a30a9a2d29 | |||
| d217c3376b | |||
| 864e075b42 | |||
| 3fe5b32de2 | |||
| 72cb9f5be6 | |||
| f708c4cd2e | |||
| 193fc8b4b6 | |||
| c61760dafd | |||
| 18dd2ae49d | |||
| def0609383 | |||
| 19a1d20a97 | |||
| 49ebacfbfb | |||
| 68253fae41 | |||
| 2dabb73d3d | |||
| 4f1b3b4ff3 | |||
| 627c8d4eb9 | |||
| 0b801888f0 | |||
| a180b89ee6 | |||
| 3e66e31117 | |||
| 2c194cdd2e | |||
| feaf502e44 | |||
| cd0afcda8c | |||
| 813c10f3f7 |
+105
-20
@@ -10,17 +10,25 @@ It is now also available to the production repair flow when a mission reaches a
|
||||
|
||||
## Runtime Flow
|
||||
|
||||
1. The browser captures webcam frames in `src/hooks/handTracking/useRemoteHandTracking.ts`.
|
||||
2. Frames are sent to the local Python backend over WebSocket.
|
||||
3. The backend runs MediaPipe hand landmark detection.
|
||||
4. The backend returns hand data including landmarks, handedness, score, center point, and `isFist`.
|
||||
5. React stores the latest snapshot in the hand tracking provider.
|
||||
6. `GrabbableObject` reads that snapshot each frame and uses fist state plus raycasting to grab objects.
|
||||
7. `HandTrackingGlove` reads the same snapshot and places the rigged `gant_l` and `gant_r` models on the detected hands when hand tracking is active.
|
||||
The frontend can run hand tracking with two interchangeable sources, selected from the debug source controller:
|
||||
|
||||
- **Browser JS** (`src/hooks/handTracking/useBrowserHandTracking.ts`) runs MediaPipe `hand_landmarker.task` directly in the browser via `@mediapipe/tasks-vision`. Default for debug.
|
||||
- **Backend** (`src/hooks/handTracking/useRemoteHandTracking.ts`) sends webcam frames as JPEG over WebSocket to a local Python process that runs MediaPipe and returns landmarks.
|
||||
|
||||
Both sources funnel into the same `HandTrackingContext` so all consumers see one shared snapshot:
|
||||
|
||||
1. The active source captures or receives landmarks.
|
||||
2. The hook applies an EMA smoothing pass on the landmarks before publishing the snapshot.
|
||||
3. `HandTrackingProvider` exposes that snapshot through React context.
|
||||
4. `GrabbableObject` reads the snapshot each frame and uses `hand.isFist` plus raycasting to grab objects.
|
||||
5. `HandTrackingVisualizer` paints the SVG hand silhouette overlay on top of the canvas — the primary visualization.
|
||||
6. `HandTrackingGlove` (opt-in, see UI And Debug) places a rigged 3D glove on each detected hand when enabled via the debug toggle.
|
||||
|
||||
All consumers — fist detection, grab raycasting, SVG silhouette, optional 3D glove — read the **same** landmarks from the snapshot. None of them depend on the others.
|
||||
|
||||
## Activation Rules
|
||||
|
||||
Hand tracking is intentionally gated so the webcam and backend are not used all the time.
|
||||
Hand tracking is gated so the webcam and runtime are only spun up when actually needed.
|
||||
|
||||
The debug activation conditions are:
|
||||
|
||||
@@ -28,16 +36,26 @@ The debug activation conditions are:
|
||||
- scene mode is `physics`
|
||||
- the player is near an interaction, is holding an object, or is hand-holding an object
|
||||
|
||||
This keeps hand tracking active while the player is inside an interaction zone, even if the camera is not aimed directly at the object.
|
||||
|
||||
The production repair activation conditions are:
|
||||
|
||||
- active `mainState` is `ebike`, `pylon`, or `farm`
|
||||
- the active mission step is `inspected`, `repairing`, `reassembling`, or `done`
|
||||
|
||||
This keeps the webcam off during `waiting`, `fragmented`, and `scanning`, then enables hand input only when the repair flow is expected to use hands.
|
||||
This keeps the webcam off during `waiting`, `fragmented`, and `scanning`.
|
||||
|
||||
In the current production repair flow, `inspected` uses a two-fists hold gesture to advance to `fragmented`. The hold must last one second and is independent from local object interaction distance once the mission is in the correct state. Keyboard input for the same transition is handled separately by the repair case trigger, so pressing `E` requires the case to be focused through the shared interaction system.
|
||||
### Linger
|
||||
|
||||
Once activation turns off (player walks back out of a trigger zone, or a mission step transitions away), the runtime stays alive for `HAND_TRACKING_LINGER_MS` (2000 ms) before being torn down. This gives MediaPipe enough time to finish initializing the webcam and load the model on a fresh entry — without the linger, a quick walk-through of a trigger zone never produces a detected hand.
|
||||
|
||||
## Provider Stability
|
||||
|
||||
`HandTrackingProvider` always renders the same JSX root (`HandTrackingRuntime`) and exposes `enabled` as a prop. Returning two different element types (`<HandTrackingContext value=IDLE>` vs `<ActiveHandTrackingProvider>`) used to be the historical shape and was the root cause of WebGL context loss: every `enabled` toggle forced React to remount the entire subtree, including the `<Canvas>`, which destroyed the WebGL renderer.
|
||||
|
||||
The two source hooks are therefore mounted in permanence with an `enabled` flag that they early-return on. No webcam or MediaPipe resources are created while `enabled` is false.
|
||||
|
||||
## StrictMode Resilience
|
||||
|
||||
In development, `<StrictMode>` mounts → unmounts → remounts each effect to surface non-idempotent code. The two source hooks delay their actual `start()` call by `HAND_TRACKING_RUNTIME_START_DELAY_MS` (80 ms) and clear the timer on cleanup, so a StrictMode double-mount or a rapid `nearby` flicker never reaches `getUserMedia` twice.
|
||||
|
||||
## Backend
|
||||
|
||||
@@ -52,7 +70,27 @@ The Python process uses MediaPipe and the local model file:
|
||||
backend/hand_landmarker.task
|
||||
```
|
||||
|
||||
The backend sends normalized hand coordinates and landmarks. The frontend treats the values as screen-space inputs, then maps them into world space with the active Three.js camera.
|
||||
The frontend sends JPEG frames at `HAND_TRACKING_FRAME_WIDTH × HAND_TRACKING_FRAME_HEIGHT` (320×240) to keep WebSocket bandwidth low. The backend sends normalized hand coordinates and landmarks.
|
||||
|
||||
## Browser MediaPipe
|
||||
|
||||
The browser path uses `hand_landmarker.task` (float16) downloaded from Google's MediaPipe model storage. The requested webcam resolution is **640×480** (`HAND_TRACKING_BROWSER_CAMERA_WIDTH/HEIGHT`), independent from the backend's 320×240. The float16 model is more sensitive than the backend Python model and needs the higher-resolution frame to detect hands reliably.
|
||||
|
||||
The MediaPipe delegate is currently `"GPU"`. CPU works too but is significantly slower; on a loaded scene the inference drops to ~5fps and the user feels noticeable lag during grab. MediaPipe creates its own WebGL context separate from Three.js, so there is no direct contention.
|
||||
|
||||
A singleton instance of `HandLandmarker` is cached in `src/lib/handTracking/browserHandTracking.ts`. `releaseBrowserHandLandmarker()` is called on cleanup and on WebGL context lost.
|
||||
|
||||
## Smoothing
|
||||
|
||||
MediaPipe at ~10 fps produces noticeable landmark jitter that, when fed raw into the scene, makes both the glove rig and any grabbed object tremble.
|
||||
|
||||
A simple exponential moving average is applied to every landmark before the snapshot is published:
|
||||
|
||||
```ts
|
||||
smoothed.x = previous.x * (1 - factor) + next.x * factor;
|
||||
```
|
||||
|
||||
The factor is `HAND_TRACKING_LANDMARK_SMOOTHING` (0.4). Hands are matched across frames by `handedness` so left/right don't bleed into each other.
|
||||
|
||||
## Frontend Data Shape
|
||||
|
||||
@@ -72,6 +110,17 @@ interface HandTrackingHand {
|
||||
|
||||
`x` and `y` are normalized camera coordinates. `z` is a relative depth value from MediaPipe, not an absolute world-space distance.
|
||||
|
||||
## Fist Detection
|
||||
|
||||
`isFist` is computed in `src/lib/handTracking/browserHandTracking.ts` (`isFist()` function) from landmarks alone — no model, no glove. The check is:
|
||||
|
||||
1. Palm center = mean of landmarks `[0, 5, 9, 13, 17]` (wrist + 4 MCPs).
|
||||
2. Palm size = distance from wrist (landmark 0) to middle MCP (landmark 9).
|
||||
3. For each of the four fingertip landmarks `[8, 12, 16, 20]`, check whether its distance to the palm center is less than `1.05 × palmSize`.
|
||||
4. `isFist === true` iff all four fingertips pass the check.
|
||||
|
||||
The flag is attached to each hand on the snapshot at the publish step (`isFist: isFist(normalizedLandmarks)`) and read directly by `GrabbableObject.tsx` — the SVG visualizer and the 3D glove never participate in the gesture decision.
|
||||
|
||||
## Grab Targeting
|
||||
|
||||
The hand grab logic lives in `src/components/three/interaction/GrabbableObject.tsx`.
|
||||
@@ -106,24 +155,60 @@ This is less expressive than true depth-aware hand movement, but it is more stab
|
||||
The current debug UI includes:
|
||||
|
||||
- `HandTrackingDebugPanel` inside `DebugOverlayLayout` for status, usage, loaded glove model, server state, hand count, and fist state
|
||||
- `HandTrackingVisualizer` for the SVG landmark wireframe fallback
|
||||
- `HandTrackingGlove` for the left-hand `gant_l` and right-hand `gant_r` models in the R3F scene
|
||||
- `HandTrackingVisualizer` for the SVG hand silhouette overlay (always on when tracking is active)
|
||||
- `HandTrackingFallback` for the last-resort hand silhouette overlay (legacy, see below)
|
||||
- `HandTrackingGlove` for the per-hand rigged glove models in the R3F scene, opt-in via the **Show Model** toggle
|
||||
- `r3f-perf` for render performance
|
||||
- `lil-gui` for scene, camera, lighting, interaction, and grab controls
|
||||
|
||||
The hand tracking debug panel is a compact HTML grid outside the canvas. `Model loaded` displays the successfully loaded glove models. The SVG hand wireframe is only a fallback while models are loading or if a glove model fails to load.
|
||||
### SVG Visualizer
|
||||
|
||||
`HandTrackingVisualizer` is the primary hand visualization. It draws a light-blue hand silhouette with a crisp dark-blue outline by:
|
||||
|
||||
1. Filling a palm polygon (landmarks `[1, 5, 9, 13, 17]` plus two synthetic wrist corners) and five finger tubes (thick rounded `stroke` along each finger's joint chain).
|
||||
2. Wrapping the whole thing in an SVG `<filter>` that uses `feMorphology` to dilate the merged alpha by 2 px and subtract the original, producing a single continuous outline around the union — no internal seams where the palm and finger tubes overlap.
|
||||
3. Shrinking every landmark toward the hand centroid by `RENDER_SCALE = 0.65` so the silhouette stays compact and doesn't dominate the screen.
|
||||
4. Overlaying the 21 raw landmarks and 21 bones as faint translucent lines and dots, so the user can still see the MediaPipe data feeding the silhouette.
|
||||
|
||||
The SVG only displays when MediaPipe is active and the debug **Show Model** toggle is off (default). When the toggle is on, the SVG hides and `HandTrackingGlove` takes over.
|
||||
|
||||
### Show Model Toggle
|
||||
|
||||
The `Hand Tracking` debug folder exposes a single visualization switch:
|
||||
|
||||
- `showHandTrackingModel = false` (default): SVG visualizer renders, 3D glove is not mounted at all.
|
||||
- `showHandTrackingModel = true`: SVG visualizer hides, 3D glove gets mounted for the detected hand(s).
|
||||
|
||||
The 3D glove is treated as opt-in legacy because it had bugs (WebGL context loss, finger rig artefacts) and its hit/grab role was never load-bearing — grab has always read landmarks directly.
|
||||
|
||||
### Fallback Overlay (legacy)
|
||||
|
||||
`HandTrackingFallback` draws a simple open-hand or fist silhouette positioned on the detected wrist landmark. It renders for any hand whose glove is in the `"error"` state in `useHandTrackingGloveStatus`. Now that the glove is opt-in and rarely mounted, the fallback effectively only fires in the rare case where the user enables `showHandTrackingModel` and the glove fails to load. It is kept on disk for that edge case but is not part of the default visual path.
|
||||
|
||||
## Glove Models
|
||||
|
||||
The current glove MVP uses `public/models/gant_l/model.gltf` and `public/models/gant_r/model.gltf`, which contain GLTF skins and armatures. Each model is positioned, oriented, and scaled from palm landmarks, then each finger bone chain is rotated toward the matching MediaPipe landmark chain.
|
||||
The 3D glove is **opt-in** via the `Show Model` debug toggle (see UI And Debug). It is not mounted by default; the SVG visualizer is the primary hand UI. The information below applies only when the toggle is enabled.
|
||||
|
||||
The glove models are intentionally smaller than the raw SVG overlay so they do not dominate the camera view.
|
||||
`HandTrackingGlove` loads `public/models/gant_l/model.gltf` for both hands. The right hand applies `scale.x = -1` at the group level to mirror the mesh, so the thumb ends up on the correct side. Both hands therefore share the same rig and the same material.
|
||||
|
||||
The historical `public/models/gant_r/model.gltf` is kept as legacy but is not loaded by the frontend — its GLB embeds three skeletons (`Hand_l`, `Hand_l_pad`, `Hand_r`) plus a `galet` mesh, which made the finger rig unreliable.
|
||||
|
||||
The `gant_l` material is set to `alphaMode: OPAQUE` with `doubleSided: true`. The opaque mode prevents transparency sorting issues that made folded fingers disappear behind the palm; the double-sided flag covers the back faces revealed by the mirror scale on the right hand.
|
||||
|
||||
Two additional glove variants exist on disk:
|
||||
|
||||
- `public/models/gant_l_pad/model.gltf`
|
||||
- `public/models/gant_r_pad/model.gltf`
|
||||
|
||||
They are intended for future swap-by-state usage but are **not yet rigged**. They cannot be animated by MediaPipe landmarks in their current form — re-exporting them from Blender with the same armature structure as `gant_l` is a prerequisite.
|
||||
|
||||
## Known Limitations
|
||||
|
||||
- Production usage is currently limited to repair mission steps that explicitly need hands.
|
||||
- MediaPipe depth is relative and currently not used for stable object depth control.
|
||||
- The virtual hit zone is an approximation based on multiple raycasts, not a real 3D collider.
|
||||
- There is no smoothing layer for hand position or depth yet.
|
||||
- The SVG hand visualization is a fallback, not the primary display when glove models load correctly.
|
||||
- The 3D glove is opt-in only (see `Show Model` toggle). Default visual is the SVG silhouette.
|
||||
- `HandTrackingFallback` is legacy and effectively unused unless the glove toggle is enabled and the glove fails to load.
|
||||
- The right glove is a mirrored copy of `gant_l` rather than its own mesh; in the future a dedicated right-hand model would give a better visual.
|
||||
- The `_pad` glove variants are not rigged yet, so swap-by-state (normal ↔ pad) is not wired in.
|
||||
- Finger bone animation is an approximate landmark-to-bone mapping; it still needs calibration for per-model twist, offsets, and smoothing.
|
||||
|
||||
@@ -16,14 +16,16 @@ Implemented missions:
|
||||
|
||||
## Main Files
|
||||
|
||||
| File | Responsibility |
|
||||
| ---------------------------------------------- | ------------------------------------------------- |
|
||||
| `src/components/three/gameplay/RepairGame.tsx` | Orchestrates the repair step machine |
|
||||
| `src/data/gameplay/repairMissions.ts` | Mission-specific data |
|
||||
| `src/types/gameplay/repairMission.ts` | Mission ids, step ids, guards |
|
||||
| `src/managers/stores/useGameStore.ts` | Global progression and mission transitions |
|
||||
| `src/world/GameStageContent.tsx` | Production placement of the three repair missions |
|
||||
| `src/world/debug/TestMap.tsx` | Debug repair playground placement |
|
||||
| File | Responsibility |
|
||||
| ----------------------------------------------------- | ------------------------------------------------- |
|
||||
| `src/components/three/gameplay/RepairGame.tsx` | Orchestrates the repair step machine |
|
||||
| `src/components/three/gameplay/RepairFocusBubble.tsx` | Dark sphere shroud + cocoon decor during focus |
|
||||
| `src/managers/stores/useRepairFocusStore.ts` | Global flag + center for the repair focus bubble |
|
||||
| `src/data/gameplay/repairMissions.ts` | Mission-specific data |
|
||||
| `src/types/gameplay/repairMission.ts` | Mission ids, step ids, guards |
|
||||
| `src/managers/stores/useGameStore.ts` | Global progression and mission transitions |
|
||||
| `src/world/GameStageContent.tsx` | Production placement of the three repair missions |
|
||||
| `src/world/debug/TestMap.tsx` | Debug repair playground placement |
|
||||
|
||||
## State Machine
|
||||
|
||||
@@ -159,23 +161,22 @@ The repair case appears near the mission object. The player can:
|
||||
|
||||
Both paths move to `fragmented`.
|
||||
|
||||
`useRepairMovementLocked()` locks player movement during focused repair steps and drives the repair movement indicator.
|
||||
|
||||
### Fragmented
|
||||
|
||||
File:
|
||||
Files:
|
||||
|
||||
```txt
|
||||
src/components/three/models/ExplodableModel.tsx
|
||||
src/utils/three/ExplodedModel.ts
|
||||
```
|
||||
|
||||
The mission object is shown split apart. A timer then moves the mission to `scanning`.
|
||||
The mission object is shown split apart. `RepairGame` mounts a **single** `ExplodableModel` instance for the entire repair flow (`fragmented` -> `done`) so the model loads once, animates from its real authored positions, and is never re-instantiated when the player advances to scanning, repairing, reassembling or done. This eliminates the visible position/rotation jumps and re-explosion that occurred when each step instantiated its own model.
|
||||
|
||||
The default delay comes from:
|
||||
`ExplodedModel.createParts` walks the GLTF tree recursively, descending through any single mesh-bearing wrapper node (e.g. `Scene > Moto > Eclatement` for the Ebike) until it reaches a node with multiple mesh-bearing children. Those children are the natural "explosion groups" authored by the modeler. This avoids exploding raw leaf meshes in local space when the model has extra empty wrapper nodes above the intended group.
|
||||
|
||||
```txt
|
||||
REPAIR_FRAGMENTATION_SEQUENCE_SECONDS
|
||||
```
|
||||
When mounted, `RepairGame` applies `RepairMissionConfig.modelRotation` and `modelScale` to the shared model so it lines up with the source inspection model in world space (e.g. the parked Ebike using `EBIKE_WORLD_ROTATION_Y` / `EBIKE_WORLD_SCALE`). The explode/reassemble lerp speed is configurable via `splitSpeed` (default `REPAIR_FRAGMENT_SPLIT_SPEED = 1.8`, ~1.5s) so each node is clearly seen leaving its origin.
|
||||
|
||||
Transition out is event-driven: the model fires `onSplitSettled(1)` when the lerp converges and `RepairGame` advances to `scanning`. A `REPAIR_FRAGMENTATION_SEQUENCE_SECONDS + 2` fallback timer guards against load failures.
|
||||
|
||||
### Scanning
|
||||
|
||||
@@ -185,50 +186,33 @@ File:
|
||||
src/components/three/gameplay/RepairScanSequence.tsx
|
||||
```
|
||||
|
||||
The scan sequence:
|
||||
The scan sequence is now stateless w.r.t. the model: it receives `parts: ExplodedPart[]` from the upstream shared `ExplodableModel` and:
|
||||
|
||||
- keeps the exploded model visible
|
||||
- receives model parts from `ExplodableModel`
|
||||
- advances an active part index over time
|
||||
- renders `RepairScanVisual` on the active part
|
||||
- reveals broken-part highlights when configured broken parts have been reached
|
||||
- reveals broken-part highlights cumulatively as scan progresses
|
||||
- when the active part has a `voiceLineId`, gates the advance on the audio's `ended` event (with a 15s ceiling fallback) so the diagnostic line plays in full
|
||||
- returns `RepairScannedBrokenPart[]` when done
|
||||
|
||||
Broken-part lookup first tries `brokenParts[].nodeName`. If no configured node matches, it falls back to the first available exploded parts. This fallback is useful while GLTF node names are still unstable, but precise `nodeName` config is safer for production.
|
||||
Broken-part lookup uses `brokenParts[].nodeName` against the exploded parts (deep traverse). When a configured node can't be matched, the available part names are logged so config drift is visible in the console.
|
||||
|
||||
### Repairing
|
||||
|
||||
File:
|
||||
For pylon/farm:
|
||||
|
||||
```txt
|
||||
src/components/three/gameplay/RepairRepairingStep.tsx
|
||||
```
|
||||
|
||||
This is the densest gameplay step.
|
||||
This is the densest gameplay step. It renders install target, placeholder markers, grabbable replacement parts, grabbable broken parts to store, placement feedback and a ready-to-install prompt. Validation requires the correct replacement part placed AND every scanned broken part deposited.
|
||||
|
||||
It renders:
|
||||
|
||||
- install target
|
||||
- placeholder markers
|
||||
- grabbable replacement parts
|
||||
- grabbable broken parts to store
|
||||
- placement feedback
|
||||
- ready-to-install prompt
|
||||
|
||||
Important local state:
|
||||
|
||||
- `placedPartIds`: replacement parts that snapped near a placeholder
|
||||
- `depositedBrokenPartIds`: broken parts stored in the case
|
||||
- `showBlockedInstallFeedback`: temporary visual feedback when install is attempted too early
|
||||
|
||||
Validation:
|
||||
For ebike (mission 1, simplified):
|
||||
|
||||
```txt
|
||||
correct replacement part placed
|
||||
AND every scanned broken part deposited
|
||||
src/components/three/gameplay/RepairEbikeRepairTrigger.tsx
|
||||
```
|
||||
|
||||
Only then does the install target call `onRepair()` and move to `reassembling`.
|
||||
Replaces the heavier grabbable UX with a single "Changez le refroidisseur" prompt. Pressing E advances directly to `reassembling`. The cercles décoratifs and grabbable parts are omitted to keep the first repair experience low-friction.
|
||||
|
||||
### Reassembling
|
||||
|
||||
@@ -238,23 +222,59 @@ File:
|
||||
src/components/three/gameplay/RepairReassemblyStep.tsx
|
||||
```
|
||||
|
||||
The exploded model animates back into assembled form and completion particles play. A timer then moves the mission to `done`.
|
||||
The shared `ExplodableModel` flips `split=false`, animating each node back to its original position (inverse of fragmented). `RepairReassemblyStep` itself is now reduced to:
|
||||
|
||||
Mission configs can override the default reassembly duration.
|
||||
- the completion particles
|
||||
- a `delayMs` timer (`REPAIR_REASSEMBLY_HOLD_MS = 1500`) that fires `onSettled` so `RepairGame` auto-advances to `done`
|
||||
|
||||
### Done
|
||||
|
||||
File:
|
||||
For pylon/farm:
|
||||
|
||||
```txt
|
||||
src/components/three/gameplay/RepairCompletionStep.tsx
|
||||
```
|
||||
|
||||
The repaired object remains visible. The player validates the completion target, then:
|
||||
The shared exploded model (now reassembled) remains visible. The player validates a green completion target, the case closes and exits, then `completeMission(mission)` advances the global game progression.
|
||||
|
||||
1. the repair case closes
|
||||
2. the case plays its exit animation
|
||||
3. `completeMission(mission)` advances the global game progression
|
||||
For ebike (mission 1, auto-complete):
|
||||
|
||||
`RepairGame` plays `narrateur_ebikerepare` directly on entry to `done`. When the audio's `ended` event fires (with `REPAIR_DONE_DIALOGUE_FALLBACK_MS = 6000` fallback) `completeMission("ebike")` is called automatically and the world hands off to the pylon mission. The bubble shrinks via `shouldFocusBubbleBeActive(done) === false`. No Validate button is shown.
|
||||
|
||||
## Focus Bubble
|
||||
|
||||
While the player is in `fragmented`, `scanning`, `repairing` or `reassembling`, `RepairGame` flips `useRepairFocusStore.active = true` and publishes the snapped world center of the repair model.
|
||||
|
||||
`RepairFocusBubble` reads the store and:
|
||||
|
||||
- renders a `BackSide` sphere (radius 1, scaled 0 → 10m) tinted `#060814` at opacity 0.92
|
||||
- grows the sphere with GSAP `expo.out` over 2.5 s when focus turns on
|
||||
- shrinks back with `expo.in` over 1.2 s when focus turns off
|
||||
- mounts a small "cocoon" decor pass inside (subtle grid floor + soft directional light + ambient) that fades in once the bubble is mostly grown
|
||||
|
||||
`Environment.tsx` and `GameStageContent.tsx` consume the same store flag to unmount the vegetation system and the zone debug visuals while the bubble is up, so trees and gizmos do not pierce the shroud. Terrain, water, sky, clouds and grass remain visible behind the bubble.
|
||||
|
||||
The bubble is mounted both in `GameStageContent` (production scene) and `TestMap` (physics test scene) so the behaviour matches in both contexts.
|
||||
|
||||
## Narrator Audio (Ebike Mission)
|
||||
|
||||
`EbikeRepairNarrator` (`src/components/game/EbikeRepairNarrator.tsx`) is a headless component mounted in `src/pages/page.tsx` next to `EbikeIntroSequence`. It subscribes to `useGameStore` and plays one-shot narrator cues at specific repair-step transitions for the `ebike` mission only:
|
||||
|
||||
| Step entered | Dialogue ID | Audio file | Subtitle | Owner |
|
||||
| ------------ | ------------------------------------ | ---------------------------------- | -------- | ---------------------- |
|
||||
| `fragmented` | `narrateur_galetscan` | `narrateur_galetscan.mp3` | cue 6 | `EbikeRepairNarrator` |
|
||||
| `scanning` | `narrateur_refroidisseur_diagnostic` | `narrateur_refroidisseurcassé.mp3` | cue 24 | `RepairScanSequence`\* |
|
||||
| `done` | `narrateur_ebikerepare` | `narrateur_ebikeréparé.mp3` | cue 7 | `RepairGame`\*\* |
|
||||
|
||||
\* The diagnostic line is triggered by the scan sequence when it lands on the broken part configured with `voiceLineId` (refroidisseur for ebike). The advance to `repairing` is gated on the audio's `ended` event so the line plays in full with the red highlight on screen.
|
||||
|
||||
\*\* `RepairGame` plays the success line directly on entering `done` so the audio's `ended` event can drive `completeMission` and hand off to pylon. A `REPAIR_DONE_DIALOGUE_FALLBACK_MS` timer guards against load failures. `EbikeRepairNarrator` no longer owns this cue.
|
||||
|
||||
A `useRef<Set<MissionStep>>` guards against double-fires (StrictMode, re-renders) and is cleared when the mission rolls back to `locked` or `waiting`, so debug-panel replays still trigger the narration.
|
||||
|
||||
Cue 7 was previously a single subtitle covering both the diagnostic line and the "Eeeet voilà!" completion line. It was split into cue 7 (completion only) and a new cue 24 (diagnostic) so the two sentences can be triggered at independent moments — they correspond to two distinct `.mp3` files.
|
||||
|
||||
The breakdown line (`narrateur_ebikecasse`, cue 5) is still triggered by `EbikeIntroSequence` at distance threshold, not by this component. Pylon and farm narrator cues are not yet wired through `EbikeRepairNarrator`; the same per-mission lookup pattern can be extended when those flows need narration.
|
||||
|
||||
## Repair Case Details
|
||||
|
||||
|
||||
+6
-12
@@ -2,24 +2,18 @@
|
||||
"version": 1,
|
||||
"cinematics": [
|
||||
{
|
||||
"id": "intro_overview",
|
||||
"id": "outro_farm_drone",
|
||||
"timecode": 0,
|
||||
"dialogueCues": [
|
||||
{
|
||||
"time": 0,
|
||||
"dialogueId": "narrateur_bienvenueaaltera"
|
||||
}
|
||||
],
|
||||
"cameraKeyframes": [
|
||||
{
|
||||
"time": 0,
|
||||
"position": [8, 5, 12],
|
||||
"target": [0, 2, 0]
|
||||
"position": [-24, 5, 65],
|
||||
"target": [-24, 2, 42]
|
||||
},
|
||||
{
|
||||
"time": 4,
|
||||
"position": [12, 4, -6],
|
||||
"target": [10, 1.4, -8]
|
||||
"time": 10,
|
||||
"position": [-24, 90, 200],
|
||||
"target": [-24, 0, 42]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
BIN
Binary file not shown.
+3
-3
@@ -39340,8 +39340,7 @@
|
||||
"rotation": [0, 0.0027, 0.0819],
|
||||
"scale": [1, 1, 1]
|
||||
}
|
||||
],
|
||||
"id": "repair:pylon"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "pylone",
|
||||
@@ -39373,7 +39372,8 @@
|
||||
"rotation": [0, 0.0027, 0.0819],
|
||||
"scale": [1, 1, 1]
|
||||
}
|
||||
]
|
||||
],
|
||||
"id": "repair:pylon"
|
||||
},
|
||||
{
|
||||
"name": "pylone",
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user