Compare commits
16 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 7f37f9a747 | |||
| 386abf06b6 | |||
| a73f9fb951 | |||
| d29b01e398 | |||
| 6edc5f7972 | |||
| ae35eb1dfb | |||
| 4de86f4e58 | |||
| 5b123f9704 | |||
| d1bf438465 | |||
| d2ce990165 | |||
| 7d2a257e84 | |||
| 58eb60292f | |||
| 73c6d7d50d | |||
| d9cacdad12 | |||
| ab88ab722f | |||
| 193fc8b4b6 |
@@ -10,17 +10,23 @@ It is now also available to the production repair flow when a mission reaches a
|
|||||||
|
|
||||||
## Runtime Flow
|
## Runtime Flow
|
||||||
|
|
||||||
1. The browser captures webcam frames in `src/hooks/handTracking/useRemoteHandTracking.ts`.
|
The frontend can run hand tracking with two interchangeable sources, selected from the debug source controller:
|
||||||
2. Frames are sent to the local Python backend over WebSocket.
|
|
||||||
3. The backend runs MediaPipe hand landmark detection.
|
- **Browser JS** (`src/hooks/handTracking/useBrowserHandTracking.ts`) runs MediaPipe `hand_landmarker.task` directly in the browser via `@mediapipe/tasks-vision`. Default for debug.
|
||||||
4. The backend returns hand data including landmarks, handedness, score, center point, and `isFist`.
|
- **Backend** (`src/hooks/handTracking/useRemoteHandTracking.ts`) sends webcam frames as JPEG over WebSocket to a local Python process that runs MediaPipe and returns landmarks.
|
||||||
5. React stores the latest snapshot in the hand tracking provider.
|
|
||||||
6. `GrabbableObject` reads that snapshot each frame and uses fist state plus raycasting to grab objects.
|
Both sources funnel into the same `HandTrackingContext` so all consumers see one shared snapshot:
|
||||||
7. `HandTrackingGlove` reads the same snapshot and places the rigged `gant_l` and `gant_r` models on the detected hands when hand tracking is active.
|
|
||||||
|
1. The active source captures or receives landmarks.
|
||||||
|
2. The hook applies an EMA smoothing pass on the landmarks before publishing the snapshot.
|
||||||
|
3. `HandTrackingProvider` exposes that snapshot through React context.
|
||||||
|
4. `GrabbableObject` reads the snapshot each frame and uses the fist state plus raycasting to grab objects.
|
||||||
|
5. `HandTrackingGlove` reads the same snapshot and places a rigged glove on each detected hand.
|
||||||
|
6. `HandTrackingVisualizer` paints an SVG wireframe overlay on top of the canvas.
|
||||||
|
|
||||||
## Activation Rules
|
## Activation Rules
|
||||||
|
|
||||||
Hand tracking is intentionally gated so the webcam and backend are not used all the time.
|
Hand tracking is gated so the webcam and runtime are only spun up when actually needed.
|
||||||
|
|
||||||
The debug activation conditions are:
|
The debug activation conditions are:
|
||||||
|
|
||||||
@@ -28,16 +34,26 @@ The debug activation conditions are:
|
|||||||
- scene mode is `physics`
|
- scene mode is `physics`
|
||||||
- the player is near an interaction, is holding an object, or is hand-holding an object
|
- the player is near an interaction, is holding an object, or is hand-holding an object
|
||||||
|
|
||||||
This keeps hand tracking active while the player is inside an interaction zone, even if the camera is not aimed directly at the object.
|
|
||||||
|
|
||||||
The production repair activation conditions are:
|
The production repair activation conditions are:
|
||||||
|
|
||||||
- active `mainState` is `ebike`, `pylon`, or `farm`
|
- active `mainState` is `ebike`, `pylon`, or `farm`
|
||||||
- the active mission step is `inspected`, `repairing`, `reassembling`, or `done`
|
- the active mission step is `inspected`, `repairing`, `reassembling`, or `done`
|
||||||
|
|
||||||
This keeps the webcam off during `waiting`, `fragmented`, and `scanning`, then enables hand input only when the repair flow is expected to use hands.
|
This keeps the webcam off during `waiting`, `fragmented`, and `scanning`.
|
||||||
|
|
||||||
In the current production repair flow, `inspected` uses a two-fists hold gesture to advance to `fragmented`. The hold must last one second and is independent from local object interaction distance once the mission is in the correct state. Keyboard input for the same transition is handled separately by the repair case trigger, so pressing `E` requires the case to be focused through the shared interaction system.
|
### Linger
|
||||||
|
|
||||||
|
Once activation turns off (player walks back out of a trigger zone, or a mission step transitions away), the runtime stays alive for `HAND_TRACKING_LINGER_MS` (2000 ms) before being torn down. This gives MediaPipe enough time to finish initializing the webcam and load the model on a fresh entry — without the linger, a quick walk-through of a trigger zone never produces a detected hand.
|
||||||
|
|
||||||
|
## Provider Stability
|
||||||
|
|
||||||
|
`HandTrackingProvider` always renders the same JSX root (`HandTrackingRuntime`) and exposes `enabled` as a prop. Returning two different element types (`<HandTrackingContext value=IDLE>` vs `<ActiveHandTrackingProvider>`) used to be the historical shape and was the root cause of WebGL context loss: every `enabled` toggle forced React to remount the entire subtree, including the `<Canvas>`, which destroyed the WebGL renderer.
|
||||||
|
|
||||||
|
The two source hooks are therefore mounted in permanence with an `enabled` flag that they early-return on. No webcam or MediaPipe resources are created while `enabled` is false.
|
||||||
|
|
||||||
|
## StrictMode Resilience
|
||||||
|
|
||||||
|
In development, `<StrictMode>` mounts → unmounts → remounts each effect to surface non-idempotent code. The two source hooks delay their actual `start()` call by `HAND_TRACKING_RUNTIME_START_DELAY_MS` (80 ms) and clear the timer on cleanup, so a StrictMode double-mount or a rapid `nearby` flicker never reaches `getUserMedia` twice.
|
||||||
|
|
||||||
## Backend
|
## Backend
|
||||||
|
|
||||||
@@ -52,7 +68,27 @@ The Python process uses MediaPipe and the local model file:
|
|||||||
backend/hand_landmarker.task
|
backend/hand_landmarker.task
|
||||||
```
|
```
|
||||||
|
|
||||||
The backend sends normalized hand coordinates and landmarks. The frontend treats the values as screen-space inputs, then maps them into world space with the active Three.js camera.
|
The frontend sends JPEG frames at `HAND_TRACKING_FRAME_WIDTH × HAND_TRACKING_FRAME_HEIGHT` (320×240) to keep WebSocket bandwidth low. The backend sends normalized hand coordinates and landmarks.
|
||||||
|
|
||||||
|
## Browser MediaPipe
|
||||||
|
|
||||||
|
The browser path uses `hand_landmarker.task` (float16) downloaded from Google's MediaPipe model storage. The requested webcam resolution is **640×480** (`HAND_TRACKING_BROWSER_CAMERA_WIDTH/HEIGHT`), independent from the backend's 320×240. The float16 model is more sensitive than the backend Python model and needs the higher-resolution frame to detect hands reliably.
|
||||||
|
|
||||||
|
The MediaPipe delegate is currently `"GPU"`. CPU works too but is significantly slower; on a loaded scene the inference drops to ~5fps and the user feels noticeable lag during grab. MediaPipe creates its own WebGL context separate from Three.js, so there is no direct contention.
|
||||||
|
|
||||||
|
A singleton instance of `HandLandmarker` is cached in `src/lib/handTracking/browserHandTracking.ts`. `releaseBrowserHandLandmarker()` is called on cleanup and on WebGL context lost.
|
||||||
|
|
||||||
|
## Smoothing
|
||||||
|
|
||||||
|
MediaPipe at ~10 fps produces noticeable landmark jitter that, when fed raw into the scene, makes both the glove rig and any grabbed object tremble.
|
||||||
|
|
||||||
|
A simple exponential moving average is applied to every landmark before the snapshot is published:
|
||||||
|
|
||||||
|
```ts
|
||||||
|
smoothed.x = previous.x * (1 - factor) + next.x * factor;
|
||||||
|
```
|
||||||
|
|
||||||
|
The factor is `HAND_TRACKING_LANDMARK_SMOOTHING` (0.4). Hands are matched across frames by `handedness` so left/right don't bleed into each other.
|
||||||
|
|
||||||
## Frontend Data Shape
|
## Frontend Data Shape
|
||||||
|
|
||||||
@@ -106,24 +142,36 @@ This is less expressive than true depth-aware hand movement, but it is more stab
|
|||||||
The current debug UI includes:
|
The current debug UI includes:
|
||||||
|
|
||||||
- `HandTrackingDebugPanel` inside `DebugOverlayLayout` for status, usage, loaded glove model, server state, hand count, and fist state
|
- `HandTrackingDebugPanel` inside `DebugOverlayLayout` for status, usage, loaded glove model, server state, hand count, and fist state
|
||||||
- `HandTrackingVisualizer` for the SVG landmark wireframe fallback
|
- `HandTrackingVisualizer` for the SVG landmark overlay
|
||||||
- `HandTrackingGlove` for the left-hand `gant_l` and right-hand `gant_r` models in the R3F scene
|
- `HandTrackingFallback` for the last-resort hand silhouette overlay
|
||||||
|
- `HandTrackingGlove` for the per-hand rigged glove models in the R3F scene
|
||||||
- `r3f-perf` for render performance
|
- `r3f-perf` for render performance
|
||||||
- `lil-gui` for scene, camera, lighting, interaction, and grab controls
|
- `lil-gui` for scene, camera, lighting, interaction, and grab controls
|
||||||
|
|
||||||
The hand tracking debug panel is a compact HTML grid outside the canvas. `Model loaded` displays the successfully loaded glove models. The SVG hand wireframe is only a fallback while models are loading or if a glove model fails to load.
|
The SVG visualizer uses a "blueish hand" style: white connection lines between landmarks, cyan circles with a dark blue outline. The outline gets thicker when the hand is detected as a fist, so the user gets a visual confirmation of the grab gesture without having to look at the debug panel.
|
||||||
|
|
||||||
|
The fallback overlay (`HandTrackingFallback`) draws a simple open-hand or fist silhouette positioned on the detected wrist landmark. It only renders for a hand whose matching glove is in the `"error"` state in `useHandTrackingGloveStatus`. This guarantees the user always sees something on their hand even when the 3D glove model fails to load.
|
||||||
|
|
||||||
## Glove Models
|
## Glove Models
|
||||||
|
|
||||||
The current glove MVP uses `public/models/gant_l/model.gltf` and `public/models/gant_r/model.gltf`, which contain GLTF skins and armatures. Each model is positioned, oriented, and scaled from palm landmarks, then each finger bone chain is rotated toward the matching MediaPipe landmark chain.
|
`HandTrackingGlove` loads `public/models/gant_l/model.gltf` for both hands. The right hand applies `scale.x = -1` at the group level to mirror the mesh, so the thumb ends up on the correct side. Both hands therefore share the same rig and the same material.
|
||||||
|
|
||||||
The glove models are intentionally smaller than the raw SVG overlay so they do not dominate the camera view.
|
The historical `public/models/gant_r/model.gltf` is kept as legacy but is not loaded by the frontend — its GLB embeds three skeletons (`Hand_l`, `Hand_l_pad`, `Hand_r`) plus a `galet` mesh, which made the finger rig unreliable.
|
||||||
|
|
||||||
|
The `gant_l` material is set to `alphaMode: OPAQUE` with `doubleSided: true`. The opaque mode prevents transparency sorting issues that made folded fingers disappear behind the palm; the double-sided flag covers the back faces revealed by the mirror scale on the right hand.
|
||||||
|
|
||||||
|
Two additional glove variants exist on disk:
|
||||||
|
|
||||||
|
- `public/models/gant_l_pad/model.gltf`
|
||||||
|
- `public/models/gant_r_pad/model.gltf`
|
||||||
|
|
||||||
|
They are intended for future swap-by-state usage but are **not yet rigged**. They cannot be animated by MediaPipe landmarks in their current form — re-exporting them from Blender with the same armature structure as `gant_l` is a prerequisite.
|
||||||
|
|
||||||
## Known Limitations
|
## Known Limitations
|
||||||
|
|
||||||
- Production usage is currently limited to repair mission steps that explicitly need hands.
|
- Production usage is currently limited to repair mission steps that explicitly need hands.
|
||||||
- MediaPipe depth is relative and currently not used for stable object depth control.
|
- MediaPipe depth is relative and currently not used for stable object depth control.
|
||||||
- The virtual hit zone is an approximation based on multiple raycasts, not a real 3D collider.
|
- The virtual hit zone is an approximation based on multiple raycasts, not a real 3D collider.
|
||||||
- There is no smoothing layer for hand position or depth yet.
|
- The right glove is a mirrored copy of `gant_l` rather than its own mesh; in the future a dedicated right-hand model would give a better visual.
|
||||||
- The SVG hand visualization is a fallback, not the primary display when glove models load correctly.
|
- The `_pad` glove variants are not rigged yet, so swap-by-state (normal ↔ pad) is not wired in.
|
||||||
- Finger bone animation is an approximate landmark-to-bone mapping; it still needs calibration for per-model twist, offsets, and smoothing.
|
- Finger bone animation is an approximate landmark-to-bone mapping; it still needs calibration for per-model twist, offsets, and smoothing.
|
||||||
|
|||||||
@@ -1,367 +0,0 @@
|
|||||||
# WebGL Context Lost - Investigation
|
|
||||||
|
|
||||||
## Résumé court
|
|
||||||
|
|
||||||
Le projet subit des pertes de contexte WebGL pendant les phases où le jeu active
|
|
||||||
ou prépare le hand tracking, les interactions physiques ou le repair game.
|
|
||||||
|
|
||||||
Le symptôme visible côté console est :
|
|
||||||
|
|
||||||
```txt
|
|
||||||
THREE.WebGLRenderer: Context Lost.
|
|
||||||
[ERROR] [WebGL] Context lost - attempting auto-restore
|
|
||||||
THREE.WebGLRenderer: Context Restored.
|
|
||||||
```
|
|
||||||
|
|
||||||
Le problème est bloquant parce que le hand tracking et le repair game sont au
|
|
||||||
coeur de l'expérience. Quand le contexte WebGL saute, la scène Three.js peut se
|
|
||||||
remonter, le joueur peut revenir au spawn, le pointer lock peut être perdu, et
|
|
||||||
les tests de gameplay deviennent instables.
|
|
||||||
|
|
||||||
## Ce qui fonctionne aujourd'hui
|
|
||||||
|
|
||||||
La page principale monte un `<Canvas>` React Three Fiber dans
|
|
||||||
`src/pages/page.tsx`.
|
|
||||||
|
|
||||||
`src/world/World.tsx` compose ensuite :
|
|
||||||
|
|
||||||
- la scène de jeu ou la scène de test physique ;
|
|
||||||
- le player ;
|
|
||||||
- les systèmes visuels de monde ;
|
|
||||||
- les gants de hand tracking ;
|
|
||||||
- les systèmes de debug.
|
|
||||||
|
|
||||||
Le hand tracking est centralisé dans
|
|
||||||
`src/providers/gameplay/HandTrackingProvider.tsx`.
|
|
||||||
|
|
||||||
Il peut utiliser deux sources :
|
|
||||||
|
|
||||||
- `browser` : MediaPipe JS dans le navigateur ;
|
|
||||||
- `backend` : backend Python local via WebSocket.
|
|
||||||
|
|
||||||
L'activation est déclenchée par :
|
|
||||||
|
|
||||||
- certaines étapes du repair game ;
|
|
||||||
- les zones d'interaction qui demandent explicitement les mains ;
|
|
||||||
- la scène Physique en debug, selon les objets présents.
|
|
||||||
|
|
||||||
## Problème observé
|
|
||||||
|
|
||||||
Les context lost arrivent dans plusieurs situations :
|
|
||||||
|
|
||||||
- entrée dans une zone d'interaction ;
|
|
||||||
- lancement du hand tracking ;
|
|
||||||
- lancement d'un repair game ;
|
|
||||||
- scène Physique avec `TestMap`, `Physics`, `AnimatedModel`, waypoints GPS et
|
|
||||||
objets interactifs ;
|
|
||||||
- source browser JS ;
|
|
||||||
- source backend.
|
|
||||||
|
|
||||||
Le fait que le crash existe avec les deux sources indique que le problème n'est
|
|
||||||
probablement pas limité au backend Python ni à MediaPipe JS seul. Le hand
|
|
||||||
tracking semble être un déclencheur fort, mais il arrive au moment où plusieurs
|
|
||||||
ressources GPU et systèmes runtime se réveillent ensemble.
|
|
||||||
|
|
||||||
## Pourquoi c'est bloquant
|
|
||||||
|
|
||||||
Ce bug bloque la feature principale du projet :
|
|
||||||
|
|
||||||
- le repair game dépend du hand tracking pour valider certaines actions ;
|
|
||||||
- les interactions main sont nécessaires pour tester les objets grabbables ;
|
|
||||||
- un context lost casse la continuité du gameplay ;
|
|
||||||
- le joueur peut être replacé au spawn après reconstruction ;
|
|
||||||
- le pointer lock peut être perdu ;
|
|
||||||
- les logs deviennent difficiles à lire parce que le jeu tente de restaurer la
|
|
||||||
scène en boucle ;
|
|
||||||
- le comportement n'est pas fiable pour une démo ou un déploiement.
|
|
||||||
|
|
||||||
Tant que ce problème n'est pas stable, on ne peut pas valider correctement :
|
|
||||||
|
|
||||||
- la mission e-bike ;
|
|
||||||
- la mission pylône ;
|
|
||||||
- la mission ferme ;
|
|
||||||
- les interactions main ;
|
|
||||||
- le switch browser/backend ;
|
|
||||||
- le comportement en build de production.
|
|
||||||
|
|
||||||
## Hypothèses principales
|
|
||||||
|
|
||||||
### 1. Pression GPU au lancement du hand tracking
|
|
||||||
|
|
||||||
MediaPipe browser peut créer ses propres ressources GPU. Si Three.js charge
|
|
||||||
déjà beaucoup de géométries, textures, ombres et modèles, l'ajout du hand
|
|
||||||
tracking peut faire passer le navigateur au-dessus d'une limite GPU.
|
|
||||||
|
|
||||||
Le stash contient une tentative de mitigation en forçant MediaPipe browser et le
|
|
||||||
backend à utiliser le CPU.
|
|
||||||
|
|
||||||
### 2. Activation trop brusque du runtime mains
|
|
||||||
|
|
||||||
Les logs montrent des transitions rapides :
|
|
||||||
|
|
||||||
```txt
|
|
||||||
Browser JS runtime starting
|
|
||||||
Runtime source selected
|
|
||||||
Runtime snapshot changed
|
|
||||||
Browser JS runtime stopped
|
|
||||||
Browser JS runtime starting
|
|
||||||
```
|
|
||||||
|
|
||||||
Ce type de start/stop rapide peut provoquer :
|
|
||||||
|
|
||||||
- création webcam ;
|
|
||||||
- création MediaPipe ;
|
|
||||||
- montage des gants ;
|
|
||||||
- update du state React ;
|
|
||||||
- re-render du monde ;
|
|
||||||
- stress GPU au même moment.
|
|
||||||
|
|
||||||
### 3. Les gants 3D sont montés trop tôt
|
|
||||||
|
|
||||||
Si les gants de hand tracking sont montés avant d'avoir de vraies mains
|
|
||||||
détectées, le jeu charge et prépare des modèles GPU sans utilité immédiate.
|
|
||||||
|
|
||||||
Le stash contient une tentative pour ne rendre les gants que lorsqu'une main
|
|
||||||
existe réellement dans le snapshot.
|
|
||||||
|
|
||||||
### 4. Re-upload textures / GLTF trop agressif
|
|
||||||
|
|
||||||
`src/utils/three/optimizeGLTFScene.ts` modifie des textures GLTF. Si cette
|
|
||||||
optimisation force trop souvent `needsUpdate`, mipmaps ou anisotropy, le
|
|
||||||
navigateur peut recharger beaucoup de textures vers le GPU.
|
|
||||||
|
|
||||||
Le stash limite cette pression en évitant de forcer les mipmaps et en abaissant
|
|
||||||
l'anisotropy.
|
|
||||||
|
|
||||||
### 5. Permission caméra au mauvais moment
|
|
||||||
|
|
||||||
Demander la caméra au moment exact où le joueur entre dans une interaction ou
|
|
||||||
lance le repair game ajoute un gros événement runtime au pire moment.
|
|
||||||
|
|
||||||
Le stash contient une tentative de warmup caméra pour obtenir la permission plus
|
|
||||||
tôt et réutiliser le stream au moment où le hand tracking devient nécessaire.
|
|
||||||
|
|
||||||
### 6. La scène Physique ajoute du bruit
|
|
||||||
|
|
||||||
La scène Physique est une scène de test volontairement riche :
|
|
||||||
|
|
||||||
- `Physics` Rapier ;
|
|
||||||
- `GrabbableObject` ;
|
|
||||||
- `TriggerObject` ;
|
|
||||||
- `RepairGame` ;
|
|
||||||
- `AnimatedModel` ;
|
|
||||||
- GPS preview ;
|
|
||||||
- waypoints verts ;
|
|
||||||
- player ;
|
|
||||||
- debug overlay.
|
|
||||||
|
|
||||||
Cette richesse est normale pour une scène de test, mais elle complique
|
|
||||||
l'investigation parce qu'elle active beaucoup de systèmes à la fois.
|
|
||||||
|
|
||||||
## Fichiers modifiés dans le stash
|
|
||||||
|
|
||||||
Le stash `stash@{0}` contient 28 fichiers modifiés, environ `+530 / -152`.
|
|
||||||
Il ne contient pas de fichiers untracked.
|
|
||||||
|
|
||||||
| Fichier | Rôle dans l'investigation |
|
|
||||||
| --------------------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
|
||||||
| `README.md` | Note sur les commandes backend depuis la racine du repo. |
|
|
||||||
| `backend/README.md` | Documentation plus claire pour lancer le backend et réparer un `.venv` cassé. |
|
|
||||||
| `backend/hand_tracker.py` | Force le backend MediaPipe en CPU. |
|
|
||||||
| `docs/user/main-feature.md` | Ajustements de documentation utilisateur. |
|
|
||||||
| `public/sounds/dialogue/subtitles/fr/electricienne.srt` | Ajustements de sous-titres, pas central pour le context lost. |
|
|
||||||
| `public/sounds/dialogue/subtitles/fr/narrateur.srt` | Ajustements de sous-titres, pas central pour le context lost. |
|
|
||||||
| `src/components/debug/DebugPlayerModel.tsx` | Ajustements de modèle debug player. |
|
|
||||||
| `src/components/three/handTracking/HandTrackingGlove.tsx` | Retire le preload automatique des gants pour réduire la pression GPU. |
|
|
||||||
| `src/components/three/interaction/GrabbableObject.tsx` | Marque les grabbables qui nécessitent vraiment le hand tracking. |
|
|
||||||
| `src/components/three/interaction/InteractableObject.tsx` | Ajoute le flag `handTracking` aux interactables. |
|
|
||||||
| `src/data/debug/testSceneConfig.ts` | Stabilise la scène Physique : sol, GPS, hauteur des waypoints. |
|
|
||||||
| `src/data/handTrackingConfig.ts` | Ajoute délai d'activation, TTL warmup caméra, delegate CPU browser. |
|
|
||||||
| `src/data/player/playerConfig.ts` | Corrige le spawn Physique avec `PLAYER_EYE_HEIGHT`. |
|
|
||||||
| `src/hooks/debug/useSceneMode.ts` | Force `game` hors debug actif pour éviter des scènes debug en prod. |
|
|
||||||
| `src/hooks/handTracking/useBothFistsHold.ts` | Sort le hold des deux poings de `useFrame` R3F vers `requestAnimationFrame`. |
|
|
||||||
| `src/hooks/handTracking/useBrowserHandTracking.ts` | Encadre `detectForVideo`, release MediaPipe en cleanup, gère les erreurs. |
|
|
||||||
| `src/hooks/three/useTerrainHeight.ts` | Ajustements terrain, liés au snap/player. |
|
|
||||||
| `src/lib/handTracking/browserHandTracking.ts` | Force delegate CPU, garde une instance MediaPipe, ajoute `releaseBrowserHandLandmarker`. |
|
|
||||||
| `src/lib/handTracking/handTrackingSession.ts` | Ajoute warmup caméra, cache stream, timeout et consommation du stream préparé. |
|
|
||||||
| `src/managers/InteractionManager.ts` | Ajoute `handTrackingNearby` pour ne pas activer les mains sur toute interaction. |
|
|
||||||
| `src/pages/page.tsx` | Gestion WebGL context lost/restored, DPR fixe, antialias off, release MediaPipe au crash. |
|
|
||||||
| `src/providers/gameplay/HandTrackingProvider.tsx` | Ajoute activation différée, snapshot queued, warmup runtime. |
|
|
||||||
| `src/types/interaction/interaction.ts` | Ajoute `handTracking` et `handTrackingNearby` aux types interaction. |
|
|
||||||
| `src/utils/debug/Debug.ts` | Synchronise l'affichage du controller hand tracking source. |
|
|
||||||
| `src/utils/three/optimizeGLTFScene.ts` | Réduit la pression GPU des textures GLTF. |
|
|
||||||
| `src/world/World.tsx` | Ne rend les gants que si une main correspondante est détectée. |
|
|
||||||
| `src/world/debug/TestMap.tsx` | Nettoie les logs, stabilise waypoints/GPS/scène Physique. |
|
|
||||||
| `src/world/player/PlayerCamera.tsx` | Ajustements pointer lock/canvas ciblé. |
|
|
||||||
|
|
||||||
## Fichiers actuellement modifiés dans le worktree
|
|
||||||
|
|
||||||
Etat observé au moment de cette note :
|
|
||||||
|
|
||||||
| Fichier | Statut |
|
|
||||||
| --------------------------------------------------------- | --------------------------------------------------------- |
|
|
||||||
| `public/models/talkie/*` | Beaucoup d'anciennes textures/fichiers `.gltf` supprimés. |
|
|
||||||
| `public/models/talkie/model.glb` | Nouveau fichier non suivi. |
|
|
||||||
| `src/components/three/handTracking/HandTrackingGlove.tsx` | Modifié. |
|
|
||||||
| `src/data/debug/testSceneConfig.ts` | Modifié. |
|
|
||||||
| `src/data/gameplay/repairMissions.ts` | Modifié. |
|
|
||||||
| `src/data/handTrackingConfig.ts` | Modifié. |
|
|
||||||
| `src/data/player/playerConfig.ts` | Modifié. |
|
|
||||||
| `src/data/world/mapLodConfig.ts` | Modifié. |
|
|
||||||
| `src/hooks/handTracking/useBrowserHandTracking.ts` | Modifié. |
|
|
||||||
| `src/hooks/handTracking/useRemoteHandTracking.ts` | Modifié. |
|
|
||||||
| `src/lib/handTracking/browserHandTracking.ts` | Modifié. |
|
|
||||||
| `src/lib/handTracking/handTrackingSession.ts` | Modifié. |
|
|
||||||
| `src/pages/page.tsx` | Modifié. |
|
|
||||||
| `src/providers/gameplay/HandTrackingProvider.tsx` | Modifié. |
|
|
||||||
| `src/utils/debug/Debug.ts` | Modifié. |
|
|
||||||
| `src/utils/three/optimizeGLTFScene.ts` | Modifié. |
|
|
||||||
| `src/world/World.tsx` | Modifié. |
|
|
||||||
| `src/world/debug/TestMap.tsx` | Modifié. |
|
|
||||||
| `src/world/player/Player.tsx` | Modifié. |
|
|
||||||
| `src/world/player/PlayerCamera.tsx` | Modifié. |
|
|
||||||
| `src/world/player/PlayerController.tsx` | Modifié. |
|
|
||||||
| `src/components/ui/RuntimeLoadingIndicator.tsx` | Nouveau fichier non suivi. |
|
|
||||||
| `src/hooks/handTracking/useHandTrackingRuntimeWarmup.ts` | Nouveau fichier non suivi. |
|
|
||||||
| `src/world/player/playerRuntimeSnapshot.ts` | Nouveau fichier non suivi. |
|
|
||||||
|
|
||||||
Attention : les fichiers supprimés/nouveaux du talkie semblent être un sujet
|
|
||||||
séparé du context lost. Il faut les garder séparés dans les commits.
|
|
||||||
|
|
||||||
## Fichiers directement impactés par le bug
|
|
||||||
|
|
||||||
### Canvas et WebGL
|
|
||||||
|
|
||||||
- `src/pages/page.tsx`
|
|
||||||
- `src/world/World.tsx`
|
|
||||||
- `src/utils/three/optimizeGLTFScene.ts`
|
|
||||||
|
|
||||||
Ces fichiers influencent directement la charge GPU, la configuration du canvas,
|
|
||||||
les ressources GLTF et le comportement au context lost/restored.
|
|
||||||
|
|
||||||
### Hand tracking
|
|
||||||
|
|
||||||
- `src/providers/gameplay/HandTrackingProvider.tsx`
|
|
||||||
- `src/hooks/handTracking/useBrowserHandTracking.ts`
|
|
||||||
- `src/hooks/handTracking/useRemoteHandTracking.ts`
|
|
||||||
- `src/hooks/handTracking/useBothFistsHold.ts`
|
|
||||||
- `src/hooks/handTracking/useHandTrackingRuntimeWarmup.ts`
|
|
||||||
- `src/lib/handTracking/browserHandTracking.ts`
|
|
||||||
- `src/lib/handTracking/handTrackingSession.ts`
|
|
||||||
- `src/data/handTrackingConfig.ts`
|
|
||||||
- `src/components/three/handTracking/HandTrackingGlove.tsx`
|
|
||||||
- `backend/hand_tracker.py`
|
|
||||||
|
|
||||||
Ces fichiers contrôlent le déclenchement, la source, la caméra, MediaPipe, le
|
|
||||||
backend et le rendu visuel des mains.
|
|
||||||
|
|
||||||
### Interactions et repair game
|
|
||||||
|
|
||||||
- `src/components/three/interaction/GrabbableObject.tsx`
|
|
||||||
- `src/components/three/interaction/InteractableObject.tsx`
|
|
||||||
- `src/managers/InteractionManager.ts`
|
|
||||||
- `src/types/interaction/interaction.ts`
|
|
||||||
- `src/components/three/gameplay/RepairGame.tsx`
|
|
||||||
- `src/hooks/gameplay/useRepairMissionStep.ts`
|
|
||||||
- `src/hooks/gameplay/useRepairMovementLocked.ts`
|
|
||||||
|
|
||||||
Ces fichiers sont impactés parce que l'entrée dans une zone ou une étape repair
|
|
||||||
peut déclencher le hand tracking.
|
|
||||||
|
|
||||||
### Player et restauration après crash
|
|
||||||
|
|
||||||
- `src/world/player/Player.tsx`
|
|
||||||
- `src/world/player/PlayerCamera.tsx`
|
|
||||||
- `src/world/player/PlayerController.tsx`
|
|
||||||
- `src/world/player/playerRuntimeSnapshot.ts`
|
|
||||||
- `src/data/player/playerConfig.ts`
|
|
||||||
|
|
||||||
Ces fichiers influencent le spawn, la caméra, le pointer lock, et la possibilité
|
|
||||||
de récupérer la dernière position après un context lost.
|
|
||||||
|
|
||||||
### Scène Physique / debug
|
|
||||||
|
|
||||||
- `src/world/debug/TestMap.tsx`
|
|
||||||
- `src/data/debug/testSceneConfig.ts`
|
|
||||||
- `src/components/debug/DebugPlayerModel.tsx`
|
|
||||||
- `src/hooks/debug/useSceneMode.ts`
|
|
||||||
- `src/utils/debug/Debug.ts`
|
|
||||||
|
|
||||||
Ces fichiers ne sont pas forcément la cause racine, mais ils créent une scène de
|
|
||||||
stress utile pour reproduire le bug.
|
|
||||||
|
|
||||||
## Ce que le stash essayait de corriger
|
|
||||||
|
|
||||||
Le stash essaye de réduire le risque de context lost avec plusieurs leviers :
|
|
||||||
|
|
||||||
1. passer MediaPipe browser/backend en CPU ;
|
|
||||||
2. libérer MediaPipe quand le runtime s'arrête ou quand WebGL saute ;
|
|
||||||
3. éviter de monter les gants sans mains détectées ;
|
|
||||||
4. retarder l'activation du hand tracking pour éviter les start/stop violents ;
|
|
||||||
5. demander la caméra plus tôt et réutiliser le stream ;
|
|
||||||
6. réduire la charge GPU du canvas avec DPR fixe et antialias off ;
|
|
||||||
7. limiter les re-uploads de textures GLTF ;
|
|
||||||
8. distinguer les interactions qui demandent vraiment le hand tracking ;
|
|
||||||
9. restaurer WebGL avec une limite pour éviter les boucles infinies ;
|
|
||||||
10. conserver la position du joueur après restauration.
|
|
||||||
|
|
||||||
## Ce qui reste à prouver
|
|
||||||
|
|
||||||
Il faut encore isoler le déclencheur exact :
|
|
||||||
|
|
||||||
- crash avec hand tracking désactivé complètement ;
|
|
||||||
- crash avec source browser JS seulement ;
|
|
||||||
- crash avec source backend seulement ;
|
|
||||||
- crash avec gants 3D désactivés ;
|
|
||||||
- crash avec MediaPipe CPU ;
|
|
||||||
- crash avec `AnimatedModel` de TestMap désactivé ;
|
|
||||||
- crash avec GPS preview/waypoints désactivés ;
|
|
||||||
- crash avec shadows/antialias/DPR réduits ;
|
|
||||||
- crash en scène game réelle, pas seulement scène Physique.
|
|
||||||
|
|
||||||
## Plan d'investigation recommandé
|
|
||||||
|
|
||||||
1. Stabiliser le worktree et ne pas mélanger assets talkie, LOD, docs backend et
|
|
||||||
context lost dans le même commit.
|
|
||||||
2. Garder le stash tant que le fix final n'est pas validé.
|
|
||||||
3. Créer un commit ou patch isolé pour les logs context lost seulement.
|
|
||||||
4. Ajouter un switch debug qui permet de couper séparément :
|
|
||||||
- hand tracking runtime ;
|
|
||||||
- gants 3D ;
|
|
||||||
- MediaPipe browser ;
|
|
||||||
- backend ;
|
|
||||||
- GPS preview ;
|
|
||||||
- AnimatedModel de TestMap.
|
|
||||||
5. Reproduire le bug avec une matrice claire.
|
|
||||||
6. Garder les changements qui diminuent réellement les context lost.
|
|
||||||
7. Supprimer les logs temporaires une fois le diagnostic terminé.
|
|
||||||
|
|
||||||
## Recommandation Git
|
|
||||||
|
|
||||||
Ne pas supprimer le stash maintenant.
|
|
||||||
|
|
||||||
Il contient du travail réel sur le context lost. Même s'il n'est pas parfait, il
|
|
||||||
sert de trace d'investigation et contient des morceaux utiles.
|
|
||||||
|
|
||||||
Avant de le supprimer, sauvegarder le patch :
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git stash show -p stash@{0} > context-lost-stash.patch
|
|
||||||
```
|
|
||||||
|
|
||||||
Ensuite seulement, si tout a été repris dans des commits propres :
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git stash drop stash@{0}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Commits logiques proposés
|
|
||||||
|
|
||||||
Séparer en plusieurs commits pour éviter un gros commit illisible :
|
|
||||||
|
|
||||||
1. `docs: document webgl context lost investigation`
|
|
||||||
2. `fix: reduce handtracking gpu pressure`
|
|
||||||
3. `fix: delay handtracking activation`
|
|
||||||
4. `fix: preserve player state after webgl restore`
|
|
||||||
5. `fix: stabilize physics debug scene`
|
|
||||||
6. `docs: clarify backend handtracking setup`
|
|
||||||
BIN
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+744
-189
File diff suppressed because it is too large
Load Diff
+170
-28
@@ -2,7 +2,7 @@ import { useEffect, useRef, useState, useMemo, useCallback } from "react";
|
|||||||
import * as THREE from "three";
|
import * as THREE from "three";
|
||||||
import { useFrame, useThree } from "@react-three/fiber";
|
import { useFrame, useThree } from "@react-three/fiber";
|
||||||
import { EbikeGPSMap } from "@/components/ebike/EbikeGPSMap";
|
import { EbikeGPSMap } from "@/components/ebike/EbikeGPSMap";
|
||||||
import { EbikeSpeedometer } from "@/components/ebike/EbikeSpeedometer";
|
import { EbikeSpeedmeter } from "@/components/ebike/EbikeSpeedmeter";
|
||||||
import { InteractableObject } from "@/components/three/interaction/InteractableObject";
|
import { InteractableObject } from "@/components/three/interaction/InteractableObject";
|
||||||
import { useLoggedGLTF } from "@/hooks/three/useLoggedGLTF";
|
import { useLoggedGLTF } from "@/hooks/three/useLoggedGLTF";
|
||||||
import { useClonedObject } from "@/hooks/three/useClonedObject";
|
import { useClonedObject } from "@/hooks/three/useClonedObject";
|
||||||
@@ -25,6 +25,12 @@ import "@/types/ebike/ebikeWindow";
|
|||||||
|
|
||||||
const EBIKE_MODEL_PATH = "/models/ebike/model.gltf";
|
const EBIKE_MODEL_PATH = "/models/ebike/model.gltf";
|
||||||
|
|
||||||
|
// Reusable vectors — allocated once to avoid per-frame GC pressure
|
||||||
|
const _phareWorldPos = new THREE.Vector3();
|
||||||
|
const _bikeForward = new THREE.Vector3();
|
||||||
|
const _aimDir = new THREE.Vector3();
|
||||||
|
const _up = new THREE.Vector3(0, 1, 0);
|
||||||
|
|
||||||
interface EbikeProps {
|
interface EbikeProps {
|
||||||
position: Vector3Tuple;
|
position: Vector3Tuple;
|
||||||
}
|
}
|
||||||
@@ -53,6 +59,7 @@ export function Ebike({ position }: EbikeProps): React.JSX.Element {
|
|||||||
const ebikeStep = useGameStore((state) => state.ebike.currentStep);
|
const ebikeStep = useGameStore((state) => state.ebike.currentStep);
|
||||||
const setMissionStep = useGameStore((state) => state.setMissionStep);
|
const setMissionStep = useGameStore((state) => state.setMissionStep);
|
||||||
const camera = useThree((state) => state.camera);
|
const camera = useThree((state) => state.camera);
|
||||||
|
const threeScene = useThree((state) => state.scene);
|
||||||
const updateEbikeSounds = useEbikeSounds();
|
const updateEbikeSounds = useEbikeSounds();
|
||||||
const repairGameOwnsEbikeModel =
|
const repairGameOwnsEbikeModel =
|
||||||
mainState === "ebike" &&
|
mainState === "ebike" &&
|
||||||
@@ -96,6 +103,19 @@ export function Ebike({ position }: EbikeProps): React.JSX.Element {
|
|||||||
]);
|
]);
|
||||||
const restingRotationRef = useRef<number>(EBIKE_WORLD_ROTATION_Y);
|
const restingRotationRef = useRef<number>(EBIKE_WORLD_ROTATION_Y);
|
||||||
const forkRef = useRef<THREE.Object3D | null>(null);
|
const forkRef = useRef<THREE.Object3D | null>(null);
|
||||||
|
const phareRef = useRef<THREE.Object3D | null>(null);
|
||||||
|
const headlightRef = useRef<THREE.SpotLight | null>(null);
|
||||||
|
// SpotLight target — must live in the scene to define the cone direction.
|
||||||
|
const headlightTarget = useMemo(() => new THREE.Object3D(), []);
|
||||||
|
// Original quaternion of the Fourche node — rotation is applied on top of this.
|
||||||
|
const forkInitialQuatRef = useRef(new THREE.Quaternion());
|
||||||
|
// Smoothed steer angle for the fork (avoids direct Euler manipulation).
|
||||||
|
const forkAngleRef = useRef(0);
|
||||||
|
// Ref copy of movementMode — useFrame closures can capture stale React state.
|
||||||
|
const movementModeRef = useRef(movementMode);
|
||||||
|
// Becomes true the first time the player mounts. After that, dismounting
|
||||||
|
// must NOT reset position back to the original spawn point.
|
||||||
|
const hasRiddenRef = useRef(false);
|
||||||
|
|
||||||
// State for debug visualization (synced from refs during useFrame)
|
// State for debug visualization (synced from refs during useFrame)
|
||||||
const [showCameraPoints, setShowCameraPoints] = useState(true);
|
const [showCameraPoints, setShowCameraPoints] = useState(true);
|
||||||
@@ -106,9 +126,42 @@ export function Ebike({ position }: EbikeProps): React.JSX.Element {
|
|||||||
parkedPosition[2],
|
parkedPosition[2],
|
||||||
]);
|
]);
|
||||||
|
|
||||||
|
// Keep movementModeRef in sync — useFrame closures capture React state at
|
||||||
|
// render time and can become stale between renders.
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
if (movementMode === "ebike") return;
|
movementModeRef.current = movementMode;
|
||||||
|
}, [movementMode]);
|
||||||
|
|
||||||
|
// SpotLight target must be in the scene to define the cone direction.
|
||||||
|
useEffect(() => {
|
||||||
|
threeScene.add(headlightTarget);
|
||||||
|
return () => { threeScene.remove(headlightTarget); };
|
||||||
|
}, [threeScene, headlightTarget]);
|
||||||
|
|
||||||
|
// Link the target to the SpotLight once it mounts.
|
||||||
|
useEffect(() => {
|
||||||
|
if (headlightRef.current) {
|
||||||
|
headlightRef.current.target = headlightTarget;
|
||||||
|
}
|
||||||
|
}, [headlightTarget]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (movementMode === "ebike") {
|
||||||
|
// Player just mounted — mark as ridden so we never reset position again.
|
||||||
|
hasRiddenRef.current = true;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (hasRiddenRef.current) {
|
||||||
|
// Player dismounted: keep the position the bike was left at.
|
||||||
|
// Just make sure the window vars are up to date for the next mount.
|
||||||
|
window.ebikeParkedPosition = restingPositionRef.current;
|
||||||
|
window.ebikeParkedRotation = restingRotationRef.current;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Bike has never been ridden yet — safe to (re)place it at the spawn point.
|
||||||
|
// This also fires when parkedPosition recalculates (e.g. terrain loads late).
|
||||||
restingPositionRef.current = parkedPosition;
|
restingPositionRef.current = parkedPosition;
|
||||||
restingRotationRef.current = EBIKE_WORLD_ROTATION_Y;
|
restingRotationRef.current = EBIKE_WORLD_ROTATION_Y;
|
||||||
lastGpsUpdatePos.current.set(...parkedPosition);
|
lastGpsUpdatePos.current.set(...parkedPosition);
|
||||||
@@ -123,11 +176,24 @@ export function Ebike({ position }: EbikeProps): React.JSX.Element {
|
|||||||
}, [movementMode, parkedPosition]);
|
}, [movementMode, parkedPosition]);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
if (model) {
|
if (!model) return;
|
||||||
const fork = model.getObjectByName("fourche");
|
|
||||||
if (fork) {
|
let forkNode: THREE.Object3D | null = null;
|
||||||
forkRef.current = fork;
|
model.traverse((child) => {
|
||||||
}
|
if (child.name.toLowerCase() === "fourche") forkNode = child;
|
||||||
|
if (child.name === "Phare") phareRef.current = child;
|
||||||
|
});
|
||||||
|
|
||||||
|
if (forkNode) {
|
||||||
|
forkRef.current = forkNode;
|
||||||
|
// Snapshot the rest-pose quaternion — steering is applied on top of this.
|
||||||
|
forkInitialQuatRef.current.copy((forkNode as THREE.Object3D).quaternion);
|
||||||
|
forkAngleRef.current = 0;
|
||||||
|
console.log("[Ebike] Fork found:", (forkNode as THREE.Object3D).name);
|
||||||
|
} else {
|
||||||
|
const names: string[] = [];
|
||||||
|
model.traverse((c) => { if (c.name) names.push(c.name); });
|
||||||
|
console.warn("[Ebike] Fork not found. All nodes:", names);
|
||||||
}
|
}
|
||||||
}, [model]);
|
}, [model]);
|
||||||
|
|
||||||
@@ -154,11 +220,48 @@ export function Ebike({ position }: EbikeProps): React.JSX.Element {
|
|||||||
}, []);
|
}, []);
|
||||||
|
|
||||||
useFrame((_, delta) => {
|
useFrame((_, delta) => {
|
||||||
|
// ── SpotLight headlight — tune the constants below ────────────────────────
|
||||||
|
// ── SpotLight headlight — tune these four constants ───────────────────────
|
||||||
|
const LIGHT_OFFSET_X = -0.7; // position : left(-) / right(+)
|
||||||
|
const LIGHT_OFFSET_Y = 1.5; // position : down(-) / up(+)
|
||||||
|
const LIGHT_OFFSET_Z = 0; // position : backward(-) / forward(+)
|
||||||
|
const LIGHT_AIM_DEG = 90; // aim rotation around Y : 0=forward, -90=left, +90=right
|
||||||
|
const LIGHT_TARGET_DIST = 20; // metres devant la position de la lumière
|
||||||
|
// ─────────────────────────────────────────────────────────────────────────
|
||||||
|
if (headlightRef.current && phareRef.current && groupRef.current) {
|
||||||
|
phareRef.current.getWorldPosition(_phareWorldPos);
|
||||||
|
groupRef.current.getWorldDirection(_bikeForward);
|
||||||
|
|
||||||
|
// Position offset in bike-local space (no GC — reusing module-level vectors)
|
||||||
|
const right = _bikeForward.clone().cross(_up).normalize();
|
||||||
|
_phareWorldPos
|
||||||
|
.addScaledVector(right, LIGHT_OFFSET_X)
|
||||||
|
.addScaledVector(_up, LIGHT_OFFSET_Y)
|
||||||
|
.addScaledVector(_bikeForward, LIGHT_OFFSET_Z);
|
||||||
|
|
||||||
|
headlightRef.current.position.copy(_phareWorldPos);
|
||||||
|
|
||||||
|
// Aim direction: rotate forward around Y by LIGHT_AIM_DEG
|
||||||
|
_aimDir
|
||||||
|
.copy(_bikeForward)
|
||||||
|
.applyAxisAngle(_up, THREE.MathUtils.degToRad(LIGHT_AIM_DEG));
|
||||||
|
|
||||||
|
headlightTarget.position
|
||||||
|
.copy(_phareWorldPos)
|
||||||
|
.addScaledVector(_aimDir, LIGHT_TARGET_DIST);
|
||||||
|
headlightTarget.updateMatrixWorld();
|
||||||
|
}
|
||||||
|
// ──────────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
if (groupRef.current) {
|
if (groupRef.current) {
|
||||||
if (movementMode === "ebike") {
|
// Use the ref — not the React state — to avoid stale closure bugs in
|
||||||
|
// R3F's frame loop (the state value may not update until the next render).
|
||||||
|
if (movementModeRef.current === "ebike") {
|
||||||
|
// Sound plays whenever the bike is actually moving (speedFactor > 5 %),
|
||||||
|
// not only while the input key is held.
|
||||||
updateEbikeSounds({
|
updateEbikeSounds({
|
||||||
mounted: true,
|
mounted: true,
|
||||||
driving: window.ebikeDriveInputActive === true,
|
driving: (window.ebikeSpeedFactor ?? 0) > 0.05,
|
||||||
breakdown: window.ebikeBreakdownActive === true,
|
breakdown: window.ebikeBreakdownActive === true,
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -169,16 +272,31 @@ export function Ebike({ position }: EbikeProps): React.JSX.Element {
|
|||||||
];
|
];
|
||||||
restingRotationRef.current = groupRef.current.rotation.y;
|
restingRotationRef.current = groupRef.current.rotation.y;
|
||||||
|
|
||||||
// Smoothly rotate the front fork ("fourche") up to 15 degrees in its own Z axis
|
// ── Fork steering via quaternion ──────────────────────────────────────
|
||||||
|
// We rotate around the fork's LOCAL Y axis (steering tube) by composing
|
||||||
|
// a fresh quaternion on top of the rest-pose snapshot taken at load time.
|
||||||
|
// This is axis-agnostic: correct regardless of how Blender exported the node.
|
||||||
|
// Tune FORK_ANGLE (radians) or negate it if the visual direction is wrong.
|
||||||
|
const FORK_ANGLE = 0.12; // 10°
|
||||||
const steerFactor = window.ebikeSteerFactor ?? 0;
|
const steerFactor = window.ebikeSteerFactor ?? 0;
|
||||||
|
|
||||||
if (forkRef.current) {
|
if (forkRef.current) {
|
||||||
// 15 degrees is 0.26 radians
|
// Smooth the angle separately so we can apply it cleanly each frame.
|
||||||
const targetForkRotation = steerFactor * 0.26;
|
forkAngleRef.current = THREE.MathUtils.lerp(
|
||||||
forkRef.current.rotation.z = THREE.MathUtils.lerp(
|
forkAngleRef.current,
|
||||||
forkRef.current.rotation.z,
|
steerFactor * FORK_ANGLE,
|
||||||
targetForkRotation,
|
|
||||||
12 * delta,
|
12 * delta,
|
||||||
);
|
);
|
||||||
|
// Build steer quat around LOCAL Y of the fork node.
|
||||||
|
const steerQuat = new THREE.Quaternion().setFromAxisAngle(
|
||||||
|
new THREE.Vector3(0, 1, 0),
|
||||||
|
forkAngleRef.current,
|
||||||
|
);
|
||||||
|
// Apply on top of rest-pose: Q_final = Q_rest × Q_steer
|
||||||
|
forkRef.current.quaternion.multiplyQuaternions(
|
||||||
|
forkInitialQuatRef.current,
|
||||||
|
steerQuat,
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Throttled GPS start position update to prevent performance loss
|
// Throttled GPS start position update to prevent performance loss
|
||||||
@@ -197,9 +315,10 @@ export function Ebike({ position }: EbikeProps): React.JSX.Element {
|
|||||||
groupRef.current.position.set(...restingPositionRef.current);
|
groupRef.current.position.set(...restingPositionRef.current);
|
||||||
groupRef.current.rotation.set(0, restingRotationRef.current, 0);
|
groupRef.current.rotation.set(0, restingRotationRef.current, 0);
|
||||||
|
|
||||||
// Reset fork rotation when parked
|
// Reset fork to rest-pose when parked
|
||||||
if (forkRef.current) {
|
if (forkRef.current) {
|
||||||
forkRef.current.rotation.z = 0;
|
forkRef.current.quaternion.copy(forkInitialQuatRef.current);
|
||||||
|
forkAngleRef.current = 0;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
window.ebikeParkedPosition = restingPositionRef.current;
|
window.ebikeParkedPosition = restingPositionRef.current;
|
||||||
@@ -329,6 +448,9 @@ export function Ebike({ position }: EbikeProps): React.JSX.Element {
|
|||||||
scale={EBIKE_WORLD_SCALE}
|
scale={EBIKE_WORLD_SCALE}
|
||||||
>
|
>
|
||||||
<primitive object={model} />
|
<primitive object={model} />
|
||||||
|
{/* radius 20 → ~7 unités monde (scale 0.35).
|
||||||
|
Sphère omnidirectionnelle pour que le raycast fonctionne
|
||||||
|
quelle que soit l'orientation de la caméra (montée ou à pied). */}
|
||||||
<InteractableObject
|
<InteractableObject
|
||||||
kind="trigger"
|
kind="trigger"
|
||||||
label={interactionLabel}
|
label={interactionLabel}
|
||||||
@@ -337,16 +459,25 @@ export function Ebike({ position }: EbikeProps): React.JSX.Element {
|
|||||||
onPress={handleInteract}
|
onPress={handleInteract}
|
||||||
>
|
>
|
||||||
<mesh>
|
<mesh>
|
||||||
<boxGeometry args={[8, 9, 2]} />
|
<sphereGeometry args={[8, 15, 12]} />
|
||||||
<meshBasicMaterial colorWrite={false} depthWrite={false} />
|
<meshBasicMaterial colorWrite={false} color={"red"} depthWrite={false} />
|
||||||
</mesh>
|
</mesh>
|
||||||
</InteractableObject>
|
</InteractableObject>
|
||||||
|
|
||||||
{/* Dynamic 3D GPS Dashboard Screen */}
|
{/* GPS + Speedmeter – same group so they are perfectly co-localised.
|
||||||
<group position={[0, 7, 0]} rotation={[0, 90, 0]}>
|
GPS: full circle (Fresnel mask), renderOrder 10 000
|
||||||
|
Speedmeter: upper-half arc overlay, renderOrder 10 001
|
||||||
|
rotation: Math.PI/2 radians = 90° (NOT the number 90 which = ~116.6°) */}
|
||||||
|
<group position={[2, 6, 0]} rotation={[0, -80, 0]}>
|
||||||
|
<EbikeSpeedmeter width={3} height={1.5} position={[0, 0.4, 0]} gaugeInnerR={0.33} gaugeOuterR={0.445}
|
||||||
|
gaugeWidth={2.5}
|
||||||
|
gaugeHeight={2.1}
|
||||||
|
gaugeOffsetX={0}
|
||||||
|
gaugeOffsetY={-0.19}
|
||||||
|
/>
|
||||||
<EbikeGPSMap
|
<EbikeGPSMap
|
||||||
width={0.8}
|
width={1.3}
|
||||||
height={0.8}
|
height={1}
|
||||||
startPos={gpsStartPos}
|
startPos={gpsStartPos}
|
||||||
destPos={destPos}
|
destPos={destPos}
|
||||||
mapImageUrl="/assets/world/gps/map_background.png"
|
mapImageUrl="/assets/world/gps/map_background.png"
|
||||||
@@ -359,15 +490,26 @@ export function Ebike({ position }: EbikeProps): React.JSX.Element {
|
|||||||
zoom={4}
|
zoom={4}
|
||||||
/>
|
/>
|
||||||
</group>
|
</group>
|
||||||
<group position={[0, 6.35, 0]} rotation={[0, 90, 0]}>
|
|
||||||
<EbikeSpeedometer />
|
|
||||||
</group>
|
|
||||||
</group>
|
</group>
|
||||||
) : null}
|
) : null}
|
||||||
|
|
||||||
|
{/* SpotLight headlight — cone aimed forward, position & target via useFrame */}
|
||||||
|
{!repairGameOwnsEbikeModel && (
|
||||||
|
<spotLight
|
||||||
|
ref={headlightRef}
|
||||||
|
intensity={100}
|
||||||
|
color="#ffca60"
|
||||||
|
angle={Math.PI / 5} // 22.5° demi-angle — cone étroit comme une torche
|
||||||
|
penumbra={0.5} // bord doux (0 = dur, 1 = très doux)
|
||||||
|
distance={50}
|
||||||
|
decay={2.5}
|
||||||
|
castShadow={false}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
|
||||||
{showCameraPoints && !repairGameOwnsEbikeModel && (
|
{showCameraPoints && !repairGameOwnsEbikeModel && (
|
||||||
<>
|
<>
|
||||||
<mesh position={camPointPos}>
|
{/* <mesh position={camPointPos}>
|
||||||
<sphereGeometry args={[0.3, 16, 16]} />
|
<sphereGeometry args={[0.3, 16, 16]} />
|
||||||
<meshStandardMaterial
|
<meshStandardMaterial
|
||||||
color="yellow"
|
color="yellow"
|
||||||
@@ -382,7 +524,7 @@ export function Ebike({ position }: EbikeProps): React.JSX.Element {
|
|||||||
emissive="cyan"
|
emissive="cyan"
|
||||||
emissiveIntensity={0.5}
|
emissiveIntensity={0.5}
|
||||||
/>
|
/>
|
||||||
</mesh>
|
</mesh> */}
|
||||||
</>
|
</>
|
||||||
)}
|
)}
|
||||||
</>
|
</>
|
||||||
|
|||||||
@@ -12,6 +12,28 @@ import {
|
|||||||
} from "@/pathfinding/WaypointAStar";
|
} from "@/pathfinding/WaypointAStar";
|
||||||
import type { Waypoint } from "@/pathfinding/types";
|
import type { Waypoint } from "@/pathfinding/types";
|
||||||
import type { Vector3Tuple } from "@/types/three/three";
|
import type { Vector3Tuple } from "@/types/three/three";
|
||||||
|
|
||||||
|
const VERT_SHADER = /* glsl */ `
|
||||||
|
varying vec2 vUv;
|
||||||
|
void main() {
|
||||||
|
vUv = uv;
|
||||||
|
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
|
||||||
|
}
|
||||||
|
`;
|
||||||
|
|
||||||
|
// Circular Fresnel mask: fully visible inside innerRadius, fades out to outerRadius
|
||||||
|
const FRAG_SHADER = /* glsl */ `
|
||||||
|
uniform sampler2D map;
|
||||||
|
uniform float innerRadius;
|
||||||
|
uniform float outerRadius;
|
||||||
|
varying vec2 vUv;
|
||||||
|
void main() {
|
||||||
|
vec4 color = texture2D(map, vUv);
|
||||||
|
float dist = length(vUv - vec2(0.5));
|
||||||
|
float mask = 1.0 - smoothstep(innerRadius, outerRadius, dist);
|
||||||
|
gl_FragColor = vec4(color.rgb, color.a * mask);
|
||||||
|
}
|
||||||
|
`;
|
||||||
function computeImageSource(
|
function computeImageSource(
|
||||||
img: HTMLImageElement | HTMLCanvasElement,
|
img: HTMLImageElement | HTMLCanvasElement,
|
||||||
baseBounds: { minX: number; maxX: number; minZ: number; maxZ: number },
|
baseBounds: { minX: number; maxX: number; minZ: number; maxZ: number },
|
||||||
@@ -126,19 +148,57 @@ export const EbikeGPSMap: React.FC<EbikeGPSMapProps> = ({
|
|||||||
// eslint-disable-next-line react-hooks/exhaustive-deps -- Canvas should only be created once
|
// eslint-disable-next-line react-hooks/exhaustive-deps -- Canvas should only be created once
|
||||||
}, []);
|
}, []);
|
||||||
|
|
||||||
// Resize the canvas whenever canvasSize changes
|
|
||||||
// Note: Modifying canvas dimensions is intentional and necessary for rendering
|
|
||||||
useEffect(() => {
|
|
||||||
// Use Object.assign to resize canvas - this is a necessary mutation for canvas rendering
|
|
||||||
Object.assign(offscreenCanvas, { width: canvasSize, height: canvasSize });
|
|
||||||
if (textureRef.current) {
|
|
||||||
textureRef.current.needsUpdate = true;
|
|
||||||
}
|
|
||||||
}, [canvasSize, offscreenCanvas]);
|
|
||||||
|
|
||||||
const textureRef = useRef<THREE.CanvasTexture | null>(null);
|
|
||||||
const animTimeRef = useRef<number>(0);
|
const animTimeRef = useRef<number>(0);
|
||||||
|
|
||||||
|
// Imperative CanvasTexture — must be declared before the resize effect below
|
||||||
|
const texture = useMemo(() => {
|
||||||
|
const tex = new THREE.CanvasTexture(offscreenCanvas);
|
||||||
|
tex.format = THREE.RGBAFormat;
|
||||||
|
tex.minFilter = THREE.LinearFilter;
|
||||||
|
tex.magFilter = THREE.LinearFilter;
|
||||||
|
return tex;
|
||||||
|
}, [offscreenCanvas]);
|
||||||
|
|
||||||
|
// ShaderMaterial with circular Fresnel mask (created once)
|
||||||
|
const shaderMat = useMemo(
|
||||||
|
() =>
|
||||||
|
new THREE.ShaderMaterial({
|
||||||
|
uniforms: {
|
||||||
|
map: { value: null },
|
||||||
|
innerRadius: { value: 0.45 },
|
||||||
|
outerRadius: { value: 0.5 },
|
||||||
|
},
|
||||||
|
vertexShader: VERT_SHADER,
|
||||||
|
fragmentShader: FRAG_SHADER,
|
||||||
|
transparent: true,
|
||||||
|
depthTest: false,
|
||||||
|
depthWrite: false,
|
||||||
|
side: THREE.DoubleSide,
|
||||||
|
toneMapped: false,
|
||||||
|
}),
|
||||||
|
[],
|
||||||
|
);
|
||||||
|
|
||||||
|
// Sync texture into uniform when it changes (canvas resize)
|
||||||
|
useEffect(() => {
|
||||||
|
shaderMat.uniforms.map.value = texture;
|
||||||
|
}, [shaderMat, texture]);
|
||||||
|
|
||||||
|
// Cleanup on unmount
|
||||||
|
useEffect(
|
||||||
|
() => () => {
|
||||||
|
shaderMat.dispose();
|
||||||
|
texture.dispose();
|
||||||
|
},
|
||||||
|
[shaderMat, texture],
|
||||||
|
);
|
||||||
|
|
||||||
|
// Resize the canvas whenever canvasSize changes (texture declared above)
|
||||||
|
useEffect(() => {
|
||||||
|
Object.assign(offscreenCanvas, { width: canvasSize, height: canvasSize });
|
||||||
|
texture.needsUpdate = true;
|
||||||
|
}, [canvasSize, offscreenCanvas, texture]);
|
||||||
|
|
||||||
// Load waypoints (localStorage with /roadNetwork.json fallback)
|
// Load waypoints (localStorage with /roadNetwork.json fallback)
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
let cancelled = false;
|
let cancelled = false;
|
||||||
@@ -492,42 +552,20 @@ export const EbikeGPSMap: React.FC<EbikeGPSMapProps> = ({
|
|||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
let animId: number;
|
let animId: number;
|
||||||
const tick = () => {
|
const tick = () => {
|
||||||
animTimeRef.current += 0.004; // Slow, premium sweep speed
|
animTimeRef.current += 0.004;
|
||||||
if (animTimeRef.current > 1) animTimeRef.current = 0;
|
if (animTimeRef.current > 1) animTimeRef.current = 0;
|
||||||
|
|
||||||
draw();
|
draw();
|
||||||
|
texture.needsUpdate = true;
|
||||||
// Update texture after draw
|
|
||||||
if (textureRef.current) {
|
|
||||||
textureRef.current.needsUpdate = true;
|
|
||||||
}
|
|
||||||
|
|
||||||
animId = requestAnimationFrame(tick);
|
animId = requestAnimationFrame(tick);
|
||||||
};
|
};
|
||||||
animId = requestAnimationFrame(tick);
|
animId = requestAnimationFrame(tick);
|
||||||
return () => cancelAnimationFrame(animId);
|
return () => cancelAnimationFrame(animId);
|
||||||
}, [draw]);
|
}, [draw, texture]);
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<mesh position={position} renderOrder={renderOrder}>
|
<mesh position={position} renderOrder={renderOrder}>
|
||||||
<planeGeometry args={[width, height]} />
|
<planeGeometry args={[width, height]} />
|
||||||
<meshBasicMaterial
|
<primitive object={shaderMat} attach="material" />
|
||||||
toneMapped={false}
|
|
||||||
transparent={true}
|
|
||||||
opacity={1}
|
|
||||||
depthTest={false}
|
|
||||||
depthWrite={false}
|
|
||||||
side={THREE.DoubleSide}
|
|
||||||
>
|
|
||||||
<canvasTexture
|
|
||||||
ref={textureRef}
|
|
||||||
attach="map"
|
|
||||||
image={offscreenCanvas}
|
|
||||||
format={THREE.RGBAFormat}
|
|
||||||
minFilter={THREE.LinearFilter}
|
|
||||||
magFilter={THREE.LinearFilter}
|
|
||||||
/>
|
|
||||||
</meshBasicMaterial>
|
|
||||||
</mesh>
|
</mesh>
|
||||||
);
|
);
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -0,0 +1,233 @@
|
|||||||
|
import { useEffect, useRef, useMemo } from "react";
|
||||||
|
import { useFrame } from "@react-three/fiber";
|
||||||
|
import { useTexture } from "@react-three/drei";
|
||||||
|
import * as THREE from "three";
|
||||||
|
import type { Vector3Tuple } from "@/types/three/three";
|
||||||
|
import "@/types/ebike/ebikeWindow";
|
||||||
|
|
||||||
|
const SPEEDOMETER_DIAL_TEXTURE = "/assets/world/gps/cadran.png";
|
||||||
|
const SPEEDOMETER_NEEDLE_TEXTURE = "/assets/world/gps/fleche.png";
|
||||||
|
|
||||||
|
export interface EbikeSpeedmeterProps {
|
||||||
|
width?: number;
|
||||||
|
height?: number;
|
||||||
|
/** Local position offset within the parent group. Default: [0, 0, 0] */
|
||||||
|
position?: Vector3Tuple;
|
||||||
|
/**
|
||||||
|
* Needle rotation.z when speedFactor = 0.
|
||||||
|
* Default: Math.PI / 2 (pointing left — 9 o'clock)
|
||||||
|
*/
|
||||||
|
minAngle?: number;
|
||||||
|
/**
|
||||||
|
* Needle rotation.z when speedFactor = 1.
|
||||||
|
* Default: -Math.PI / 2 (pointing right — 3 o'clock)
|
||||||
|
*/
|
||||||
|
maxAngle?: number;
|
||||||
|
renderOrder?: number;
|
||||||
|
/**
|
||||||
|
* Inner radius of the gauge-fill arc, as a fraction of the canvas half-width.
|
||||||
|
* Tune this to align the fill with the cadran.png track. Default: 0.33
|
||||||
|
*/
|
||||||
|
gaugeInnerR?: number;
|
||||||
|
/**
|
||||||
|
* Outer radius of the gauge-fill arc, as a fraction of the canvas half-width.
|
||||||
|
* Tune this to align the fill with the cadran.png track. Default: 0.445
|
||||||
|
*/
|
||||||
|
gaugeOuterR?: number;
|
||||||
|
/**
|
||||||
|
* Width of the gauge-fill plane. Defaults to `width` when omitted.
|
||||||
|
* Lets you resize the fill independently of the cadran/needle.
|
||||||
|
*/
|
||||||
|
gaugeWidth?: number;
|
||||||
|
/**
|
||||||
|
* Height of the gauge-fill plane. Defaults to `height` when omitted.
|
||||||
|
* Lets you resize the fill independently of the cadran/needle.
|
||||||
|
*/
|
||||||
|
gaugeHeight?: number;
|
||||||
|
/**
|
||||||
|
* Horizontal offset of the arc pivot from the canvas centre.
|
||||||
|
* Expressed as a fraction of the canvas size: -0.1 = shift 10 % to the left,
|
||||||
|
* +0.1 = shift 10 % to the right. Default: 0
|
||||||
|
*/
|
||||||
|
gaugeOffsetX?: number;
|
||||||
|
/**
|
||||||
|
* Vertical offset of the arc pivot from its default position.
|
||||||
|
* Expressed as a fraction of the canvas size: -0.1 = shift upward (toward top
|
||||||
|
* of the plane), +0.1 = shift downward. Default: 0
|
||||||
|
*/
|
||||||
|
gaugeOffsetY?: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
// The needle pivot is always at -height*0.38 in local space,
|
||||||
|
// which is always 12 % from the bottom of the plane (UV y = 0.12).
|
||||||
|
// With Three.js flipY texture convention, canvas y = (1 - 0.12) * size = 0.88 * size.
|
||||||
|
const NEEDLE_PIVOT_UV_Y = 0.12; // fraction from bottom
|
||||||
|
|
||||||
|
export function EbikeSpeedmeter({
|
||||||
|
width = 0.8,
|
||||||
|
height = 0.8,
|
||||||
|
position = [0, 0, 0],
|
||||||
|
minAngle = Math.PI / 2,
|
||||||
|
maxAngle = -Math.PI / 2,
|
||||||
|
renderOrder = 1000,
|
||||||
|
gaugeInnerR = 0.33,
|
||||||
|
gaugeOuterR = 0.445,
|
||||||
|
gaugeWidth,
|
||||||
|
gaugeHeight,
|
||||||
|
gaugeOffsetX = 0,
|
||||||
|
gaugeOffsetY = 0,
|
||||||
|
}: EbikeSpeedmeterProps): React.JSX.Element {
|
||||||
|
// Fall back to the main dimensions when gauge-specific ones aren't provided
|
||||||
|
const fillW = gaugeWidth ?? width;
|
||||||
|
const fillH = gaugeHeight ?? height;
|
||||||
|
const needleGroupRef = useRef<THREE.Group>(null);
|
||||||
|
const speedFactorRef = useRef(0);
|
||||||
|
|
||||||
|
// ── Dial & needle textures ──────────────────────────────────────────────────
|
||||||
|
const [dialTexture, needleTexture] = useTexture([
|
||||||
|
SPEEDOMETER_DIAL_TEXTURE,
|
||||||
|
SPEEDOMETER_NEEDLE_TEXTURE,
|
||||||
|
]) as [THREE.Texture, THREE.Texture];
|
||||||
|
|
||||||
|
const needleWidth = width * 0.68;
|
||||||
|
const needleHeight = needleWidth / 2;
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
[dialTexture, needleTexture].forEach((tex) => {
|
||||||
|
tex.colorSpace = THREE.SRGBColorSpace;
|
||||||
|
tex.needsUpdate = true;
|
||||||
|
});
|
||||||
|
}, [dialTexture, needleTexture]);
|
||||||
|
|
||||||
|
// ── Gauge-fill canvas ───────────────────────────────────────────────────────
|
||||||
|
const fillCanvas = useMemo(() => {
|
||||||
|
const c = document.createElement("canvas");
|
||||||
|
c.width = 256;
|
||||||
|
c.height = 256;
|
||||||
|
return c;
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
const fillTexture = useMemo(() => {
|
||||||
|
const tex = new THREE.CanvasTexture(fillCanvas);
|
||||||
|
tex.format = THREE.RGBAFormat;
|
||||||
|
tex.minFilter = THREE.LinearFilter;
|
||||||
|
tex.magFilter = THREE.LinearFilter;
|
||||||
|
return tex;
|
||||||
|
}, [fillCanvas]);
|
||||||
|
|
||||||
|
useEffect(
|
||||||
|
() => () => {
|
||||||
|
fillTexture.dispose();
|
||||||
|
},
|
||||||
|
[fillTexture],
|
||||||
|
);
|
||||||
|
|
||||||
|
// ── Frame loop ──────────────────────────────────────────────────────────────
|
||||||
|
useFrame((_, delta) => {
|
||||||
|
// 1. Smooth speed factor
|
||||||
|
const target = THREE.MathUtils.clamp(window.ebikeSpeedFactor ?? 0, 0, 1);
|
||||||
|
speedFactorRef.current = THREE.MathUtils.lerp(
|
||||||
|
speedFactorRef.current,
|
||||||
|
target,
|
||||||
|
Math.min(1, delta * 10),
|
||||||
|
);
|
||||||
|
|
||||||
|
// 2. Needle rotation
|
||||||
|
if (needleGroupRef.current) {
|
||||||
|
needleGroupRef.current.rotation.z = THREE.MathUtils.lerp(
|
||||||
|
minAngle,
|
||||||
|
maxAngle,
|
||||||
|
speedFactorRef.current,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
// 3. Draw gauge fill -------------------------------------------------------
|
||||||
|
const ctx = fillCanvas.getContext("2d", { alpha: true });
|
||||||
|
if (!ctx) return;
|
||||||
|
|
||||||
|
const size = fillCanvas.width;
|
||||||
|
ctx.clearRect(0, 0, size, size);
|
||||||
|
|
||||||
|
// Default centre: horizontal middle + needle-pivot height.
|
||||||
|
// gaugeOffsetX/Y shift the pivot so the arc aligns with cadran.png.
|
||||||
|
const cx = size * (0.5 + gaugeOffsetX);
|
||||||
|
const cy = size * ((1 - NEEDLE_PIVOT_UV_Y) + gaugeOffsetY); // default ≈ 0.88 × size
|
||||||
|
|
||||||
|
const outerR = size * gaugeOuterR;
|
||||||
|
const innerR = size * gaugeInnerR;
|
||||||
|
|
||||||
|
// Arc sweeps clockwise from π (left) to current needle angle
|
||||||
|
const arcStart = Math.PI;
|
||||||
|
const arcEnd = Math.PI + speedFactorRef.current * Math.PI;
|
||||||
|
|
||||||
|
if (speedFactorRef.current > 0.005) {
|
||||||
|
// Radial gradient using #3F67DD — slightly transparent at inner edge,
|
||||||
|
// fully solid at outer edge for a depth effect.
|
||||||
|
const radial = ctx.createRadialGradient(cx, cy, innerR, cx, cy, outerR);
|
||||||
|
radial.addColorStop(0, "rgba(191, 234, 255, 0)"); // inner edge
|
||||||
|
radial.addColorStop(0.7, "rgba(118, 152, 255, 0.95)"); // outer edge
|
||||||
|
|
||||||
|
// Annular sector shape (outer arc + inner arc reversed)
|
||||||
|
ctx.beginPath();
|
||||||
|
ctx.arc(cx, cy, outerR, arcStart, arcEnd, false);
|
||||||
|
ctx.arc(cx, cy, innerR, arcEnd, arcStart, true);
|
||||||
|
ctx.closePath();
|
||||||
|
|
||||||
|
ctx.fillStyle = radial;
|
||||||
|
ctx.shadowBlur = 16;
|
||||||
|
ctx.shadowColor = "#3F67DD";
|
||||||
|
ctx.fill();
|
||||||
|
ctx.shadowBlur = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
fillTexture.needsUpdate = true;
|
||||||
|
});
|
||||||
|
|
||||||
|
return (
|
||||||
|
<group renderOrder={renderOrder} position={position}>
|
||||||
|
{/* Gauge fill — behind the cadran frame (size controlled by gaugeWidth/gaugeHeight) */}
|
||||||
|
<mesh renderOrder={renderOrder - 1} position={[0, 0, -0.001]}>
|
||||||
|
<planeGeometry args={[fillW, fillH]} />
|
||||||
|
<meshBasicMaterial
|
||||||
|
map={fillTexture}
|
||||||
|
transparent
|
||||||
|
depthTest={false}
|
||||||
|
depthWrite={false}
|
||||||
|
toneMapped={false}
|
||||||
|
side={THREE.DoubleSide}
|
||||||
|
/>
|
||||||
|
</mesh>
|
||||||
|
|
||||||
|
{/* Dial frame (cadran.png) */}
|
||||||
|
<mesh renderOrder={renderOrder}>
|
||||||
|
<planeGeometry args={[width, height]} />
|
||||||
|
<meshBasicMaterial
|
||||||
|
map={dialTexture}
|
||||||
|
transparent
|
||||||
|
depthTest={false}
|
||||||
|
depthWrite={false}
|
||||||
|
toneMapped={false}
|
||||||
|
side={THREE.DoubleSide}
|
||||||
|
/>
|
||||||
|
</mesh>
|
||||||
|
|
||||||
|
{/* Needle — pivot at bottom-centre of the arc */}
|
||||||
|
<group ref={needleGroupRef} position={[0, -height * 0.38, 0.002]} rotation={[0, 0, 0]}>
|
||||||
|
<mesh
|
||||||
|
position={[0, needleHeight / 2, 0]}
|
||||||
|
renderOrder={renderOrder + 1}
|
||||||
|
>
|
||||||
|
<planeGeometry args={[needleWidth, needleHeight]} />
|
||||||
|
<meshBasicMaterial
|
||||||
|
map={needleTexture}
|
||||||
|
transparent
|
||||||
|
depthTest={false}
|
||||||
|
depthWrite={false}
|
||||||
|
toneMapped={false}
|
||||||
|
side={THREE.DoubleSide}
|
||||||
|
/>
|
||||||
|
</mesh>
|
||||||
|
</group>
|
||||||
|
</group>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -15,11 +15,15 @@ import {
|
|||||||
REPAIR_CASE_OPEN_ROTATION_OFFSET_DEGREES,
|
REPAIR_CASE_OPEN_ROTATION_OFFSET_DEGREES,
|
||||||
REPAIR_CASE_CLOSE_SOUND_PATH,
|
REPAIR_CASE_CLOSE_SOUND_PATH,
|
||||||
REPAIR_CASE_OPEN_SOUND_PATH,
|
REPAIR_CASE_OPEN_SOUND_PATH,
|
||||||
|
REPAIR_CASE_PART_ANCHOR_FALLBACK_QUATERNION,
|
||||||
|
REPAIR_CASE_PART_ANCHOR_FALLBACKS,
|
||||||
|
REPAIR_CASE_PART_ANCHOR_NAMES,
|
||||||
REPAIR_CASE_PLACEHOLDER_NAME_PREFIX,
|
REPAIR_CASE_PLACEHOLDER_NAME_PREFIX,
|
||||||
REPAIR_CASE_POP_DURATION,
|
REPAIR_CASE_POP_DURATION,
|
||||||
REPAIR_CASE_POP_Y_OFFSET,
|
REPAIR_CASE_POP_Y_OFFSET,
|
||||||
REPAIR_CASE_ROTATION_AMPLITUDE_DEGREES,
|
REPAIR_CASE_ROTATION_AMPLITUDE_DEGREES,
|
||||||
REPAIR_CASE_ROTATION_RESET_SPEED,
|
REPAIR_CASE_ROTATION_RESET_SPEED,
|
||||||
|
type RepairCasePartAnchorName,
|
||||||
} from "@/data/gameplay/repairCaseConfig";
|
} from "@/data/gameplay/repairCaseConfig";
|
||||||
import { useClonedObject } from "@/hooks/three/useClonedObject";
|
import { useClonedObject } from "@/hooks/three/useClonedObject";
|
||||||
import { useLoggedGLTF } from "@/hooks/three/useLoggedGLTF";
|
import { useLoggedGLTF } from "@/hooks/three/useLoggedGLTF";
|
||||||
@@ -32,6 +36,10 @@ export interface RepairCasePlaceholder {
|
|||||||
position: Vector3Tuple;
|
position: Vector3Tuple;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export type RepairCasePartAnchors = Partial<
|
||||||
|
Record<RepairCasePartAnchorName, Vector3Tuple>
|
||||||
|
>;
|
||||||
|
|
||||||
interface RepairCaseModelProps extends ModelTransformProps {
|
interface RepairCaseModelProps extends ModelTransformProps {
|
||||||
modelPath: string;
|
modelPath: string;
|
||||||
open: boolean;
|
open: boolean;
|
||||||
@@ -40,6 +48,7 @@ interface RepairCaseModelProps extends ModelTransformProps {
|
|||||||
onPlaceholdersChange?:
|
onPlaceholdersChange?:
|
||||||
| ((placeholders: readonly RepairCasePlaceholder[]) => void)
|
| ((placeholders: readonly RepairCasePlaceholder[]) => void)
|
||||||
| undefined;
|
| undefined;
|
||||||
|
onAnchorsChange?: ((anchors: RepairCasePartAnchors) => void) | undefined;
|
||||||
onExitComplete?: (() => void) | undefined;
|
onExitComplete?: (() => void) | undefined;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -59,6 +68,7 @@ export function RepairCaseModel({
|
|||||||
exiting = false,
|
exiting = false,
|
||||||
floating = true,
|
floating = true,
|
||||||
onPlaceholdersChange,
|
onPlaceholdersChange,
|
||||||
|
onAnchorsChange,
|
||||||
onExitComplete,
|
onExitComplete,
|
||||||
position = [0, 0, 0],
|
position = [0, 0, 0],
|
||||||
rotation = [0, 0, 0],
|
rotation = [0, 0, 0],
|
||||||
@@ -81,6 +91,7 @@ export function RepairCaseModel({
|
|||||||
const pop = useRef({ scale: 0.001, yOffset: REPAIR_CASE_POP_Y_OFFSET });
|
const pop = useRef({ scale: 0.001, yOffset: REPAIR_CASE_POP_Y_OFFSET });
|
||||||
const onExitCompleteRef = useRef(onExitComplete);
|
const onExitCompleteRef = useRef(onExitComplete);
|
||||||
const onPlaceholdersChangeRef = useRef(onPlaceholdersChange);
|
const onPlaceholdersChangeRef = useRef(onPlaceholdersChange);
|
||||||
|
const onAnchorsChangeRef = useRef(onAnchorsChange);
|
||||||
const initialOpen = useRef(open);
|
const initialOpen = useRef(open);
|
||||||
const previousOpen = useRef(open);
|
const previousOpen = useRef(open);
|
||||||
const openedRotationZ = useRef(0);
|
const openedRotationZ = useRef(0);
|
||||||
@@ -89,6 +100,12 @@ export function RepairCaseModel({
|
|||||||
const placeholderSignature = useRef("__initial__");
|
const placeholderSignature = useRef("__initial__");
|
||||||
const placeholderPosition = useRef(new THREE.Vector3());
|
const placeholderPosition = useRef(new THREE.Vector3());
|
||||||
const placeholderLocalPosition = useRef(new THREE.Vector3());
|
const placeholderLocalPosition = useRef(new THREE.Vector3());
|
||||||
|
const anchorNodes = useRef<Map<RepairCasePartAnchorName, THREE.Object3D>>(
|
||||||
|
new Map(),
|
||||||
|
);
|
||||||
|
const anchorSignature = useRef("__initial__");
|
||||||
|
const anchorWorldPosition = useRef(new THREE.Vector3());
|
||||||
|
const anchorLocalPosition = useRef(new THREE.Vector3());
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
onExitCompleteRef.current = onExitComplete;
|
onExitCompleteRef.current = onExitComplete;
|
||||||
@@ -98,6 +115,10 @@ export function RepairCaseModel({
|
|||||||
onPlaceholdersChangeRef.current = onPlaceholdersChange;
|
onPlaceholdersChangeRef.current = onPlaceholdersChange;
|
||||||
}, [onPlaceholdersChange]);
|
}, [onPlaceholdersChange]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
onAnchorsChangeRef.current = onAnchorsChange;
|
||||||
|
}, [onAnchorsChange]);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
const popAnimation = pop.current;
|
const popAnimation = pop.current;
|
||||||
|
|
||||||
@@ -153,6 +174,37 @@ export function RepairCaseModel({
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
|
// Resolve part anchor nodes (cabledroit, cablegauche, pucehaut, pucebas,
|
||||||
|
// refroidisseur). Existing GLTF nodes are reused and their meshes are
|
||||||
|
// hidden so the standalone model injected at the same position becomes
|
||||||
|
// the only visible representation. Missing nodes are created on the fly
|
||||||
|
// at the configured fallback case-local position.
|
||||||
|
anchorNodes.current = new Map();
|
||||||
|
REPAIR_CASE_PART_ANCHOR_NAMES.forEach((anchorName) => {
|
||||||
|
let node = model.getObjectByName(anchorName);
|
||||||
|
if (node) {
|
||||||
|
node.traverse((descendant) => {
|
||||||
|
if ((descendant as THREE.Mesh).isMesh) {
|
||||||
|
descendant.visible = false;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
} else {
|
||||||
|
const placeholder = new THREE.Object3D();
|
||||||
|
placeholder.name = anchorName;
|
||||||
|
const fallback = REPAIR_CASE_PART_ANCHOR_FALLBACKS[anchorName];
|
||||||
|
placeholder.position.set(fallback[0], fallback[1], fallback[2]);
|
||||||
|
placeholder.quaternion.set(
|
||||||
|
REPAIR_CASE_PART_ANCHOR_FALLBACK_QUATERNION[0],
|
||||||
|
REPAIR_CASE_PART_ANCHOR_FALLBACK_QUATERNION[1],
|
||||||
|
REPAIR_CASE_PART_ANCHOR_FALLBACK_QUATERNION[2],
|
||||||
|
REPAIR_CASE_PART_ANCHOR_FALLBACK_QUATERNION[3],
|
||||||
|
);
|
||||||
|
model.add(placeholder);
|
||||||
|
node = placeholder;
|
||||||
|
}
|
||||||
|
anchorNodes.current.set(anchorName, node);
|
||||||
|
});
|
||||||
|
|
||||||
if (lid) {
|
if (lid) {
|
||||||
lid.rotation.z =
|
lid.rotation.z =
|
||||||
openedRotationZ.current +
|
openedRotationZ.current +
|
||||||
@@ -250,6 +302,31 @@ export function RepairCaseModel({
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (anchorNodes.current.size > 0) {
|
||||||
|
const anchors: RepairCasePartAnchors = {};
|
||||||
|
const signatureParts: string[] = [];
|
||||||
|
anchorNodes.current.forEach((node, anchorName) => {
|
||||||
|
node.getWorldPosition(anchorWorldPosition.current);
|
||||||
|
anchorLocalPosition.current.copy(anchorWorldPosition.current);
|
||||||
|
group.parent?.worldToLocal(anchorLocalPosition.current);
|
||||||
|
const tuple: Vector3Tuple = [
|
||||||
|
anchorLocalPosition.current.x,
|
||||||
|
anchorLocalPosition.current.y,
|
||||||
|
anchorLocalPosition.current.z,
|
||||||
|
];
|
||||||
|
anchors[anchorName] = tuple;
|
||||||
|
signatureParts.push(
|
||||||
|
`${anchorName}:${tuple.map((value) => value.toFixed(3)).join(",")}`,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
signatureParts.sort();
|
||||||
|
const nextAnchorSignature = signatureParts.join("|");
|
||||||
|
if (nextAnchorSignature !== anchorSignature.current) {
|
||||||
|
anchorSignature.current = nextAnchorSignature;
|
||||||
|
onAnchorsChangeRef.current?.(anchors);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
animationActiveRef.current = isNear;
|
animationActiveRef.current = isNear;
|
||||||
|
|
||||||
if (animationActiveRef.current) {
|
if (animationActiveRef.current) {
|
||||||
|
|||||||
@@ -1,7 +1,11 @@
|
|||||||
import { Suspense, useEffect, useMemo, useState } from "react";
|
import { Suspense, useEffect, useMemo, useState } from "react";
|
||||||
import { useGLTF } from "@react-three/drei";
|
import { useGLTF } from "@react-three/drei";
|
||||||
import { ExplodableModel } from "@/components/three/models/ExplodableModel";
|
import { ExplodableModel } from "@/components/three/models/ExplodableModel";
|
||||||
import type { RepairCasePlaceholder } from "@/components/three/gameplay/RepairCaseModel";
|
import type { ExplodedNodeAnchors } from "@/components/three/models/ExplodableModel";
|
||||||
|
import type {
|
||||||
|
RepairCasePartAnchors,
|
||||||
|
RepairCasePlaceholder,
|
||||||
|
} from "@/components/three/gameplay/RepairCaseModel";
|
||||||
import { RepairCompletionStep } from "@/components/three/gameplay/RepairCompletionStep";
|
import { RepairCompletionStep } from "@/components/three/gameplay/RepairCompletionStep";
|
||||||
import { RepairInspectionObject } from "@/components/three/gameplay/RepairInspectionObject";
|
import { RepairInspectionObject } from "@/components/three/gameplay/RepairInspectionObject";
|
||||||
import { RepairMissionCase } from "@/components/three/gameplay/RepairMissionCase";
|
import { RepairMissionCase } from "@/components/three/gameplay/RepairMissionCase";
|
||||||
@@ -63,12 +67,15 @@ export function RepairGame({
|
|||||||
const [casePlaceholders, setCasePlaceholders] = useState<
|
const [casePlaceholders, setCasePlaceholders] = useState<
|
||||||
readonly RepairCasePlaceholder[]
|
readonly RepairCasePlaceholder[]
|
||||||
>([]);
|
>([]);
|
||||||
|
const [caseAnchors, setCaseAnchors] = useState<RepairCasePartAnchors>({});
|
||||||
|
const [brokenAnchors, setBrokenAnchors] = useState<ExplodedNodeAnchors>({});
|
||||||
const [scannedBrokenParts, setScannedBrokenParts] = useState<
|
const [scannedBrokenParts, setScannedBrokenParts] = useState<
|
||||||
readonly RepairScannedBrokenPart[]
|
readonly RepairScannedBrokenPart[]
|
||||||
>([]);
|
>([]);
|
||||||
const parsedScale = toVector3Scale(scale);
|
const parsedScale = toVector3Scale(scale);
|
||||||
const snappedPosition = useTerrainSnappedPosition(position);
|
const snappedPosition = useTerrainSnappedPosition(position);
|
||||||
const readyForFragmentation = step === "inspected";
|
const readyForFragmentation = step === "inspected";
|
||||||
|
const brokenNodeNames = useMemo(() => getBrokenNodeNames(config), [config]);
|
||||||
|
|
||||||
useRepairFragmentationInput({
|
useRepairFragmentationInput({
|
||||||
enabled: mainState === mission && readyForFragmentation,
|
enabled: mainState === mission && readyForFragmentation,
|
||||||
@@ -81,6 +88,8 @@ export function RepairGame({
|
|||||||
|
|
||||||
const timeoutId = window.setTimeout(() => {
|
const timeoutId = window.setTimeout(() => {
|
||||||
setCasePlaceholders([]);
|
setCasePlaceholders([]);
|
||||||
|
setCaseAnchors({});
|
||||||
|
setBrokenAnchors({});
|
||||||
setScannedBrokenParts([]);
|
setScannedBrokenParts([]);
|
||||||
}, 0);
|
}, 0);
|
||||||
|
|
||||||
@@ -136,12 +145,24 @@ export function RepairGame({
|
|||||||
/>
|
/>
|
||||||
) : null}
|
) : null}
|
||||||
{step === "repairing" ? (
|
{step === "repairing" ? (
|
||||||
<RepairRepairingStep
|
<>
|
||||||
brokenParts={scannedBrokenParts}
|
<ExplodableModel
|
||||||
config={config}
|
modelPath={config.modelPath}
|
||||||
placeholders={casePlaceholders}
|
scale={config.modelScale ?? 1}
|
||||||
onRepair={() => setMissionStep(mission, "reassembling")}
|
split
|
||||||
/>
|
hideNodeNames={brokenNodeNames}
|
||||||
|
nodeAnchorNames={brokenNodeNames}
|
||||||
|
onNodeAnchorsChange={setBrokenAnchors}
|
||||||
|
/>
|
||||||
|
<RepairRepairingStep
|
||||||
|
anchors={caseAnchors}
|
||||||
|
brokenAnchors={brokenAnchors}
|
||||||
|
brokenParts={scannedBrokenParts}
|
||||||
|
config={config}
|
||||||
|
placeholders={casePlaceholders}
|
||||||
|
onRepair={() => setMissionStep(mission, "reassembling")}
|
||||||
|
/>
|
||||||
|
</>
|
||||||
) : null}
|
) : null}
|
||||||
{step === "reassembling" ? (
|
{step === "reassembling" ? (
|
||||||
<RepairReassemblyStep
|
<RepairReassemblyStep
|
||||||
@@ -159,6 +180,7 @@ export function RepairGame({
|
|||||||
<RepairMissionCase
|
<RepairMissionCase
|
||||||
config={config}
|
config={config}
|
||||||
onPlaceholdersChange={setCasePlaceholders}
|
onPlaceholdersChange={setCasePlaceholders}
|
||||||
|
onAnchorsChange={setCaseAnchors}
|
||||||
open={step === "repairing"}
|
open={step === "repairing"}
|
||||||
zoomed={step === "repairing"}
|
zoomed={step === "repairing"}
|
||||||
showFragmentationPrompt={readyForFragmentation}
|
showFragmentationPrompt={readyForFragmentation}
|
||||||
@@ -188,3 +210,15 @@ function getRepairMissionModelPaths(config: RepairMissionConfig): string[] {
|
|||||||
]),
|
]),
|
||||||
];
|
];
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function getBrokenNodeNames(config: RepairMissionConfig): readonly string[] {
|
||||||
|
const names = new Set<string>();
|
||||||
|
config.brokenParts.forEach((part) => {
|
||||||
|
if (part.targetNodeName) names.add(part.targetNodeName);
|
||||||
|
else if (part.nodeName) names.add(part.nodeName);
|
||||||
|
});
|
||||||
|
config.replacementParts.forEach((part) => {
|
||||||
|
if (part.targetNodeName) names.add(part.targetNodeName);
|
||||||
|
});
|
||||||
|
return Array.from(names);
|
||||||
|
}
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
import {
|
import {
|
||||||
RepairCaseModel,
|
RepairCaseModel,
|
||||||
|
type RepairCasePartAnchors,
|
||||||
type RepairCasePlaceholder,
|
type RepairCasePlaceholder,
|
||||||
} from "@/components/three/gameplay/RepairCaseModel";
|
} from "@/components/three/gameplay/RepairCaseModel";
|
||||||
import { RepairPromptVideo } from "@/components/three/gameplay/RepairPromptVideo";
|
import { RepairPromptVideo } from "@/components/three/gameplay/RepairPromptVideo";
|
||||||
@@ -19,6 +20,7 @@ interface RepairMissionCaseProps {
|
|||||||
onPlaceholdersChange?:
|
onPlaceholdersChange?:
|
||||||
| ((placeholders: readonly RepairCasePlaceholder[]) => void)
|
| ((placeholders: readonly RepairCasePlaceholder[]) => void)
|
||||||
| undefined;
|
| undefined;
|
||||||
|
onAnchorsChange?: ((anchors: RepairCasePartAnchors) => void) | undefined;
|
||||||
onExitComplete?: (() => void) | undefined;
|
onExitComplete?: (() => void) | undefined;
|
||||||
open?: boolean;
|
open?: boolean;
|
||||||
zoomed?: boolean;
|
zoomed?: boolean;
|
||||||
@@ -30,6 +32,7 @@ export function RepairMissionCase({
|
|||||||
config,
|
config,
|
||||||
exiting = false,
|
exiting = false,
|
||||||
onPlaceholdersChange,
|
onPlaceholdersChange,
|
||||||
|
onAnchorsChange,
|
||||||
onExitComplete,
|
onExitComplete,
|
||||||
open = false,
|
open = false,
|
||||||
zoomed = false,
|
zoomed = false,
|
||||||
@@ -57,6 +60,7 @@ export function RepairMissionCase({
|
|||||||
exiting={exiting}
|
exiting={exiting}
|
||||||
onExitComplete={onExitComplete}
|
onExitComplete={onExitComplete}
|
||||||
onPlaceholdersChange={onPlaceholdersChange}
|
onPlaceholdersChange={onPlaceholdersChange}
|
||||||
|
onAnchorsChange={onAnchorsChange}
|
||||||
open={open}
|
open={open}
|
||||||
floating={!zoomed}
|
floating={!zoomed}
|
||||||
position={modelPosition}
|
position={modelPosition}
|
||||||
@@ -70,6 +74,7 @@ export function RepairMissionCase({
|
|||||||
exiting={exiting}
|
exiting={exiting}
|
||||||
onExitComplete={onExitComplete}
|
onExitComplete={onExitComplete}
|
||||||
onPlaceholdersChange={onPlaceholdersChange}
|
onPlaceholdersChange={onPlaceholdersChange}
|
||||||
|
onAnchorsChange={onAnchorsChange}
|
||||||
open={open}
|
open={open}
|
||||||
floating={!zoomed}
|
floating={!zoomed}
|
||||||
position={modelPosition}
|
position={modelPosition}
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import { toVector3Scale } from "@/utils/three/scale";
|
|||||||
interface RepairObjectModelProps extends ModelTransformProps {
|
interface RepairObjectModelProps extends ModelTransformProps {
|
||||||
label: string;
|
label: string;
|
||||||
modelPath: string;
|
modelPath: string;
|
||||||
|
ghosted?: boolean;
|
||||||
}
|
}
|
||||||
|
|
||||||
interface RepairObjectModelBoundaryProps extends RepairObjectModelProps {
|
interface RepairObjectModelBoundaryProps extends RepairObjectModelProps {
|
||||||
@@ -73,6 +74,7 @@ export function RepairObjectModel({
|
|||||||
position = [0, 0, 0],
|
position = [0, 0, 0],
|
||||||
rotation = [0, 0, 0],
|
rotation = [0, 0, 0],
|
||||||
scale = 1,
|
scale = 1,
|
||||||
|
ghosted = false,
|
||||||
}: RepairObjectModelProps): React.JSX.Element {
|
}: RepairObjectModelProps): React.JSX.Element {
|
||||||
return (
|
return (
|
||||||
<RepairObjectModelBoundary
|
<RepairObjectModelBoundary
|
||||||
@@ -87,6 +89,7 @@ export function RepairObjectModel({
|
|||||||
position={position}
|
position={position}
|
||||||
rotation={rotation}
|
rotation={rotation}
|
||||||
scale={scale}
|
scale={scale}
|
||||||
|
opacity={ghosted ? 0.35 : 1}
|
||||||
/>
|
/>
|
||||||
</RepairObjectModelBoundary>
|
</RepairObjectModelBoundary>
|
||||||
);
|
);
|
||||||
|
|||||||
@@ -1,6 +1,10 @@
|
|||||||
import { useEffect, useRef, useState } from "react";
|
import { useEffect, useRef, useState } from "react";
|
||||||
import * as THREE from "three";
|
import * as THREE from "three";
|
||||||
import type { RepairCasePlaceholder } from "@/components/three/gameplay/RepairCaseModel";
|
import type {
|
||||||
|
RepairCasePartAnchors,
|
||||||
|
RepairCasePlaceholder,
|
||||||
|
} from "@/components/three/gameplay/RepairCaseModel";
|
||||||
|
import type { ExplodedNodeAnchors } from "@/components/three/models/ExplodableModel";
|
||||||
import { RepairObjectModel } from "@/components/three/gameplay/RepairObjectModel";
|
import { RepairObjectModel } from "@/components/three/gameplay/RepairObjectModel";
|
||||||
import { RepairPromptVideo } from "@/components/three/gameplay/RepairPromptVideo";
|
import { RepairPromptVideo } from "@/components/three/gameplay/RepairPromptVideo";
|
||||||
import { GrabbableObject } from "@/components/three/interaction/GrabbableObject";
|
import { GrabbableObject } from "@/components/three/interaction/GrabbableObject";
|
||||||
@@ -38,6 +42,8 @@ const STORED_BROKEN_PART_COLOR = "#38bdf8";
|
|||||||
let hasWarnedMissingPlaceholders = false;
|
let hasWarnedMissingPlaceholders = false;
|
||||||
|
|
||||||
interface RepairRepairingStepProps {
|
interface RepairRepairingStepProps {
|
||||||
|
anchors?: RepairCasePartAnchors;
|
||||||
|
brokenAnchors?: ExplodedNodeAnchors;
|
||||||
brokenParts: readonly RepairScannedBrokenPart[];
|
brokenParts: readonly RepairScannedBrokenPart[];
|
||||||
config: RepairMissionConfig;
|
config: RepairMissionConfig;
|
||||||
placeholders: readonly RepairCasePlaceholder[];
|
placeholders: readonly RepairCasePlaceholder[];
|
||||||
@@ -63,6 +69,8 @@ interface RepairPartPlacementFeedbackProps {
|
|||||||
}
|
}
|
||||||
|
|
||||||
export function RepairRepairingStep({
|
export function RepairRepairingStep({
|
||||||
|
anchors = {},
|
||||||
|
brokenAnchors = {},
|
||||||
brokenParts,
|
brokenParts,
|
||||||
config,
|
config,
|
||||||
placeholders,
|
placeholders,
|
||||||
@@ -76,12 +84,15 @@ export function RepairRepairingStep({
|
|||||||
const [depositedBrokenPartIds, setDepositedBrokenPartIds] = useState<
|
const [depositedBrokenPartIds, setDepositedBrokenPartIds] = useState<
|
||||||
Record<string, boolean>
|
Record<string, boolean>
|
||||||
>({});
|
>({});
|
||||||
|
const [heldPartByLockGroup, setHeldPartByLockGroup] = useState<
|
||||||
|
Record<string, string>
|
||||||
|
>({});
|
||||||
const [showBlockedInstallFeedback, setShowBlockedInstallFeedback] =
|
const [showBlockedInstallFeedback, setShowBlockedInstallFeedback] =
|
||||||
useState(false);
|
useState(false);
|
||||||
const replacementParts = getReplacementParts(config);
|
const replacementParts = getReplacementParts(config);
|
||||||
const brokenPartsToDeposit = getBrokenPartsToDeposit(config, brokenParts);
|
const brokenPartsToDeposit = getBrokenPartsToDeposit(config, brokenParts);
|
||||||
const requiredReplacementPart = replacementParts.find(
|
const requiredReplacementPart = replacementParts.find((part) =>
|
||||||
(part) => part.id === config.requiredReplacementPartId,
|
config.requiredReplacementPartIds.includes(part.id),
|
||||||
);
|
);
|
||||||
const requiredReplacementLabel =
|
const requiredReplacementLabel =
|
||||||
requiredReplacementPart?.label ?? config.label;
|
requiredReplacementPart?.label ?? config.label;
|
||||||
@@ -89,15 +100,16 @@ export function RepairRepairingStep({
|
|||||||
const placeholderPositions = placeholderTargets.map(
|
const placeholderPositions = placeholderTargets.map(
|
||||||
(target) => target.position,
|
(target) => target.position,
|
||||||
);
|
);
|
||||||
const hasCorrectPartPlaced = Boolean(
|
const hasCorrectPartPlaced = config.requiredReplacementPartIds.some(
|
||||||
placedPartIds[config.requiredReplacementPartId],
|
(id) => placedPartIds[id],
|
||||||
);
|
);
|
||||||
const hasDepositedBrokenParts = brokenPartsToDeposit.every(
|
const hasDepositedBrokenParts = brokenPartsToDeposit.every(
|
||||||
(part) => depositedBrokenPartIds[part.id],
|
(part) => depositedBrokenPartIds[part.id],
|
||||||
);
|
);
|
||||||
const hasWrongPartPlaced = replacementParts.some(
|
const hasWrongPartPlaced = replacementParts.some(
|
||||||
(part) =>
|
(part) =>
|
||||||
part.id !== config.requiredReplacementPartId && placedPartIds[part.id],
|
!config.requiredReplacementPartIds.includes(part.id) &&
|
||||||
|
placedPartIds[part.id],
|
||||||
);
|
);
|
||||||
const isReadyToInstall = hasCorrectPartPlaced && hasDepositedBrokenParts;
|
const isReadyToInstall = hasCorrectPartPlaced && hasDepositedBrokenParts;
|
||||||
const installColor = isReadyToInstall
|
const installColor = isReadyToInstall
|
||||||
@@ -177,6 +189,24 @@ export function RepairRepairingStep({
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function handleReplacementGrabChange(
|
||||||
|
part: RepairMissionPartConfig,
|
||||||
|
held: boolean,
|
||||||
|
): void {
|
||||||
|
if (!part.caseLockGroup) return;
|
||||||
|
const group = part.caseLockGroup;
|
||||||
|
setHeldPartByLockGroup((current) => {
|
||||||
|
if (held) {
|
||||||
|
if (current[group] === part.id) return current;
|
||||||
|
return { ...current, [group]: part.id };
|
||||||
|
}
|
||||||
|
if (current[group] !== part.id) return current;
|
||||||
|
const next = { ...current };
|
||||||
|
delete next[group];
|
||||||
|
return next;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<group ref={groupRef}>
|
<group ref={groupRef}>
|
||||||
<RepairInstallTarget
|
<RepairInstallTarget
|
||||||
@@ -192,15 +222,23 @@ export function RepairRepairingStep({
|
|||||||
<RepairPlaceholderMarkers positions={placeholderPositions} />
|
<RepairPlaceholderMarkers positions={placeholderPositions} />
|
||||||
|
|
||||||
{replacementParts.map((part, index) => {
|
{replacementParts.map((part, index) => {
|
||||||
|
const anchorPosition = part.caseAnchor
|
||||||
|
? anchors[part.caseAnchor]
|
||||||
|
: undefined;
|
||||||
const placeholderPosition =
|
const placeholderPosition =
|
||||||
|
anchorPosition ??
|
||||||
placeholderPositions[index % placeholderPositions.length] ??
|
placeholderPositions[index % placeholderPositions.length] ??
|
||||||
placeholderPositions[0]!;
|
placeholderPositions[0]!;
|
||||||
const isPlaced = Boolean(placedPartIds[part.id]);
|
const isPlaced = Boolean(placedPartIds[part.id]);
|
||||||
const feedbackState = getReplacementFeedbackState(
|
const feedbackState = getReplacementFeedbackState(
|
||||||
part.id,
|
part.id,
|
||||||
config.requiredReplacementPartId,
|
config.requiredReplacementPartIds,
|
||||||
isPlaced,
|
isPlaced,
|
||||||
);
|
);
|
||||||
|
const lockedByOther =
|
||||||
|
part.caseLockGroup !== undefined &&
|
||||||
|
heldPartByLockGroup[part.caseLockGroup] !== undefined &&
|
||||||
|
heldPartByLockGroup[part.caseLockGroup] !== part.id;
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<GrabbableObject
|
<GrabbableObject
|
||||||
@@ -208,7 +246,11 @@ export function RepairRepairingStep({
|
|||||||
position={placeholderPosition}
|
position={placeholderPosition}
|
||||||
colliders="ball"
|
colliders="ball"
|
||||||
handControlled
|
handControlled
|
||||||
|
disabled={lockedByOther}
|
||||||
label={`Prendre ${part.label}`}
|
label={`Prendre ${part.label}`}
|
||||||
|
onGrabChange={(held) => {
|
||||||
|
handleReplacementGrabChange(part, held);
|
||||||
|
}}
|
||||||
onPositionChange={(position) => {
|
onPositionChange={(position) => {
|
||||||
handleReplacementPosition(part.id, position);
|
handleReplacementPosition(part.id, position);
|
||||||
}}
|
}}
|
||||||
@@ -224,6 +266,7 @@ export function RepairRepairingStep({
|
|||||||
label={part.label}
|
label={part.label}
|
||||||
modelPath={part.modelPath ?? config.modelPath}
|
modelPath={part.modelPath ?? config.modelPath}
|
||||||
scale={0.36}
|
scale={0.36}
|
||||||
|
ghosted={lockedByOther}
|
||||||
/>
|
/>
|
||||||
<RepairPartPlacementFeedback state={feedbackState} />
|
<RepairPartPlacementFeedback state={feedbackState} />
|
||||||
</group>
|
</group>
|
||||||
@@ -232,14 +275,18 @@ export function RepairRepairingStep({
|
|||||||
})}
|
})}
|
||||||
|
|
||||||
{brokenPartsToDeposit.map((part, index) => {
|
{brokenPartsToDeposit.map((part, index) => {
|
||||||
const startOffset =
|
const fallbackOffset =
|
||||||
BROKEN_PART_START_OFFSETS[index % BROKEN_PART_START_OFFSETS.length] ??
|
BROKEN_PART_START_OFFSETS[index % BROKEN_PART_START_OFFSETS.length] ??
|
||||||
BROKEN_PART_START_OFFSETS[0]!;
|
BROKEN_PART_START_OFFSETS[0]!;
|
||||||
const startPosition: Vector3Tuple = [
|
const fallbackPosition: Vector3Tuple = [
|
||||||
REPAIR_CASE_FOCUS_POSITION[0] + startOffset[0],
|
REPAIR_CASE_FOCUS_POSITION[0] + fallbackOffset[0],
|
||||||
REPAIR_CASE_FOCUS_POSITION[1] + startOffset[1],
|
REPAIR_CASE_FOCUS_POSITION[1] + fallbackOffset[1],
|
||||||
REPAIR_CASE_FOCUS_POSITION[2] + startOffset[2],
|
REPAIR_CASE_FOCUS_POSITION[2] + fallbackOffset[2],
|
||||||
];
|
];
|
||||||
|
const anchorPosition = part.targetNodeName
|
||||||
|
? brokenAnchors[part.targetNodeName]
|
||||||
|
: undefined;
|
||||||
|
const startPosition: Vector3Tuple = anchorPosition ?? fallbackPosition;
|
||||||
const targetPositions = getBrokenPartTargetPositions(
|
const targetPositions = getBrokenPartTargetPositions(
|
||||||
part,
|
part,
|
||||||
placeholderTargets,
|
placeholderTargets,
|
||||||
@@ -387,12 +434,12 @@ function getPlacementFeedbackColor(
|
|||||||
|
|
||||||
function getReplacementFeedbackState(
|
function getReplacementFeedbackState(
|
||||||
partId: string,
|
partId: string,
|
||||||
requiredPartId: string,
|
requiredPartIds: readonly string[],
|
||||||
isPlaced: boolean,
|
isPlaced: boolean,
|
||||||
): RepairPartPlacementFeedbackProps["state"] {
|
): RepairPartPlacementFeedbackProps["state"] {
|
||||||
if (!isPlaced) return null;
|
if (!isPlaced) return null;
|
||||||
|
|
||||||
return partId === requiredPartId ? "valid" : "invalid";
|
return requiredPartIds.includes(partId) ? "valid" : "invalid";
|
||||||
}
|
}
|
||||||
|
|
||||||
function getPlaceholderTargets(
|
function getPlaceholderTargets(
|
||||||
@@ -466,9 +513,12 @@ function getReplacementParts(
|
|||||||
): readonly RepairMissionPartConfig[] {
|
): readonly RepairMissionPartConfig[] {
|
||||||
if (config.replacementParts.length > 0) return config.replacementParts;
|
if (config.replacementParts.length > 0) return config.replacementParts;
|
||||||
|
|
||||||
|
const fallbackId =
|
||||||
|
config.requiredReplacementPartIds[0] ?? `${config.id}-replacement`;
|
||||||
|
|
||||||
return [
|
return [
|
||||||
{
|
{
|
||||||
id: config.requiredReplacementPartId,
|
id: fallbackId,
|
||||||
label: config.label,
|
label: config.label,
|
||||||
modelPath: config.modelPath,
|
modelPath: config.modelPath,
|
||||||
},
|
},
|
||||||
@@ -486,5 +536,6 @@ function getBrokenPartsToDeposit(
|
|||||||
label: part.label,
|
label: part.label,
|
||||||
modelPath: part.modelPath ?? config.modelPath,
|
modelPath: part.modelPath ?? config.modelPath,
|
||||||
...(part.caseSlotName ? { caseSlotName: part.caseSlotName } : {}),
|
...(part.caseSlotName ? { caseSlotName: part.caseSlotName } : {}),
|
||||||
|
...(part.targetNodeName ? { targetNodeName: part.targetNodeName } : {}),
|
||||||
}));
|
}));
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -97,6 +97,9 @@ function getScannedBrokenParts(
|
|||||||
...(match.config.caseSlotName
|
...(match.config.caseSlotName
|
||||||
? { caseSlotName: match.config.caseSlotName }
|
? { caseSlotName: match.config.caseSlotName }
|
||||||
: {}),
|
: {}),
|
||||||
|
...(match.config.targetNodeName
|
||||||
|
? { targetNodeName: match.config.targetNodeName }
|
||||||
|
: {}),
|
||||||
};
|
};
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,6 +12,11 @@ import { useLoggedGLTF } from "@/hooks/three/useLoggedGLTF";
|
|||||||
import type { HandTrackingLandmark } from "@/types/handTracking/handTracking";
|
import type { HandTrackingLandmark } from "@/types/handTracking/handTracking";
|
||||||
import { logModelLoadError } from "@/utils/three/modelLoadLogger";
|
import { logModelLoadError } from "@/utils/three/modelLoadLogger";
|
||||||
|
|
||||||
|
// Both gloves share the same source mesh (gant_l). The right glove is
|
||||||
|
// rendered by mirroring scale.x at the group level — this is more
|
||||||
|
// reliable than the historical gant_r GLTF, which embeds multiple
|
||||||
|
// skeletons (Hand_l, Hand_l_pad, Hand_r) and was breaking the finger
|
||||||
|
// rig.
|
||||||
const GLOVE_CONFIGS: Record<
|
const GLOVE_CONFIGS: Record<
|
||||||
HandTrackingGloveHandedness,
|
HandTrackingGloveHandedness,
|
||||||
{
|
{
|
||||||
@@ -24,8 +29,8 @@ const GLOVE_CONFIGS: Record<
|
|||||||
rootNodeName: "Armature",
|
rootNodeName: "Armature",
|
||||||
},
|
},
|
||||||
right: {
|
right: {
|
||||||
modelPath: "/models/gant_r/model.gltf",
|
modelPath: "/models/gant_l/model.gltf",
|
||||||
rootNodeName: "Hand_r",
|
rootNodeName: "Armature",
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -226,7 +231,10 @@ function applyFingerPose(
|
|||||||
_boneTargetQuaternion
|
_boneTargetQuaternion
|
||||||
.copy(_boneDeltaQuaternion)
|
.copy(_boneDeltaQuaternion)
|
||||||
.multiply(pose.restQuaternion);
|
.multiply(pose.restQuaternion);
|
||||||
pose.bone.quaternion.slerp(_boneTargetQuaternion, 0.45);
|
// Lower slerp factor = smoother but more latency. MediaPipe at
|
||||||
|
// ~10fps produces noisy landmark frames; smoothing cuts the
|
||||||
|
// jitter the user sees on every finger bone.
|
||||||
|
pose.bone.quaternion.slerp(_boneTargetQuaternion, 0.3);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -334,12 +342,18 @@ function HandTrackingGloveModel({
|
|||||||
_matrix.makeBasis(_xAxis, _yAxis, _zAxis);
|
_matrix.makeBasis(_xAxis, _yAxis, _zAxis);
|
||||||
_targetQuaternion.setFromRotationMatrix(_matrix);
|
_targetQuaternion.setFromRotationMatrix(_matrix);
|
||||||
|
|
||||||
group.position.lerp(_targetPosition, Math.min(1, delta * 18));
|
// Lower factor (was 18) damps the glove jitter caused by noisy
|
||||||
group.quaternion.slerp(_targetQuaternion, Math.min(1, delta * 18));
|
// landmarks while keeping a responsive feel.
|
||||||
|
group.position.lerp(_targetPosition, Math.min(1, delta * 12));
|
||||||
|
group.quaternion.slerp(_targetQuaternion, Math.min(1, delta * 12));
|
||||||
|
|
||||||
const palmLength = _wristPosition.distanceTo(_middlePosition);
|
const palmLength = _wristPosition.distanceTo(_middlePosition);
|
||||||
const scale = palmLength * GLOVE_MODEL_SCALE;
|
const scale = palmLength * GLOVE_MODEL_SCALE;
|
||||||
group.scale.setScalar(scale);
|
// Both gloves use the gant_l mesh; flip X for the right hand so the
|
||||||
|
// thumb ends up on the correct side instead of being a left-glove
|
||||||
|
// clone on the right hand.
|
||||||
|
const mirrorSignX = handedness === "right" ? -1 : 1;
|
||||||
|
group.scale.set(scale * mirrorSignX, scale, scale);
|
||||||
group.updateMatrixWorld(true);
|
group.updateMatrixWorld(true);
|
||||||
applyFingerPose(fingerPoseChains, trackedHand.landmarks, camera);
|
applyFingerPose(fingerPoseChains, trackedHand.landmarks, camera);
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -34,6 +34,8 @@ interface GrabbableObjectProps {
|
|||||||
colliders?: ColliderShape;
|
colliders?: ColliderShape;
|
||||||
label?: string;
|
label?: string;
|
||||||
handControlled?: boolean;
|
handControlled?: boolean;
|
||||||
|
disabled?: boolean;
|
||||||
|
onGrabChange?: (held: boolean) => void;
|
||||||
onPositionChange?: (position: THREE.Vector3) => void;
|
onPositionChange?: (position: THREE.Vector3) => void;
|
||||||
onSnap?: (position: THREE.Vector3) => void;
|
onSnap?: (position: THREE.Vector3) => void;
|
||||||
snapDuration?: number;
|
snapDuration?: number;
|
||||||
@@ -131,6 +133,8 @@ export function GrabbableObject({
|
|||||||
colliders = GRAB_DEFAULT_COLLIDERS,
|
colliders = GRAB_DEFAULT_COLLIDERS,
|
||||||
label = GRAB_DEFAULT_LABEL,
|
label = GRAB_DEFAULT_LABEL,
|
||||||
handControlled = false,
|
handControlled = false,
|
||||||
|
disabled = false,
|
||||||
|
onGrabChange,
|
||||||
onPositionChange,
|
onPositionChange,
|
||||||
onSnap,
|
onSnap,
|
||||||
snapDuration = 0.25,
|
snapDuration = 0.25,
|
||||||
@@ -152,6 +156,19 @@ export function GrabbableObject({
|
|||||||
};
|
};
|
||||||
}, []);
|
}, []);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (!disabled) return;
|
||||||
|
if (isHolding.current) {
|
||||||
|
isHolding.current = false;
|
||||||
|
onGrabChange?.(false);
|
||||||
|
}
|
||||||
|
if (isHandHolding.current) {
|
||||||
|
isHandHolding.current = false;
|
||||||
|
InteractionManager.getInstance().setHandHolding(false);
|
||||||
|
onGrabChange?.(false);
|
||||||
|
}
|
||||||
|
}, [disabled, onGrabChange]);
|
||||||
|
|
||||||
function snapToNearestTarget(): void {
|
function snapToNearestTarget(): void {
|
||||||
const body = rbRef.current;
|
const body = rbRef.current;
|
||||||
if (!body || snapTargets.length === 0 || snapRadius <= 0) return;
|
if (!body || snapTargets.length === 0 || snapRadius <= 0) return;
|
||||||
@@ -242,14 +259,16 @@ export function GrabbableObject({
|
|||||||
useFrame(() => {
|
useFrame(() => {
|
||||||
if (!rbRef.current) return;
|
if (!rbRef.current) return;
|
||||||
|
|
||||||
const fistHand = handControlled
|
|
||||||
? hands.find((hand) => hand.isFist)
|
|
||||||
: undefined;
|
|
||||||
|
|
||||||
const t = rbRef.current.translation();
|
const t = rbRef.current.translation();
|
||||||
_currentPos.set(t.x, t.y, t.z);
|
_currentPos.set(t.x, t.y, t.z);
|
||||||
onPositionChange?.(_currentPos);
|
onPositionChange?.(_currentPos);
|
||||||
|
|
||||||
|
if (disabled) return;
|
||||||
|
|
||||||
|
const fistHand = handControlled
|
||||||
|
? hands.find((hand) => hand.isFist)
|
||||||
|
: undefined;
|
||||||
|
|
||||||
if (fistHand) {
|
if (fistHand) {
|
||||||
const handCenter = getHandCenterPoint(fistHand);
|
const handCenter = getHandCenterPoint(fistHand);
|
||||||
|
|
||||||
@@ -267,15 +286,20 @@ export function GrabbableObject({
|
|||||||
? getHandHit(groupRef.current, camera, _cameraPos, handCenter)
|
? getHandHit(groupRef.current, camera, _cameraPos, handCenter)
|
||||||
: null;
|
: null;
|
||||||
|
|
||||||
isHandHolding.current = Boolean(hit);
|
const hadHit = Boolean(hit);
|
||||||
InteractionManager.getInstance().setHandHolding(isHandHolding.current);
|
if (hadHit) {
|
||||||
|
isHandHolding.current = true;
|
||||||
|
InteractionManager.getInstance().setHandHolding(true);
|
||||||
|
onGrabChange?.(true);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
if (isHandHolding.current) {
|
if (isHandHolding.current) {
|
||||||
snapToNearestTarget();
|
snapToNearestTarget();
|
||||||
|
isHandHolding.current = false;
|
||||||
|
InteractionManager.getInstance().setHandHolding(false);
|
||||||
|
onGrabChange?.(false);
|
||||||
}
|
}
|
||||||
isHandHolding.current = false;
|
|
||||||
InteractionManager.getInstance().setHandHolding(false);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if (!isHolding.current && !isHandHolding.current) return;
|
if (!isHolding.current && !isHandHolding.current) return;
|
||||||
@@ -311,35 +335,41 @@ export function GrabbableObject({
|
|||||||
position={position}
|
position={position}
|
||||||
>
|
>
|
||||||
<group ref={groupRef}>
|
<group ref={groupRef}>
|
||||||
<InteractableObject
|
{disabled ? (
|
||||||
kind="grab"
|
children
|
||||||
label={label}
|
) : (
|
||||||
position={position}
|
<InteractableObject
|
||||||
bodyRef={rbRef}
|
kind="grab"
|
||||||
onPress={() => {
|
label={label}
|
||||||
isHolding.current = true;
|
position={position}
|
||||||
}}
|
bodyRef={rbRef}
|
||||||
onRelease={() => {
|
onPress={() => {
|
||||||
isHolding.current = false;
|
isHolding.current = true;
|
||||||
snapToNearestTarget();
|
onGrabChange?.(true);
|
||||||
if (
|
}}
|
||||||
!rbRef.current ||
|
onRelease={() => {
|
||||||
grabDebugParams.throwBoost === GRAB_THROW_BOOST_DEFAULT
|
isHolding.current = false;
|
||||||
)
|
onGrabChange?.(false);
|
||||||
return;
|
snapToNearestTarget();
|
||||||
const v = rbRef.current.linvel();
|
if (
|
||||||
rbRef.current.setLinvel(
|
!rbRef.current ||
|
||||||
{
|
grabDebugParams.throwBoost === GRAB_THROW_BOOST_DEFAULT
|
||||||
x: v.x * grabDebugParams.throwBoost,
|
)
|
||||||
y: v.y * grabDebugParams.throwBoost,
|
return;
|
||||||
z: v.z * grabDebugParams.throwBoost,
|
const v = rbRef.current.linvel();
|
||||||
},
|
rbRef.current.setLinvel(
|
||||||
true,
|
{
|
||||||
);
|
x: v.x * grabDebugParams.throwBoost,
|
||||||
}}
|
y: v.y * grabDebugParams.throwBoost,
|
||||||
>
|
z: v.z * grabDebugParams.throwBoost,
|
||||||
{children}
|
},
|
||||||
</InteractableObject>
|
true,
|
||||||
|
);
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{children}
|
||||||
|
</InteractableObject>
|
||||||
|
)}
|
||||||
</group>
|
</group>
|
||||||
</RigidBody>
|
</RigidBody>
|
||||||
</group>
|
</group>
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
import type { ReactNode } from "react";
|
import type { ReactNode } from "react";
|
||||||
import { Component, useEffect, useMemo } from "react";
|
import { Component, useEffect, useMemo, useRef } from "react";
|
||||||
|
import * as THREE from "three";
|
||||||
import { useFrame } from "@react-three/fiber";
|
import { useFrame } from "@react-three/fiber";
|
||||||
import { useLoggedGLTF } from "@/hooks/three/useLoggedGLTF";
|
import { useLoggedGLTF } from "@/hooks/three/useLoggedGLTF";
|
||||||
import { useClonedObject } from "@/hooks/three/useClonedObject";
|
import { useClonedObject } from "@/hooks/three/useClonedObject";
|
||||||
@@ -9,6 +10,10 @@ import type { ModelTransformProps, Vector3Tuple } from "@/types/three/three";
|
|||||||
import { logModelLoadError } from "@/utils/three/modelLoadLogger";
|
import { logModelLoadError } from "@/utils/three/modelLoadLogger";
|
||||||
import { toVector3Scale } from "@/utils/three/scale";
|
import { toVector3Scale } from "@/utils/three/scale";
|
||||||
|
|
||||||
|
export type ExplodedNodeAnchors = Readonly<Record<string, Vector3Tuple>>;
|
||||||
|
|
||||||
|
const _anchorWorld = new THREE.Vector3();
|
||||||
|
|
||||||
interface ModelErrorBoundaryProps {
|
interface ModelErrorBoundaryProps {
|
||||||
children: ReactNode;
|
children: ReactNode;
|
||||||
modelPath: string;
|
modelPath: string;
|
||||||
@@ -67,6 +72,9 @@ interface ExplodableModelInnerProps extends ModelTransformProps {
|
|||||||
split: boolean;
|
split: boolean;
|
||||||
splitDistance?: number;
|
splitDistance?: number;
|
||||||
onPartsReady?: (parts: readonly ExplodedPart[]) => void;
|
onPartsReady?: (parts: readonly ExplodedPart[]) => void;
|
||||||
|
hideNodeNames?: readonly string[];
|
||||||
|
nodeAnchorNames?: readonly string[];
|
||||||
|
onNodeAnchorsChange?: (anchors: ExplodedNodeAnchors) => void;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function ExplodableModel(
|
export function ExplodableModel(
|
||||||
@@ -93,6 +101,9 @@ function ExplodableModelInner({
|
|||||||
scale = 1,
|
scale = 1,
|
||||||
splitDistance = 1.2,
|
splitDistance = 1.2,
|
||||||
onPartsReady,
|
onPartsReady,
|
||||||
|
hideNodeNames,
|
||||||
|
nodeAnchorNames,
|
||||||
|
onNodeAnchorsChange,
|
||||||
}: ExplodableModelInnerProps): React.JSX.Element {
|
}: ExplodableModelInnerProps): React.JSX.Element {
|
||||||
const { scene } = useLoggedGLTF(modelPath, {
|
const { scene } = useLoggedGLTF(modelPath, {
|
||||||
scope: "ExplodableModel",
|
scope: "ExplodableModel",
|
||||||
@@ -106,6 +117,24 @@ function ExplodableModelInner({
|
|||||||
[model, splitDistance],
|
[model, splitDistance],
|
||||||
);
|
);
|
||||||
const parsedScale = toVector3Scale(scale);
|
const parsedScale = toVector3Scale(scale);
|
||||||
|
const anchorSignatureRef = useRef("");
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (!hideNodeNames || hideNodeNames.length === 0) return;
|
||||||
|
const hidden: THREE.Object3D[] = [];
|
||||||
|
model.traverse((child) => {
|
||||||
|
if (hideNodeNames.includes(child.name)) {
|
||||||
|
hidden.push(child);
|
||||||
|
child.visible = false;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
return () => {
|
||||||
|
hidden.forEach((object) => {
|
||||||
|
object.visible = true;
|
||||||
|
});
|
||||||
|
};
|
||||||
|
}, [hideNodeNames, model]);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
explodedModel.setSplit(split);
|
explodedModel.setSplit(split);
|
||||||
@@ -117,6 +146,35 @@ function ExplodableModelInner({
|
|||||||
|
|
||||||
useFrame((_, delta) => {
|
useFrame((_, delta) => {
|
||||||
explodedModel.update(delta);
|
explodedModel.update(delta);
|
||||||
|
|
||||||
|
if (
|
||||||
|
!onNodeAnchorsChange ||
|
||||||
|
!nodeAnchorNames ||
|
||||||
|
nodeAnchorNames.length === 0
|
||||||
|
) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const anchors: Record<string, Vector3Tuple> = {};
|
||||||
|
nodeAnchorNames.forEach((name) => {
|
||||||
|
const node = model.getObjectByName(name);
|
||||||
|
if (!node) return;
|
||||||
|
node.getWorldPosition(_anchorWorld);
|
||||||
|
anchors[name] = [_anchorWorld.x, _anchorWorld.y, _anchorWorld.z];
|
||||||
|
});
|
||||||
|
|
||||||
|
const signature = nodeAnchorNames
|
||||||
|
.map((name) => {
|
||||||
|
const a = anchors[name];
|
||||||
|
return a
|
||||||
|
? `${name}:${a[0].toFixed(3)},${a[1].toFixed(3)},${a[2].toFixed(3)}`
|
||||||
|
: `${name}:?`;
|
||||||
|
})
|
||||||
|
.join("|");
|
||||||
|
|
||||||
|
if (signature === anchorSignatureRef.current) return;
|
||||||
|
anchorSignatureRef.current = signature;
|
||||||
|
onNodeAnchorsChange(anchors);
|
||||||
});
|
});
|
||||||
|
|
||||||
return (
|
return (
|
||||||
|
|||||||
@@ -17,10 +17,29 @@ function applyShadowSettings(
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function applyOpacity(object: THREE.Object3D, opacity: number): void {
|
||||||
|
object.traverse((child) => {
|
||||||
|
if (!(child instanceof THREE.Mesh)) return;
|
||||||
|
|
||||||
|
const materials = Array.isArray(child.material)
|
||||||
|
? child.material
|
||||||
|
: [child.material];
|
||||||
|
|
||||||
|
materials.forEach((material) => {
|
||||||
|
if (!(material instanceof THREE.Material)) return;
|
||||||
|
material.transparent = opacity < 1;
|
||||||
|
material.opacity = opacity;
|
||||||
|
material.depthWrite = opacity >= 1;
|
||||||
|
material.needsUpdate = true;
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
interface SimpleModelConfig extends ModelTransformProps {
|
interface SimpleModelConfig extends ModelTransformProps {
|
||||||
modelPath: string;
|
modelPath: string;
|
||||||
castShadow?: boolean;
|
castShadow?: boolean;
|
||||||
receiveShadow?: boolean;
|
receiveShadow?: boolean;
|
||||||
|
opacity?: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
interface SimpleModelProps extends SimpleModelConfig {
|
interface SimpleModelProps extends SimpleModelConfig {
|
||||||
@@ -34,6 +53,7 @@ export function SimpleModel({
|
|||||||
scale = 1,
|
scale = 1,
|
||||||
castShadow = true,
|
castShadow = true,
|
||||||
receiveShadow = true,
|
receiveShadow = true,
|
||||||
|
opacity = 1,
|
||||||
children,
|
children,
|
||||||
}: SimpleModelProps): React.JSX.Element {
|
}: SimpleModelProps): React.JSX.Element {
|
||||||
const { scene } = useLoggedGLTF(modelPath, {
|
const { scene } = useLoggedGLTF(modelPath, {
|
||||||
@@ -48,6 +68,10 @@ export function SimpleModel({
|
|||||||
applyShadowSettings(model, castShadow, receiveShadow);
|
applyShadowSettings(model, castShadow, receiveShadow);
|
||||||
}, [castShadow, model, receiveShadow]);
|
}, [castShadow, model, receiveShadow]);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
applyOpacity(model, opacity);
|
||||||
|
}, [model, opacity]);
|
||||||
|
|
||||||
const parsedScale =
|
const parsedScale =
|
||||||
typeof scale === "number" ? ([scale, scale, scale] as Vector3Tuple) : scale;
|
typeof scale === "number" ? ([scale, scale, scale] as Vector3Tuple) : scale;
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import { Crosshair } from "@/components/ui/Crosshair";
|
import { Crosshair } from "@/components/ui/Crosshair";
|
||||||
import { DebugOverlayLayout } from "@/components/ui/debug/DebugOverlayLayout";
|
import { DebugOverlayLayout } from "@/components/ui/debug/DebugOverlayLayout";
|
||||||
import { GameSettingsMenu } from "@/components/ui/GameSettingsMenu";
|
import { GameSettingsMenu } from "@/components/ui/GameSettingsMenu";
|
||||||
|
import { HandTrackingFallback } from "@/components/ui/HandTrackingFallback";
|
||||||
import { HandTrackingVisualizer } from "@/components/ui/HandTrackingVisualizer";
|
import { HandTrackingVisualizer } from "@/components/ui/HandTrackingVisualizer";
|
||||||
import { InteractPrompt } from "@/components/ui/InteractPrompt";
|
import { InteractPrompt } from "@/components/ui/InteractPrompt";
|
||||||
import { RepairMovementLockIndicator } from "@/components/ui/RepairMovementLockIndicator";
|
import { RepairMovementLockIndicator } from "@/components/ui/RepairMovementLockIndicator";
|
||||||
@@ -15,6 +16,7 @@ export function GameUI(): React.JSX.Element {
|
|||||||
<RepairMovementLockIndicator />
|
<RepairMovementLockIndicator />
|
||||||
<InteractPrompt />
|
<InteractPrompt />
|
||||||
<HandTrackingVisualizer />
|
<HandTrackingVisualizer />
|
||||||
|
<HandTrackingFallback />
|
||||||
<Subtitles />
|
<Subtitles />
|
||||||
<TalkieDialogueOverlay />
|
<TalkieDialogueOverlay />
|
||||||
<GameSettingsMenu />
|
<GameSettingsMenu />
|
||||||
|
|||||||
@@ -0,0 +1,82 @@
|
|||||||
|
import { useHandTrackingSnapshot } from "@/hooks/handTracking/useHandTrackingSnapshot";
|
||||||
|
import {
|
||||||
|
useHandTrackingGloveStatus,
|
||||||
|
type HandTrackingGloveHandedness,
|
||||||
|
} from "@/hooks/handTracking/useHandTrackingGloveStatus";
|
||||||
|
|
||||||
|
// Simple schematic silhouettes used as a last-resort fallback when the
|
||||||
|
// rigged glove model has failed to load. Both icons share the same
|
||||||
|
// 48x48 viewBox and the same stroke/fill rules from the .css.
|
||||||
|
|
||||||
|
const OpenHandShape = (): React.JSX.Element => (
|
||||||
|
<>
|
||||||
|
<ellipse cx="9" cy="30" rx="3" ry="6" transform="rotate(-25 9 30)" />
|
||||||
|
<rect x="14" y="8" width="4" height="22" rx="2" />
|
||||||
|
<rect x="20" y="4" width="4" height="26" rx="2" />
|
||||||
|
<rect x="26" y="6" width="4" height="24" rx="2" />
|
||||||
|
<rect x="32" y="10" width="4" height="20" rx="2" />
|
||||||
|
<rect x="10" y="26" width="28" height="18" rx="6" />
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
|
||||||
|
const FistShape = (): React.JSX.Element => (
|
||||||
|
<>
|
||||||
|
<ellipse cx="8" cy="26" rx="3" ry="5" />
|
||||||
|
<rect x="10" y="14" width="28" height="30" rx="10" />
|
||||||
|
<circle cx="15" cy="14" r="3" />
|
||||||
|
<circle cx="21" cy="13" r="3" />
|
||||||
|
<circle cx="27" cy="13" r="3" />
|
||||||
|
<circle cx="33" cy="14" r="3" />
|
||||||
|
</>
|
||||||
|
);
|
||||||
|
|
||||||
|
function getHandedness(raw: string): HandTrackingGloveHandedness | null {
|
||||||
|
const lower = raw.toLowerCase();
|
||||||
|
if (lower === "left" || lower === "right") return lower;
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function HandTrackingFallback(): React.JSX.Element | null {
|
||||||
|
const { hands } = useHandTrackingSnapshot();
|
||||||
|
const gloveStatus = useHandTrackingGloveStatus((state) => state.gloves);
|
||||||
|
|
||||||
|
const visibleHands = hands.flatMap((hand, index) => {
|
||||||
|
const handedness = getHandedness(hand.handedness);
|
||||||
|
if (!handedness) return [];
|
||||||
|
if (gloveStatus[handedness] !== "error") return [];
|
||||||
|
|
||||||
|
const wrist = hand.landmarks[0];
|
||||||
|
if (!wrist) return [];
|
||||||
|
|
||||||
|
return [{ hand, handedness, wrist, index }];
|
||||||
|
});
|
||||||
|
|
||||||
|
if (visibleHands.length === 0) return null;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="hand-tracking-fallback" aria-hidden="true">
|
||||||
|
{visibleHands.map(({ hand, handedness, wrist, index }) => {
|
||||||
|
// MediaPipe coords are mirrored (selfie cam), keep the same
|
||||||
|
// mapping the SVG visualizer uses.
|
||||||
|
const leftPercent = (1 - wrist.x) * 100;
|
||||||
|
const topPercent = wrist.y * 100;
|
||||||
|
const flipX = handedness === "right" ? -1 : 1;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<svg
|
||||||
|
key={`${handedness}-${index}`}
|
||||||
|
className="hand-tracking-fallback__icon"
|
||||||
|
viewBox="0 0 48 48"
|
||||||
|
style={{
|
||||||
|
left: `${leftPercent}%`,
|
||||||
|
top: `${topPercent}%`,
|
||||||
|
transform: `translate(-50%, -50%) scaleX(${flipX})`,
|
||||||
|
}}
|
||||||
|
>
|
||||||
|
{hand.isFist ? <FistShape /> : <OpenHandShape />}
|
||||||
|
</svg>
|
||||||
|
);
|
||||||
|
})}
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -26,6 +26,12 @@ const HAND_CONNECTIONS: Array<[number, number]> = [
|
|||||||
[0, 17],
|
[0, 17],
|
||||||
];
|
];
|
||||||
|
|
||||||
|
const LANDMARK_FILL = "#67e8f9"; // cyan-300, opaque interior
|
||||||
|
const LANDMARK_STROKE = "#0c4a6e"; // sky-900, dark blue outline
|
||||||
|
const LANDMARK_STROKE_FIST = "#1e3a8a"; // blue-900, thicker accent when fist
|
||||||
|
const CONNECTION_STROKE = "#ffffff"; // white bones
|
||||||
|
const INDEX_TIP_LANDMARK = 8;
|
||||||
|
|
||||||
export function HandTrackingVisualizer(): React.JSX.Element | null {
|
export function HandTrackingVisualizer(): React.JSX.Element | null {
|
||||||
const { hands, status } = useHandTrackingSnapshot();
|
const { hands, status } = useHandTrackingSnapshot();
|
||||||
const showHandTrackingSvg = useDebugStore((debug) =>
|
const showHandTrackingSvg = useDebugStore((debug) =>
|
||||||
@@ -50,7 +56,9 @@ export function HandTrackingVisualizer(): React.JSX.Element | null {
|
|||||||
const landmarks = hand.landmarks;
|
const landmarks = hand.landmarks;
|
||||||
if (landmarks.length === 0) return null;
|
if (landmarks.length === 0) return null;
|
||||||
|
|
||||||
const color = hand.isFist ? "#facc15" : "#38bdf8";
|
const landmarkStroke = hand.isFist
|
||||||
|
? LANDMARK_STROKE_FIST
|
||||||
|
: LANDMARK_STROKE;
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<g key={`${hand.handedness}-${handIndex}`}>
|
<g key={`${hand.handedness}-${handIndex}`}>
|
||||||
@@ -66,8 +74,8 @@ export function HandTrackingVisualizer(): React.JSX.Element | null {
|
|||||||
y1={`${fromPoint.y * 100}%`}
|
y1={`${fromPoint.y * 100}%`}
|
||||||
x2={`${(1 - toPoint.x) * 100}%`}
|
x2={`${(1 - toPoint.x) * 100}%`}
|
||||||
y2={`${toPoint.y * 100}%`}
|
y2={`${toPoint.y * 100}%`}
|
||||||
stroke={color}
|
stroke={CONNECTION_STROKE}
|
||||||
strokeWidth="2"
|
strokeWidth="2.5"
|
||||||
strokeLinecap="round"
|
strokeLinecap="round"
|
||||||
/>
|
/>
|
||||||
);
|
);
|
||||||
@@ -78,8 +86,10 @@ export function HandTrackingVisualizer(): React.JSX.Element | null {
|
|||||||
key={landmarkIndex}
|
key={landmarkIndex}
|
||||||
cx={`${(1 - landmark.x) * 100}%`}
|
cx={`${(1 - landmark.x) * 100}%`}
|
||||||
cy={`${landmark.y * 100}%`}
|
cy={`${landmark.y * 100}%`}
|
||||||
r={landmarkIndex === 8 ? 5 : 3}
|
r={landmarkIndex === INDEX_TIP_LANDMARK ? 6 : 4}
|
||||||
fill={landmarkIndex === 8 ? "#ffffff" : color}
|
fill={LANDMARK_FILL}
|
||||||
|
stroke={landmarkStroke}
|
||||||
|
strokeWidth={hand.isFist ? 2.5 : 2}
|
||||||
/>
|
/>
|
||||||
))}
|
))}
|
||||||
</g>
|
</g>
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ export interface CameraTransform {
|
|||||||
}
|
}
|
||||||
|
|
||||||
export const EBIKE_CAMERA_TRANSFORM: CameraTransform = {
|
export const EBIKE_CAMERA_TRANSFORM: CameraTransform = {
|
||||||
position: [-2.6, 4.5, 0],
|
position: [-1, 1, 0],
|
||||||
rotation: [-10, -90, 0],
|
rotation: [-10, -90, 0],
|
||||||
};
|
};
|
||||||
|
|
||||||
|
|||||||
@@ -135,7 +135,7 @@ export const galleryModels: GalleryModel[] = [
|
|||||||
},
|
},
|
||||||
{ id: "potager", name: "Potager", path: "/models/potager/potager.gltf" },
|
{ id: "potager", name: "Potager", path: "/models/potager/potager.gltf" },
|
||||||
{ id: "puce", name: "Puce", path: "/models/puce/model.gltf" },
|
{ id: "puce", name: "Puce", path: "/models/puce/model.gltf" },
|
||||||
{ id: "pylone", name: "Pylône", path: "/models/pylone/model.gltf" },
|
{ id: "pylone", name: "Pylône", path: "/models/pylone/model.glb" },
|
||||||
{
|
{
|
||||||
id: "refroidisseur",
|
id: "refroidisseur",
|
||||||
name: "Refroidisseur",
|
name: "Refroidisseur",
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ export const REPAIR_CASE_MODEL_PATH = "/models/packderelance/model.gltf";
|
|||||||
export const REPAIR_CASE_OPEN_SOUND_PATH = "/sounds/effect/open-malette.mp3";
|
export const REPAIR_CASE_OPEN_SOUND_PATH = "/sounds/effect/open-malette.mp3";
|
||||||
export const REPAIR_CASE_CLOSE_SOUND_PATH = "/sounds/effect/close-malette.mp3";
|
export const REPAIR_CASE_CLOSE_SOUND_PATH = "/sounds/effect/close-malette.mp3";
|
||||||
|
|
||||||
export const REPAIR_CASE_LID_NODE_NAME = "partiesup";
|
export const REPAIR_CASE_LID_NODE_NAME = "partsup";
|
||||||
export const REPAIR_CASE_CLOSED_ROTATION_OFFSET_DEGREES = 0;
|
export const REPAIR_CASE_CLOSED_ROTATION_OFFSET_DEGREES = 0;
|
||||||
export const REPAIR_CASE_OPEN_ROTATION_OFFSET_DEGREES = 115;
|
export const REPAIR_CASE_OPEN_ROTATION_OFFSET_DEGREES = 115;
|
||||||
export const REPAIR_CASE_ANIMATION_DURATION = 0.8;
|
export const REPAIR_CASE_ANIMATION_DURATION = 0.8;
|
||||||
@@ -27,3 +27,50 @@ export const REPAIR_CASE_FOCUS_SCALE = 2.25;
|
|||||||
export const REPAIR_CASE_PLACEHOLDER_NAME_PREFIX = "placeholder_";
|
export const REPAIR_CASE_PLACEHOLDER_NAME_PREFIX = "placeholder_";
|
||||||
export const REPAIR_CASE_PLACEHOLDER_SNAP_RADIUS = 0.65;
|
export const REPAIR_CASE_PLACEHOLDER_SNAP_RADIUS = 0.65;
|
||||||
export const REPAIR_CASE_PLACEHOLDER_SNAP_DURATION = 0.25;
|
export const REPAIR_CASE_PLACEHOLDER_SNAP_DURATION = 0.25;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Names of nodes inside the packderelance GLTF where standalone part models
|
||||||
|
* are anchored (visually injected). The original meshes under these nodes are
|
||||||
|
* hidden at runtime so the standalone model takes their place.
|
||||||
|
*
|
||||||
|
* Some entries (e.g. `refroidisseur`) do not exist as nodes in the GLTF; an
|
||||||
|
* empty Object3D is created at mount time at the corresponding case-local
|
||||||
|
* fallback position so the anchoring pipeline is uniform.
|
||||||
|
*/
|
||||||
|
export const REPAIR_CASE_PART_ANCHOR_NAMES = [
|
||||||
|
"cabledroit",
|
||||||
|
"cablegauche",
|
||||||
|
"pucehaut",
|
||||||
|
"pucebas",
|
||||||
|
"refroidisseur",
|
||||||
|
] as const;
|
||||||
|
|
||||||
|
export type RepairCasePartAnchorName =
|
||||||
|
(typeof REPAIR_CASE_PART_ANCHOR_NAMES)[number];
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Case-local positions used when an anchor node is missing from the GLTF.
|
||||||
|
* Values are expressed in the case model's local coordinate system (the case
|
||||||
|
* is rendered at small intrinsic scale; magnitudes are in the 0.01-0.25 range
|
||||||
|
* to match the existing nodes such as `cabledroit`).
|
||||||
|
*/
|
||||||
|
export const REPAIR_CASE_PART_ANCHOR_FALLBACKS: Record<
|
||||||
|
RepairCasePartAnchorName,
|
||||||
|
Vector3Tuple
|
||||||
|
> = {
|
||||||
|
cabledroit: [0.0087, 0.0139, 0.1921],
|
||||||
|
cablegauche: [0.0087, 0.0139, 0.2477],
|
||||||
|
pucehaut: [-0.0207, 0.009, -0.0479],
|
||||||
|
pucebas: [0.0987, 0.009, -0.0479],
|
||||||
|
refroidisseur: [0.05, 0.014, 0.05],
|
||||||
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Quaternion applied to anchor nodes that are created at runtime (because
|
||||||
|
* the corresponding node is absent from the GLTF). Matches the rotation of
|
||||||
|
* the existing part nodes in packderelance to keep visual orientation
|
||||||
|
* consistent.
|
||||||
|
*/
|
||||||
|
export const REPAIR_CASE_PART_ANCHOR_FALLBACK_QUATERNION = [
|
||||||
|
0.7071068286895752, 0, 0, 0.7071068286895752,
|
||||||
|
] as const satisfies readonly [number, number, number, number];
|
||||||
|
|||||||
@@ -25,26 +25,48 @@ export const REPAIR_MISSIONS: Record<RepairMissionId, RepairMissionConfig> = {
|
|||||||
interactUiPath: REPAIR_INTERACT_UI_PATH,
|
interactUiPath: REPAIR_INTERACT_UI_PATH,
|
||||||
brokenUiPath: REPAIR_BROKEN_UI_PATH,
|
brokenUiPath: REPAIR_BROKEN_UI_PATH,
|
||||||
case: DEFAULT_REPAIR_CASE,
|
case: DEFAULT_REPAIR_CASE,
|
||||||
requiredReplacementPartId: "ebike-cooling-core-replacement",
|
requiredReplacementPartIds: ["ebike-cooling-core-replacement"],
|
||||||
brokenParts: [
|
brokenParts: [
|
||||||
{
|
{
|
||||||
id: "ebike-cooling-core",
|
id: "ebike-cooling-core",
|
||||||
label: "Cooling core",
|
label: "Cooling core",
|
||||||
modelPath: "/models/refroidisseur/model.gltf",
|
modelPath: "/models/refroidisseur/model.gltf",
|
||||||
nodeName: "refroidisseur",
|
nodeName: "refroidisseur",
|
||||||
|
targetNodeName: "refroidisseur",
|
||||||
caseSlotName: "placeholder_1",
|
caseSlotName: "placeholder_1",
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
replacementParts: [
|
replacementParts: [
|
||||||
{
|
{
|
||||||
id: "ebike-cooling-core-replacement",
|
id: "ebike-cooling-core-replacement",
|
||||||
label: "Replacement cooling core",
|
label: "Refroidisseur",
|
||||||
modelPath: "/models/refroidisseur/model.gltf",
|
modelPath: "/models/refroidisseur/model.gltf",
|
||||||
|
caseAnchor: "refroidisseur",
|
||||||
|
targetNodeName: "refroidisseur",
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: "ebike-glove-distractor",
|
id: "ebike-cable-right-distractor",
|
||||||
label: "Insulation glove",
|
label: "Câble droit",
|
||||||
modelPath: "/models/gant_l/model.gltf",
|
modelPath: "/models/cable1/model.gltf",
|
||||||
|
caseAnchor: "cabledroit",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "ebike-cable-left-distractor",
|
||||||
|
label: "Câble gauche",
|
||||||
|
modelPath: "/models/cable2/model.gltf",
|
||||||
|
caseAnchor: "cablegauche",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "ebike-puce-haut-distractor",
|
||||||
|
label: "Puce haute",
|
||||||
|
modelPath: "/models/puce/model.gltf",
|
||||||
|
caseAnchor: "pucehaut",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "ebike-puce-bas-distractor",
|
||||||
|
label: "Puce basse",
|
||||||
|
modelPath: "/models/puce/model.gltf",
|
||||||
|
caseAnchor: "pucebas",
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
@@ -53,43 +75,52 @@ export const REPAIR_MISSIONS: Record<RepairMissionId, RepairMissionConfig> = {
|
|||||||
label: "Power pylon",
|
label: "Power pylon",
|
||||||
description:
|
description:
|
||||||
"Restore the pylon lamp relay and damaged panel before reconnecting the grid",
|
"Restore the pylon lamp relay and damaged panel before reconnecting the grid",
|
||||||
modelPath: "/models/pylone/model.gltf",
|
modelPath: "/models/pylone/model.glb",
|
||||||
stageUiPath: "/assets/world/UI/pylon-mission-notification.webm",
|
stageUiPath: "/assets/world/UI/pylon-mission-notification.webm",
|
||||||
interactUiPath: REPAIR_INTERACT_UI_PATH,
|
interactUiPath: REPAIR_INTERACT_UI_PATH,
|
||||||
brokenUiPath: REPAIR_BROKEN_UI_PATH,
|
brokenUiPath: REPAIR_BROKEN_UI_PATH,
|
||||||
case: DEFAULT_REPAIR_CASE,
|
case: DEFAULT_REPAIR_CASE,
|
||||||
reassemblySeconds: 1.8,
|
reassemblySeconds: 1.8,
|
||||||
requiredReplacementPartId: "pylon-grid-relay-replacement",
|
requiredReplacementPartIds: [
|
||||||
scanPartSeconds: 1.4,
|
"pylon-cable-right-replacement",
|
||||||
brokenParts: [
|
"pylon-cable-left-replacement",
|
||||||
{
|
|
||||||
id: "pylon-grid-relay",
|
|
||||||
label: "Grid relay",
|
|
||||||
nodeName: "lampe",
|
|
||||||
caseSlotName: "placeholder_1",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
id: "pylon-damaged-panel",
|
|
||||||
label: "Damaged solar panel",
|
|
||||||
nodeName: "panneau2",
|
|
||||||
caseSlotName: "placeholder_2",
|
|
||||||
},
|
|
||||||
],
|
],
|
||||||
|
scanPartSeconds: 1.4,
|
||||||
|
brokenParts: [],
|
||||||
replacementParts: [
|
replacementParts: [
|
||||||
{
|
{
|
||||||
id: "pylon-grid-relay-replacement",
|
id: "pylon-cable-right-replacement",
|
||||||
label: "Replacement grid relay",
|
label: "Câble droit",
|
||||||
modelPath: "/models/pylone/model.gltf",
|
modelPath: "/models/cable1/model.gltf",
|
||||||
|
caseAnchor: "cabledroit",
|
||||||
|
caseLockGroup: "pylon-cable",
|
||||||
|
targetNodeName: "cable2",
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: "pylon-stone-distractor",
|
id: "pylon-cable-left-replacement",
|
||||||
label: "Stone counterweight",
|
label: "Câble gauche",
|
||||||
modelPath: "/models/galet/model.gltf",
|
modelPath: "/models/cable2/model.gltf",
|
||||||
|
caseAnchor: "cablegauche",
|
||||||
|
caseLockGroup: "pylon-cable",
|
||||||
|
targetNodeName: "cable2",
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: "pylon-cooling-distractor",
|
id: "pylon-cooling-distractor",
|
||||||
label: "Cooling core",
|
label: "Refroidisseur",
|
||||||
modelPath: "/models/refroidisseur/model.gltf",
|
modelPath: "/models/refroidisseur/model.gltf",
|
||||||
|
caseAnchor: "refroidisseur",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "pylon-puce-haut-distractor",
|
||||||
|
label: "Puce haute",
|
||||||
|
modelPath: "/models/puce/model.gltf",
|
||||||
|
caseAnchor: "pucehaut",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
id: "pylon-puce-bas-distractor",
|
||||||
|
label: "Puce basse",
|
||||||
|
modelPath: "/models/puce/model.gltf",
|
||||||
|
caseAnchor: "pucebas",
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
},
|
},
|
||||||
@@ -104,7 +135,7 @@ export const REPAIR_MISSIONS: Record<RepairMissionId, RepairMissionConfig> = {
|
|||||||
brokenUiPath: REPAIR_BROKEN_UI_PATH,
|
brokenUiPath: REPAIR_BROKEN_UI_PATH,
|
||||||
case: DEFAULT_REPAIR_CASE,
|
case: DEFAULT_REPAIR_CASE,
|
||||||
reassemblySeconds: 1.2,
|
reassemblySeconds: 1.2,
|
||||||
requiredReplacementPartId: "farm-irrigation-pump-replacement",
|
requiredReplacementPartIds: ["farm-irrigation-pump-replacement"],
|
||||||
scanPartSeconds: 0.9,
|
scanPartSeconds: 0.9,
|
||||||
brokenParts: [
|
brokenParts: [
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -1,5 +1,11 @@
|
|||||||
export const HAND_TRACKING_FRAME_WIDTH = 320;
|
export const HAND_TRACKING_FRAME_WIDTH = 320;
|
||||||
export const HAND_TRACKING_FRAME_HEIGHT = 240;
|
export const HAND_TRACKING_FRAME_HEIGHT = 240;
|
||||||
|
// The browser MediaPipe model (hand_landmarker.task float16) is more
|
||||||
|
// sensitive than the backend Python model and needs a higher-resolution
|
||||||
|
// frame to detect hands reliably. The backend keeps 320x240 because that
|
||||||
|
// is the JPEG payload size sent over the WebSocket.
|
||||||
|
export const HAND_TRACKING_BROWSER_CAMERA_WIDTH = 640;
|
||||||
|
export const HAND_TRACKING_BROWSER_CAMERA_HEIGHT = 480;
|
||||||
export const HAND_TRACKING_TARGET_FPS = 10;
|
export const HAND_TRACKING_TARGET_FPS = 10;
|
||||||
export const HAND_TRACKING_JPEG_QUALITY = 0.55;
|
export const HAND_TRACKING_JPEG_QUALITY = 0.55;
|
||||||
export const HAND_TRACKING_CAMERA_TIMEOUT_MS = 8_000;
|
export const HAND_TRACKING_CAMERA_TIMEOUT_MS = 8_000;
|
||||||
@@ -8,9 +14,21 @@ export const HAND_TRACKING_BROWSER_WASM_URL =
|
|||||||
"https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@0.10.35/wasm";
|
"https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@0.10.35/wasm";
|
||||||
export const HAND_TRACKING_BROWSER_MODEL_URL =
|
export const HAND_TRACKING_BROWSER_MODEL_URL =
|
||||||
"https://storage.googleapis.com/mediapipe-models/hand_landmarker/hand_landmarker/float16/1/hand_landmarker.task";
|
"https://storage.googleapis.com/mediapipe-models/hand_landmarker/hand_landmarker/float16/1/hand_landmarker.task";
|
||||||
export const HAND_TRACKING_BROWSER_DELEGATE: "CPU" | "GPU" = "CPU";
|
export const HAND_TRACKING_BROWSER_DELEGATE: "CPU" | "GPU" = "GPU";
|
||||||
|
|
||||||
// Delay before the runtime actually starts after `enabled` flips to true.
|
// Delay before the runtime actually starts after `enabled` flips to true.
|
||||||
// Absorbs React StrictMode's mount/unmount/mount cycle in dev and rapid
|
// Absorbs React StrictMode's mount/unmount/mount cycle in dev and rapid
|
||||||
// `nearby` toggles at trigger borders. Invisible to the user (~5 frames).
|
// `nearby` toggles at trigger borders. Invisible to the user (~5 frames).
|
||||||
export const HAND_TRACKING_RUNTIME_START_DELAY_MS = 80;
|
export const HAND_TRACKING_RUNTIME_START_DELAY_MS = 80;
|
||||||
|
|
||||||
|
// How long the hand tracking stays active after the trigger condition
|
||||||
|
// (nearby / holding / repair step) turns off. Gives MediaPipe enough time
|
||||||
|
// to initialize webcam + model + first frame inference before we cleanup,
|
||||||
|
// so the user actually sees their hands when entering a zone briefly.
|
||||||
|
export const HAND_TRACKING_LINGER_MS = 2000;
|
||||||
|
|
||||||
|
// EMA weight applied to the latest landmark frame. Lower = smoother but
|
||||||
|
// laggier; higher = more responsive but more jitter from raw MediaPipe
|
||||||
|
// noise. 0.4 keeps the glove and grabbed objects from trembling without
|
||||||
|
// feeling sluggish.
|
||||||
|
export const HAND_TRACKING_LANDMARK_SMOOTHING = 0.4;
|
||||||
|
|||||||
@@ -9,7 +9,7 @@ export const MAP_INSTANCING_ASSETS = {
|
|||||||
},
|
},
|
||||||
pylone: {
|
pylone: {
|
||||||
mapName: "pylone",
|
mapName: "pylone",
|
||||||
modelPath: "/models/pylone/model.gltf",
|
modelPath: "/models/pylone/model.glb",
|
||||||
scaleMultiplier: 1,
|
scaleMultiplier: 1,
|
||||||
castShadow: true,
|
castShadow: true,
|
||||||
receiveShadow: true,
|
receiveShadow: true,
|
||||||
|
|||||||
@@ -1,7 +1,8 @@
|
|||||||
import { useEffect, useRef, useState } from "react";
|
import { useEffect, useRef, useState } from "react";
|
||||||
import {
|
import {
|
||||||
HAND_TRACKING_FRAME_HEIGHT,
|
HAND_TRACKING_BROWSER_CAMERA_HEIGHT,
|
||||||
HAND_TRACKING_FRAME_WIDTH,
|
HAND_TRACKING_BROWSER_CAMERA_WIDTH,
|
||||||
|
HAND_TRACKING_LANDMARK_SMOOTHING,
|
||||||
HAND_TRACKING_RUNTIME_START_DELAY_MS,
|
HAND_TRACKING_RUNTIME_START_DELAY_MS,
|
||||||
HAND_TRACKING_TARGET_FPS,
|
HAND_TRACKING_TARGET_FPS,
|
||||||
} from "@/data/handTrackingConfig";
|
} from "@/data/handTrackingConfig";
|
||||||
@@ -10,11 +11,15 @@ import {
|
|||||||
getBrowserHandLandmarker,
|
getBrowserHandLandmarker,
|
||||||
releaseBrowserHandLandmarker,
|
releaseBrowserHandLandmarker,
|
||||||
} from "@/lib/handTracking/browserHandTracking";
|
} from "@/lib/handTracking/browserHandTracking";
|
||||||
|
import { smoothHands } from "@/lib/handTracking/handSmoothing";
|
||||||
import {
|
import {
|
||||||
INITIAL_HAND_TRACKING_SNAPSHOT,
|
INITIAL_HAND_TRACKING_SNAPSHOT,
|
||||||
getCameraStreamWithTimeout,
|
getCameraStreamWithTimeout,
|
||||||
} from "@/lib/handTracking/handTrackingSession";
|
} from "@/lib/handTracking/handTrackingSession";
|
||||||
import type { HandTrackingSnapshot } from "@/types/handTracking/handTracking";
|
import type {
|
||||||
|
HandTrackingHand,
|
||||||
|
HandTrackingSnapshot,
|
||||||
|
} from "@/types/handTracking/handTracking";
|
||||||
import { logger } from "@/utils/core/Logger";
|
import { logger } from "@/utils/core/Logger";
|
||||||
|
|
||||||
interface UseBrowserHandTrackingOptions {
|
interface UseBrowserHandTrackingOptions {
|
||||||
@@ -30,6 +35,7 @@ export function useBrowserHandTracking({
|
|||||||
const videoRef = useRef<HTMLVideoElement | null>(null);
|
const videoRef = useRef<HTMLVideoElement | null>(null);
|
||||||
const streamRef = useRef<MediaStream | null>(null);
|
const streamRef = useRef<MediaStream | null>(null);
|
||||||
const intervalRef = useRef<number | null>(null);
|
const intervalRef = useRef<number | null>(null);
|
||||||
|
const previousHandsRef = useRef<HandTrackingHand[]>([]);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
if (!enabled) {
|
if (!enabled) {
|
||||||
@@ -51,6 +57,7 @@ export function useBrowserHandTracking({
|
|||||||
streamRef.current?.getTracks().forEach((track) => track.stop());
|
streamRef.current?.getTracks().forEach((track) => track.stop());
|
||||||
streamRef.current = null;
|
streamRef.current = null;
|
||||||
videoRef.current = null;
|
videoRef.current = null;
|
||||||
|
previousHandsRef.current = [];
|
||||||
releaseBrowserHandLandmarker();
|
releaseBrowserHandLandmarker();
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -66,8 +73,8 @@ export function useBrowserHandTracking({
|
|||||||
try {
|
try {
|
||||||
const stream = await getCameraStreamWithTimeout({
|
const stream = await getCameraStreamWithTimeout({
|
||||||
video: {
|
video: {
|
||||||
width: HAND_TRACKING_FRAME_WIDTH,
|
width: HAND_TRACKING_BROWSER_CAMERA_WIDTH,
|
||||||
height: HAND_TRACKING_FRAME_HEIGHT,
|
height: HAND_TRACKING_BROWSER_CAMERA_HEIGHT,
|
||||||
facingMode: "user",
|
facingMode: "user",
|
||||||
},
|
},
|
||||||
audio: false,
|
audio: false,
|
||||||
@@ -124,7 +131,13 @@ export function useBrowserHandTracking({
|
|||||||
video,
|
video,
|
||||||
performance.now(),
|
performance.now(),
|
||||||
);
|
);
|
||||||
const hands = convertBrowserHandResult(result);
|
const rawHands = convertBrowserHandResult(result);
|
||||||
|
const hands = smoothHands(
|
||||||
|
previousHandsRef.current,
|
||||||
|
rawHands,
|
||||||
|
HAND_TRACKING_LANDMARK_SMOOTHING,
|
||||||
|
);
|
||||||
|
previousHandsRef.current = hands;
|
||||||
|
|
||||||
setSnapshot((current) => ({
|
setSnapshot((current) => ({
|
||||||
...current,
|
...current,
|
||||||
|
|||||||
@@ -3,10 +3,12 @@ import {
|
|||||||
HAND_TRACKING_FRAME_HEIGHT,
|
HAND_TRACKING_FRAME_HEIGHT,
|
||||||
HAND_TRACKING_FRAME_WIDTH,
|
HAND_TRACKING_FRAME_WIDTH,
|
||||||
HAND_TRACKING_JPEG_QUALITY,
|
HAND_TRACKING_JPEG_QUALITY,
|
||||||
|
HAND_TRACKING_LANDMARK_SMOOTHING,
|
||||||
HAND_TRACKING_RESPONSE_TIMEOUT_MS,
|
HAND_TRACKING_RESPONSE_TIMEOUT_MS,
|
||||||
HAND_TRACKING_RUNTIME_START_DELAY_MS,
|
HAND_TRACKING_RUNTIME_START_DELAY_MS,
|
||||||
HAND_TRACKING_TARGET_FPS,
|
HAND_TRACKING_TARGET_FPS,
|
||||||
} from "@/data/handTrackingConfig";
|
} from "@/data/handTrackingConfig";
|
||||||
|
import { smoothHands } from "@/lib/handTracking/handSmoothing";
|
||||||
import { getHandTrackingWsUrl } from "@/utils/handTracking/handTrackingEndpoint";
|
import { getHandTrackingWsUrl } from "@/utils/handTracking/handTrackingEndpoint";
|
||||||
import {
|
import {
|
||||||
INITIAL_HAND_TRACKING_SNAPSHOT,
|
INITIAL_HAND_TRACKING_SNAPSHOT,
|
||||||
@@ -95,6 +97,7 @@ export function useRemoteHandTracking({
|
|||||||
const sendIntervalRef = useRef<number | null>(null);
|
const sendIntervalRef = useRef<number | null>(null);
|
||||||
const responseTimeoutRef = useRef<number | null>(null);
|
const responseTimeoutRef = useRef<number | null>(null);
|
||||||
const waitingForResponseRef = useRef(false);
|
const waitingForResponseRef = useRef(false);
|
||||||
|
const previousHandsRef = useRef<HandTrackingHand[]>([]);
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
if (!enabled) {
|
if (!enabled) {
|
||||||
@@ -128,6 +131,7 @@ export function useRemoteHandTracking({
|
|||||||
streamRef.current = null;
|
streamRef.current = null;
|
||||||
videoRef.current = null;
|
videoRef.current = null;
|
||||||
canvasRef.current = null;
|
canvasRef.current = null;
|
||||||
|
previousHandsRef.current = [];
|
||||||
};
|
};
|
||||||
|
|
||||||
const markResponseReceived = (): void => {
|
const markResponseReceived = (): void => {
|
||||||
@@ -259,10 +263,16 @@ export function useRemoteHandTracking({
|
|||||||
}
|
}
|
||||||
|
|
||||||
if (data.type === "hands") {
|
if (data.type === "hands") {
|
||||||
|
const smoothedHands = smoothHands(
|
||||||
|
previousHandsRef.current,
|
||||||
|
data.hands,
|
||||||
|
HAND_TRACKING_LANDMARK_SMOOTHING,
|
||||||
|
);
|
||||||
|
previousHandsRef.current = smoothedHands;
|
||||||
setSnapshot((current) => ({
|
setSnapshot((current) => ({
|
||||||
...current,
|
...current,
|
||||||
hands: data.hands,
|
hands: smoothedHands,
|
||||||
usageStatus: data.hands.some((hand) => hand.isFist)
|
usageStatus: smoothedHands.some((hand) => hand.isFist)
|
||||||
? "active"
|
? "active"
|
||||||
: "available",
|
: "available",
|
||||||
serverStatus: null,
|
serverStatus: null,
|
||||||
|
|||||||
@@ -1789,6 +1789,26 @@ canvas {
|
|||||||
filter: drop-shadow(0 0 8px rgba(56, 189, 248, 0.55));
|
filter: drop-shadow(0 0 8px rgba(56, 189, 248, 0.55));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.hand-tracking-fallback {
|
||||||
|
position: fixed;
|
||||||
|
inset: 0;
|
||||||
|
z-index: 14;
|
||||||
|
width: 100vw;
|
||||||
|
height: 100vh;
|
||||||
|
pointer-events: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
.hand-tracking-fallback__icon {
|
||||||
|
position: absolute;
|
||||||
|
width: 96px;
|
||||||
|
height: 96px;
|
||||||
|
fill: #67e8f9;
|
||||||
|
stroke: #0c4a6e;
|
||||||
|
stroke-width: 2;
|
||||||
|
stroke-linejoin: round;
|
||||||
|
filter: drop-shadow(0 0 8px rgba(56, 189, 248, 0.55));
|
||||||
|
}
|
||||||
|
|
||||||
/* Zustand game state debug UI */
|
/* Zustand game state debug UI */
|
||||||
.game-state-debug-panel {
|
.game-state-debug-panel {
|
||||||
display: grid;
|
display: grid;
|
||||||
|
|||||||
@@ -0,0 +1,64 @@
|
|||||||
|
import type {
|
||||||
|
HandTrackingHand,
|
||||||
|
HandTrackingLandmark,
|
||||||
|
} from "@/types/handTracking/handTracking";
|
||||||
|
|
||||||
|
function lerp(previous: number, next: number, factor: number): number {
|
||||||
|
return previous * (1 - factor) + next * factor;
|
||||||
|
}
|
||||||
|
|
||||||
|
function smoothLandmark(
|
||||||
|
previous: HandTrackingLandmark,
|
||||||
|
next: HandTrackingLandmark,
|
||||||
|
factor: number,
|
||||||
|
): HandTrackingLandmark {
|
||||||
|
return {
|
||||||
|
x: lerp(previous.x, next.x, factor),
|
||||||
|
y: lerp(previous.y, next.y, factor),
|
||||||
|
z: lerp(previous.z, next.z, factor),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
function smoothHand(
|
||||||
|
previous: HandTrackingHand,
|
||||||
|
next: HandTrackingHand,
|
||||||
|
factor: number,
|
||||||
|
): HandTrackingHand {
|
||||||
|
return {
|
||||||
|
...next,
|
||||||
|
x: lerp(previous.x, next.x, factor),
|
||||||
|
y: lerp(previous.y, next.y, factor),
|
||||||
|
z: lerp(previous.z, next.z, factor),
|
||||||
|
landmarks: next.landmarks.map((landmark, index) => {
|
||||||
|
const previousLandmark = previous.landmarks[index];
|
||||||
|
if (!previousLandmark) return landmark;
|
||||||
|
return smoothLandmark(previousLandmark, landmark, factor);
|
||||||
|
}),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Apply an exponential moving average to the landmark positions of each
|
||||||
|
* detected hand. MediaPipe lands per-frame positions with noticeable
|
||||||
|
* jitter (especially at ~10fps), and feeding those raw values into the
|
||||||
|
* scene makes both the glove rig and any grabbed object tremble.
|
||||||
|
*
|
||||||
|
* `factor` is the weight given to the latest sample (0 = previous frame
|
||||||
|
* only, 1 = no smoothing). Hands are matched between frames by
|
||||||
|
* handedness so left/right don't bleed into each other.
|
||||||
|
*/
|
||||||
|
export function smoothHands(
|
||||||
|
previousHands: HandTrackingHand[],
|
||||||
|
nextHands: HandTrackingHand[],
|
||||||
|
factor: number,
|
||||||
|
): HandTrackingHand[] {
|
||||||
|
if (factor >= 1) return nextHands;
|
||||||
|
|
||||||
|
return nextHands.map((next) => {
|
||||||
|
const previous = previousHands.find(
|
||||||
|
(candidate) => candidate.handedness === next.handedness,
|
||||||
|
);
|
||||||
|
if (!previous) return next;
|
||||||
|
return smoothHand(previous, next, factor);
|
||||||
|
});
|
||||||
|
}
|
||||||
@@ -1,4 +1,6 @@
|
|||||||
import type { ReactNode } from "react";
|
import type { ReactNode } from "react";
|
||||||
|
import { useEffect, useState } from "react";
|
||||||
|
import { HAND_TRACKING_LINGER_MS } from "@/data/handTrackingConfig";
|
||||||
import { useSceneMode } from "@/hooks/debug/useSceneMode";
|
import { useSceneMode } from "@/hooks/debug/useSceneMode";
|
||||||
import { useDebugStore } from "@/hooks/debug/useDebugStore";
|
import { useDebugStore } from "@/hooks/debug/useDebugStore";
|
||||||
import { useInteraction } from "@/hooks/interaction/useInteraction";
|
import { useInteraction } from "@/hooks/interaction/useInteraction";
|
||||||
@@ -38,37 +40,74 @@ export function HandTrackingProvider({
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
const { nearby, holding, handHolding } = useInteraction();
|
const { nearby, holding, handHolding } = useInteraction();
|
||||||
const enabled =
|
const requested =
|
||||||
repairNeedsHands ||
|
repairNeedsHands ||
|
||||||
(sceneMode === "physics" && (nearby || holding || handHolding));
|
(sceneMode === "physics" && (nearby || holding || handHolding));
|
||||||
|
|
||||||
if (!enabled) {
|
// Keep the runtime active a little after `requested` turns off so
|
||||||
return (
|
// MediaPipe has time to initialize the webcam + model + first frame
|
||||||
<HandTrackingContext value={HAND_TRACKING_IDLE_SNAPSHOT}>
|
// before being torn down. Without this, a quick walk-through of a
|
||||||
{children}
|
// trigger zone never produces a detected hand and the user sees
|
||||||
</HandTrackingContext>
|
// nothing.
|
||||||
);
|
const enabled = useLingeredFlag(requested, HAND_TRACKING_LINGER_MS);
|
||||||
}
|
|
||||||
|
|
||||||
return <ActiveHandTrackingProvider>{children}</ActiveHandTrackingProvider>;
|
// Always render the same JSX root (HandTrackingRuntime). Returning
|
||||||
|
// different element types from this provider would force React to
|
||||||
|
// remount its entire subtree — including the <Canvas> below — every
|
||||||
|
// time `enabled` toggles, which destroys the WebGL context.
|
||||||
|
return (
|
||||||
|
<HandTrackingRuntime enabled={enabled}>{children}</HandTrackingRuntime>
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
function ActiveHandTrackingProvider({
|
function useLingeredFlag(value: boolean, lingerMs: number): boolean {
|
||||||
|
const [latched, setLatched] = useState(value);
|
||||||
|
|
||||||
|
// Asymmetric sync: snap up immediately when `value` becomes true,
|
||||||
|
// debounce the down transition by `lingerMs`. The setLatched(true)
|
||||||
|
// call below is intentionally a direct setState inside an effect
|
||||||
|
// because that is exactly the pattern we want (mirror upward edge,
|
||||||
|
// delay downward edge), and there is no equivalent without it.
|
||||||
|
useEffect(() => {
|
||||||
|
if (value) {
|
||||||
|
// eslint-disable-next-line react-hooks/set-state-in-effect -- intentional upward edge sync, see hook comment
|
||||||
|
setLatched(true);
|
||||||
|
return undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
const timer = window.setTimeout(() => {
|
||||||
|
setLatched(false);
|
||||||
|
}, lingerMs);
|
||||||
|
|
||||||
|
return () => {
|
||||||
|
window.clearTimeout(timer);
|
||||||
|
};
|
||||||
|
}, [value, lingerMs]);
|
||||||
|
|
||||||
|
return latched;
|
||||||
|
}
|
||||||
|
|
||||||
|
function HandTrackingRuntime({
|
||||||
|
enabled,
|
||||||
children,
|
children,
|
||||||
}: {
|
}: {
|
||||||
|
enabled: boolean;
|
||||||
children: ReactNode;
|
children: ReactNode;
|
||||||
}): React.JSX.Element {
|
}): React.JSX.Element {
|
||||||
const handTrackingSource = useDebugStore((debug) =>
|
const handTrackingSource = useDebugStore((debug) =>
|
||||||
debug.getHandTrackingSource(),
|
debug.getHandTrackingSource(),
|
||||||
);
|
);
|
||||||
const backendSnapshot = useRemoteHandTracking({
|
const backendSnapshot = useRemoteHandTracking({
|
||||||
enabled: handTrackingSource === "backend",
|
enabled: enabled && handTrackingSource === "backend",
|
||||||
});
|
});
|
||||||
const browserSnapshot = useBrowserHandTracking({
|
const browserSnapshot = useBrowserHandTracking({
|
||||||
enabled: handTrackingSource === "browser",
|
enabled: enabled && handTrackingSource === "browser",
|
||||||
});
|
});
|
||||||
const snapshot =
|
const snapshot = !enabled
|
||||||
handTrackingSource === "browser" ? browserSnapshot : backendSnapshot;
|
? HAND_TRACKING_IDLE_SNAPSHOT
|
||||||
|
: handTrackingSource === "browser"
|
||||||
|
? browserSnapshot
|
||||||
|
: backendSnapshot;
|
||||||
|
|
||||||
return <HandTrackingContext value={snapshot}>{children}</HandTrackingContext>;
|
return <HandTrackingContext value={snapshot}>{children}</HandTrackingContext>;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ import type {
|
|||||||
Vector3Scale,
|
Vector3Scale,
|
||||||
Vector3Tuple,
|
Vector3Tuple,
|
||||||
} from "@/types/three/three";
|
} from "@/types/three/three";
|
||||||
|
import type { RepairCasePartAnchorName } from "@/data/gameplay/repairCaseConfig";
|
||||||
|
|
||||||
export const REPAIR_MISSION_IDS = ["ebike", "pylon", "farm"] as const;
|
export const REPAIR_MISSION_IDS = ["ebike", "pylon", "farm"] as const;
|
||||||
|
|
||||||
@@ -24,7 +25,28 @@ export interface RepairMissionPartConfig {
|
|||||||
id: string;
|
id: string;
|
||||||
label: string;
|
label: string;
|
||||||
nodeName?: string;
|
nodeName?: string;
|
||||||
|
/**
|
||||||
|
* Name of a node inside the broken model where this part should snap on
|
||||||
|
* install. Used by replacement parts that target a slot in the broken
|
||||||
|
* model itself (e.g. pylon cable installs at the world-position of the
|
||||||
|
* pylon's `cable2` node), and by broken parts that should spawn at their
|
||||||
|
* original location on the broken model rather than a static offset.
|
||||||
|
*/
|
||||||
|
targetNodeName?: string;
|
||||||
caseSlotName?: string;
|
caseSlotName?: string;
|
||||||
|
/**
|
||||||
|
* Anchor name in the packderelance case where this replacement part is
|
||||||
|
* visually injected. When set, the part spawns at the world-position of
|
||||||
|
* that anchor instead of a generic placeholder slot.
|
||||||
|
*/
|
||||||
|
caseAnchor?: RepairCasePartAnchorName;
|
||||||
|
/**
|
||||||
|
* Group identifier for mutually exclusive replacement parts (e.g. pylon
|
||||||
|
* cables: only one cable can be held/installed at a time). When one part
|
||||||
|
* of the group is held, others in the same group are visually ghosted
|
||||||
|
* and non-interactive.
|
||||||
|
*/
|
||||||
|
caseLockGroup?: string;
|
||||||
modelPath?: string;
|
modelPath?: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -33,6 +55,7 @@ export interface RepairScannedBrokenPart {
|
|||||||
label: string;
|
label: string;
|
||||||
modelPath: string;
|
modelPath: string;
|
||||||
caseSlotName?: string;
|
caseSlotName?: string;
|
||||||
|
targetNodeName?: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
export interface RepairMissionConfig {
|
export interface RepairMissionConfig {
|
||||||
@@ -46,7 +69,13 @@ export interface RepairMissionConfig {
|
|||||||
brokenUiPath: string;
|
brokenUiPath: string;
|
||||||
case: RepairMissionCaseConfig;
|
case: RepairMissionCaseConfig;
|
||||||
reassemblySeconds?: number;
|
reassemblySeconds?: number;
|
||||||
requiredReplacementPartId: string;
|
/**
|
||||||
|
* Replacement part IDs accepted as the correct install. Multiple values
|
||||||
|
* are used when several alternatives are valid (e.g. pylon accepts either
|
||||||
|
* cable model). Install validation succeeds when any one of these parts
|
||||||
|
* is snapped into a placeholder slot.
|
||||||
|
*/
|
||||||
|
requiredReplacementPartIds: readonly string[];
|
||||||
scanPartSeconds?: number;
|
scanPartSeconds?: number;
|
||||||
brokenParts: readonly RepairMissionPartConfig[];
|
brokenParts: readonly RepairMissionPartConfig[];
|
||||||
replacementParts: readonly RepairMissionPartConfig[];
|
replacementParts: readonly RepairMissionPartConfig[];
|
||||||
|
|||||||
@@ -516,14 +516,29 @@ export function PlayerController({
|
|||||||
);
|
);
|
||||||
window.ebikeSteerFactor = steerFactor;
|
window.ebikeSteerFactor = steerFactor;
|
||||||
|
|
||||||
|
// ── Ebike camera tuning ──────────────────────────────────────────────────
|
||||||
|
// All motion effects in one place — set to 0 to fully disable each one.
|
||||||
|
/** Lateral camera drift when steering (0 = no sway) */
|
||||||
|
const CAM_SWAY_SIDE = -0.5;
|
||||||
|
/** Vertical camera drop when steering (0 = no recoil) */
|
||||||
|
const CAM_SWAY_VERTICAL = 0;
|
||||||
|
/** Position lerp factor. 1 = instant snap, lower = more lag/trail */
|
||||||
|
const CAM_POS_LERP = 1;
|
||||||
|
/** FOV boost at full speed in degrees (0 = constant FOV) */
|
||||||
|
const CAM_FOV_BOOST = 0.15; // speed × 0.15, capped at 3° → subtle speed sensation
|
||||||
|
/** How fast FOV lerps toward target (lower = slower breathing) */
|
||||||
|
const CAM_FOV_LERP = 4;
|
||||||
|
/** Visual body lean in radians at max steer (20° = 0.349 rad) */
|
||||||
|
const BIKE_LEAN = THREE.MathUtils.degToRad(10);
|
||||||
|
// ─────────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
const speed = velocity.current.length();
|
const speed = velocity.current.length();
|
||||||
const targetFov = 60 + Math.min(speed * 0.35, 9);
|
|
||||||
const perspectiveCam = camera as THREE.PerspectiveCamera;
|
const perspectiveCam = camera as THREE.PerspectiveCamera;
|
||||||
// eslint-disable-next-line react-hooks/immutability -- Three.js camera.fov must be mutated directly for dynamic FOV changes during frame updates
|
// eslint-disable-next-line react-hooks/immutability -- Three.js camera.fov must be mutated directly for dynamic FOV changes during frame updates
|
||||||
perspectiveCam.fov = THREE.MathUtils.lerp(
|
perspectiveCam.fov = THREE.MathUtils.lerp(
|
||||||
perspectiveCam.fov,
|
perspectiveCam.fov,
|
||||||
targetFov,
|
60 + Math.min(speed * CAM_FOV_BOOST, 3),
|
||||||
6 * dt,
|
CAM_FOV_LERP * dt,
|
||||||
);
|
);
|
||||||
perspectiveCam.updateProjectionMatrix();
|
perspectiveCam.updateProjectionMatrix();
|
||||||
|
|
||||||
@@ -532,9 +547,8 @@ export function PlayerController({
|
|||||||
);
|
);
|
||||||
cameraOffset.applyAxisAngle(_up, ebikeAngle.current);
|
cameraOffset.applyAxisAngle(_up, ebikeAngle.current);
|
||||||
|
|
||||||
const swingX = -Math.abs(steerFactor) * 1.5;
|
const swingX = -Math.abs(steerFactor) * CAM_SWAY_VERTICAL;
|
||||||
const swingZ = steerFactor > 0 ? steerFactor * 2.5 : steerFactor * 1.0;
|
const swingZ = steerFactor * CAM_SWAY_SIDE;
|
||||||
|
|
||||||
const cameraSwing = new THREE.Vector3(swingX, 0, swingZ);
|
const cameraSwing = new THREE.Vector3(swingX, 0, swingZ);
|
||||||
cameraSwing.applyAxisAngle(_up, ebikeAngle.current);
|
cameraSwing.applyAxisAngle(_up, ebikeAngle.current);
|
||||||
cameraOffset.add(cameraSwing);
|
cameraOffset.add(cameraSwing);
|
||||||
@@ -543,7 +557,7 @@ export function PlayerController({
|
|||||||
.copy(capsule.current.end)
|
.copy(capsule.current.end)
|
||||||
.add(cameraOffset);
|
.add(cameraOffset);
|
||||||
|
|
||||||
camera.position.lerp(targetCamPos, 12 * dt);
|
camera.position.lerp(targetCamPos, CAM_POS_LERP);
|
||||||
|
|
||||||
const pitchRad = THREE.MathUtils.degToRad(
|
const pitchRad = THREE.MathUtils.degToRad(
|
||||||
EBIKE_CAMERA_TRANSFORM.rotation[0],
|
EBIKE_CAMERA_TRANSFORM.rotation[0],
|
||||||
@@ -563,8 +577,12 @@ export function PlayerController({
|
|||||||
capsule.current.end.y - PLAYER_EYE_HEIGHT,
|
capsule.current.end.y - PLAYER_EYE_HEIGHT,
|
||||||
capsule.current.end.z,
|
capsule.current.end.z,
|
||||||
);
|
);
|
||||||
const leanAngle = steerFactor * 0.26;
|
ebikeVisual.rotation.set(
|
||||||
ebikeVisual.rotation.set(0, ebikeAngle.current, leanAngle, "YXZ");
|
steerFactor * -BIKE_LEAN,
|
||||||
|
ebikeAngle.current,
|
||||||
|
0,
|
||||||
|
"YXZ",
|
||||||
|
);
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
camera.position.copy(capsule.current.end);
|
camera.position.copy(capsule.current.end);
|
||||||
|
|||||||
Reference in New Issue
Block a user