feat add left hand tracking glove model
This commit is contained in:
@@ -41,7 +41,8 @@ This document describes the code that exists today in the repository.
|
||||
- `src/components/debug/DebugPerf.tsx` lazily mounts `r3f-perf` in debug mode.
|
||||
- `src/components/ui/debug/DebugOverlayLayout.tsx` mounts the compact HTML debug overlay when enabled from `lil-gui`.
|
||||
- `src/components/ui/debug/GameStateDebugPanel.tsx` exposes current game state, main/sub-state switching, previous/next step controls, and reset.
|
||||
- `src/components/ui/debug/HandTrackingDebugPanel.tsx` shows hand tracking status, usage, model-loaded placeholder, hand count, and fist state while hand tracking is active.
|
||||
- `src/components/ui/debug/HandTrackingDebugPanel.tsx` shows hand tracking status, usage, loaded glove model, hand count, and fist state while hand tracking is active.
|
||||
- `src/components/three/handTracking/HandTrackingLeftGlove.tsx` places the rigged `gant_l` model on the detected left hand in the debug physics scene.
|
||||
- `src/components/debug/scene/DebugHelpers.tsx` mounts debug helpers.
|
||||
- `src/components/debug/scene/DebugCameraControls.tsx` mounts the free debug camera.
|
||||
- `lil-gui` global debug controls include camera mode, scene mode, `R3F Perf`, and `Debug Overlay`; interaction-specific controls live in the `Interaction` folder.
|
||||
@@ -50,6 +51,7 @@ This document describes the code that exists today in the repository.
|
||||
|
||||
- `src/components/three/models/` contains reusable model helpers such as `ExplodableModel`.
|
||||
- `src/components/three/interaction/` contains reusable interaction wrappers such as `InteractableObject`, `TriggerObject`, and `GrabbableObject`.
|
||||
- `src/components/three/handTracking/` contains R3F hand tracking debug models such as the left glove overlay.
|
||||
- `src/components/three/gameplay/` contains the current core repair gameplay prototype: the repair case, repair game zone, and module slots.
|
||||
- `src/components/three/world/` contains reusable world/environment objects such as `SkyModel`.
|
||||
|
||||
|
||||
@@ -16,6 +16,7 @@ The feature is scoped to the debug physics scene rather than production gameplay
|
||||
4. The backend returns hand data including landmarks, handedness, score, center point, and `isFist`.
|
||||
5. React stores the latest snapshot in the hand tracking provider.
|
||||
6. `GrabbableObject` reads that snapshot each frame and uses fist state plus raycasting to grab objects.
|
||||
7. `HandTrackingLeftGlove` reads the same snapshot and places the rigged `gant_l` model on the detected left hand in the debug physics scene.
|
||||
|
||||
## Activation Rules
|
||||
|
||||
@@ -104,12 +105,19 @@ The final hold distance is clamped between the configured grab minimum and maxim
|
||||
|
||||
The current debug UI includes:
|
||||
|
||||
- `HandTrackingDebugPanel` inside `DebugOverlayLayout` for status, usage, model-loaded placeholder, server state, hand count, and fist state
|
||||
- `HandTrackingDebugPanel` inside `DebugOverlayLayout` for status, usage, loaded glove model, server state, hand count, and fist state
|
||||
- `HandTrackingVisualizer` for the SVG landmark wireframe
|
||||
- `HandTrackingLeftGlove` for the left-hand `gant_l` model in the R3F scene
|
||||
- `r3f-perf` for render performance
|
||||
- `lil-gui` for scene, camera, lighting, interaction, and grab controls
|
||||
|
||||
The hand tracking debug panel is a compact HTML grid outside the canvas. `Model loaded` is currently hardcoded to `none` until model-loading information is wired into the hand tracking flow. The hand wireframe is also HTML/SVG, not a 3D hand model.
|
||||
The hand tracking debug panel is a compact HTML grid outside the canvas. `Model loaded` displays `gant_l` when a left hand is detected, otherwise `none`. The hand wireframe is also HTML/SVG, not a 3D hand model.
|
||||
|
||||
## Left Glove Model
|
||||
|
||||
The current left glove MVP uses `public/models/gant_l/model.gltf`, which contains a GLTF skin and armature. For now the model is positioned, oriented, and scaled as a whole from palm landmarks instead of driving individual finger bones.
|
||||
|
||||
The right hand is intentionally ignored in this MVP. The available right-hand models are static in the current assets and are not mapped to MediaPipe bones yet.
|
||||
|
||||
## Known Limitations
|
||||
|
||||
@@ -118,3 +126,4 @@ The hand tracking debug panel is a compact HTML grid outside the canvas. `Model
|
||||
- The virtual hit zone is an approximation based on multiple raycasts, not a real 3D collider.
|
||||
- There is no smoothing layer for hand position or depth yet.
|
||||
- The hand visualization is an SVG landmark wireframe.
|
||||
- The left glove follows the palm as a whole; finger-by-finger bone animation still requires a verified landmark-to-bone mapping and smoothing.
|
||||
|
||||
Reference in New Issue
Block a user