fix hand tracking glove rendering

This commit is contained in:
Tom Boullay
2026-05-02 11:32:00 +02:00
parent bdc06f772f
commit fe662ebe7d
16 changed files with 319 additions and 155 deletions
+2 -2
View File
@@ -42,7 +42,7 @@ This document describes the code that exists today in the repository.
- `src/components/ui/debug/DebugOverlayLayout.tsx` mounts the compact HTML debug overlay when enabled from `lil-gui`. - `src/components/ui/debug/DebugOverlayLayout.tsx` mounts the compact HTML debug overlay when enabled from `lil-gui`.
- `src/components/ui/debug/GameStateDebugPanel.tsx` exposes current game state, main/sub-state switching, previous/next step controls, and reset. - `src/components/ui/debug/GameStateDebugPanel.tsx` exposes current game state, main/sub-state switching, previous/next step controls, and reset.
- `src/components/ui/debug/HandTrackingDebugPanel.tsx` shows hand tracking status, usage, loaded glove model, hand count, and fist state while hand tracking is active. - `src/components/ui/debug/HandTrackingDebugPanel.tsx` shows hand tracking status, usage, loaded glove model, hand count, and fist state while hand tracking is active.
- `src/components/three/handTracking/HandTrackingLeftGlove.tsx` places the rigged `gant_l` model on the detected left hand in the debug physics scene. - `src/components/three/handTracking/HandTrackingGlove.tsx` places the rigged `gant_l` and `gant_r` models on detected hands in the debug physics scene.
- `src/components/debug/scene/DebugHelpers.tsx` mounts debug helpers. - `src/components/debug/scene/DebugHelpers.tsx` mounts debug helpers.
- `src/components/debug/scene/DebugCameraControls.tsx` mounts the free debug camera. - `src/components/debug/scene/DebugCameraControls.tsx` mounts the free debug camera.
- `lil-gui` global debug controls include camera mode, scene mode, `R3F Perf`, and `Debug Overlay`; interaction-specific controls live in the `Interaction` folder. - `lil-gui` global debug controls include camera mode, scene mode, `R3F Perf`, and `Debug Overlay`; interaction-specific controls live in the `Interaction` folder.
@@ -51,7 +51,7 @@ This document describes the code that exists today in the repository.
- `src/components/three/models/` contains reusable model helpers such as `ExplodableModel`. - `src/components/three/models/` contains reusable model helpers such as `ExplodableModel`.
- `src/components/three/interaction/` contains reusable interaction wrappers such as `InteractableObject`, `TriggerObject`, and `GrabbableObject`. - `src/components/three/interaction/` contains reusable interaction wrappers such as `InteractableObject`, `TriggerObject`, and `GrabbableObject`.
- `src/components/three/handTracking/` contains R3F hand tracking debug models such as the left glove overlay. - `src/components/three/handTracking/` contains R3F hand tracking debug models such as the glove overlays.
- `src/components/three/gameplay/` contains the current core repair gameplay prototype: the repair case, repair game zone, and module slots. - `src/components/three/gameplay/` contains the current core repair gameplay prototype: the repair case, repair game zone, and module slots.
- `src/components/three/world/` contains reusable world/environment objects such as `SkyModel`. - `src/components/three/world/` contains reusable world/environment objects such as `SkyModel`.
+9 -9
View File
@@ -16,7 +16,7 @@ The feature is scoped to the debug physics scene rather than production gameplay
4. The backend returns hand data including landmarks, handedness, score, center point, and `isFist`. 4. The backend returns hand data including landmarks, handedness, score, center point, and `isFist`.
5. React stores the latest snapshot in the hand tracking provider. 5. React stores the latest snapshot in the hand tracking provider.
6. `GrabbableObject` reads that snapshot each frame and uses fist state plus raycasting to grab objects. 6. `GrabbableObject` reads that snapshot each frame and uses fist state plus raycasting to grab objects.
7. `HandTrackingLeftGlove` reads the same snapshot and places the rigged `gant_l` model on the detected left hand in the debug physics scene. 7. `HandTrackingGlove` reads the same snapshot and places the rigged `gant_l` and `gant_r` models on the detected hands in the debug physics scene.
## Activation Rules ## Activation Rules
@@ -106,18 +106,18 @@ The final hold distance is clamped between the configured grab minimum and maxim
The current debug UI includes: The current debug UI includes:
- `HandTrackingDebugPanel` inside `DebugOverlayLayout` for status, usage, loaded glove model, server state, hand count, and fist state - `HandTrackingDebugPanel` inside `DebugOverlayLayout` for status, usage, loaded glove model, server state, hand count, and fist state
- `HandTrackingVisualizer` for the SVG landmark wireframe - `HandTrackingVisualizer` for the SVG landmark wireframe fallback
- `HandTrackingLeftGlove` for the left-hand `gant_l` model in the R3F scene - `HandTrackingGlove` for the left-hand `gant_l` and right-hand `gant_r` models in the R3F scene
- `r3f-perf` for render performance - `r3f-perf` for render performance
- `lil-gui` for scene, camera, lighting, interaction, and grab controls - `lil-gui` for scene, camera, lighting, interaction, and grab controls
The hand tracking debug panel is a compact HTML grid outside the canvas. `Model loaded` displays `gant_l` when a left hand is detected, otherwise `none`. The hand wireframe is also HTML/SVG, not a 3D hand model. The hand tracking debug panel is a compact HTML grid outside the canvas. `Model loaded` displays the successfully loaded glove models. The SVG hand wireframe is only a fallback while models are loading or if a glove model fails to load.
## Left Glove Model ## Glove Models
The current left glove MVP uses `public/models/gant_l/model.gltf`, which contains a GLTF skin and armature. For now the model is positioned, oriented, and scaled as a whole from palm landmarks instead of driving individual finger bones. The current glove MVP uses `public/models/gant_l/model.gltf` and `public/models/gant_r/model.gltf`, which contain GLTF skins and armatures. For now each model is positioned, oriented, and scaled as a whole from palm landmarks instead of driving individual finger bones.
The right hand is intentionally ignored in this MVP. The available right-hand models are static in the current assets and are not mapped to MediaPipe bones yet. The glove models are intentionally smaller than the raw SVG overlay so they do not dominate the camera view.
## Known Limitations ## Known Limitations
@@ -125,5 +125,5 @@ The right hand is intentionally ignored in this MVP. The available right-hand mo
- MediaPipe depth is relative and can be noisy. - MediaPipe depth is relative and can be noisy.
- The virtual hit zone is an approximation based on multiple raycasts, not a real 3D collider. - The virtual hit zone is an approximation based on multiple raycasts, not a real 3D collider.
- There is no smoothing layer for hand position or depth yet. - There is no smoothing layer for hand position or depth yet.
- The hand visualization is an SVG landmark wireframe. - The SVG hand visualization is a fallback, not the primary display when glove models load correctly.
- The left glove follows the palm as a whole; finger-by-finger bone animation still requires a verified landmark-to-bone mapping and smoothing. - Each glove follows the palm as a whole; finger-by-finger bone animation still requires a verified landmark-to-bone mapping and smoothing.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,235 @@
import type { ReactNode } from "react";
import { Component, useEffect, useMemo, useRef } from "react";
import { useFrame, useThree } from "@react-three/fiber";
import { useGLTF } from "@react-three/drei";
import * as THREE from "three";
import { clone } from "three/addons/utils/SkeletonUtils.js";
import { useHandTrackingSnapshot } from "@/hooks/handTracking/useHandTrackingSnapshot";
import {
useHandTrackingGloveStatus,
type HandTrackingGloveHandedness,
} from "@/hooks/handTracking/useHandTrackingGloveStatus";
import { useLoggedGLTF } from "@/hooks/three/useLoggedGLTF";
import type { HandTrackingLandmark } from "@/types/handTracking/handTracking";
import { logModelLoadError } from "@/utils/three/modelLoadLogger";
const GLOVE_MODEL_PATHS: Record<HandTrackingGloveHandedness, string> = {
left: "/models/gant_l/model.gltf",
right: "/models/gant_r/model.gltf",
};
const GLOVE_ROOT_NODE_NAMES: Record<HandTrackingGloveHandedness, string> = {
left: "Hand_l",
right: "Hand_r",
};
const HAND_SPACE_DISTANCE = 2.4;
const HAND_DEPTH_SCALE = 0.45;
const GLOVE_SCALE = 0.17;
const _cameraPosition = new THREE.Vector3();
const _direction = new THREE.Vector3();
const _xAxis = new THREE.Vector3();
const _yAxis = new THREE.Vector3();
const _zAxis = new THREE.Vector3();
const _matrix = new THREE.Matrix4();
const _targetQuaternion = new THREE.Quaternion();
const _targetPosition = new THREE.Vector3();
const _wristPosition = new THREE.Vector3();
const _indexPosition = new THREE.Vector3();
const _middlePosition = new THREE.Vector3();
const _ringPosition = new THREE.Vector3();
const _pinkyPosition = new THREE.Vector3();
interface HandTrackingGloveProps {
handedness: HandTrackingGloveHandedness;
}
interface HandTrackingGloveErrorBoundaryProps {
children: ReactNode;
handedness: HandTrackingGloveHandedness;
modelPath: string;
}
class HandTrackingGloveErrorBoundary extends Component<
HandTrackingGloveErrorBoundaryProps,
{ hasError: boolean }
> {
constructor(props: HandTrackingGloveErrorBoundaryProps) {
super(props);
this.state = { hasError: false };
}
static getDerivedStateFromError(): { hasError: boolean } {
return { hasError: true };
}
componentDidCatch(error: Error): void {
useHandTrackingGloveStatus
.getState()
.setGloveStatus(this.props.handedness, "error");
logModelLoadError(
{
modelPath: this.props.modelPath,
scope: `HandTrackingGlove.${this.props.handedness}`,
scale: GLOVE_SCALE,
},
error,
);
}
render(): ReactNode {
if (this.state.hasError) return null;
return this.props.children;
}
}
function landmarkToWorldPoint(
landmark: HandTrackingLandmark,
camera: THREE.Camera,
target: THREE.Vector3,
): THREE.Vector3 {
_cameraPosition.setFromMatrixPosition(camera.matrixWorld);
target.set((1 - landmark.x) * 2 - 1, -landmark.y * 2 + 1, 0.5);
target.unproject(camera);
_direction.copy(target).sub(_cameraPosition).normalize();
target
.copy(_cameraPosition)
.addScaledVector(
_direction,
HAND_SPACE_DISTANCE - landmark.z * HAND_DEPTH_SCALE,
);
return target;
}
function matchesHandedness(
handHandedness: string,
targetHandedness: HandTrackingGloveHandedness,
): boolean {
return handHandedness.toLowerCase() === targetHandedness;
}
function HandTrackingGloveModel({
handedness,
}: HandTrackingGloveProps): React.JSX.Element | null {
const groupRef = useRef<THREE.Group>(null);
const { camera } = useThree();
const { hands } = useHandTrackingSnapshot();
const setGloveStatus = useHandTrackingGloveStatus(
(state) => state.setGloveStatus,
);
const modelPath = GLOVE_MODEL_PATHS[handedness];
const gltf = useLoggedGLTF(modelPath, {
scope: `HandTrackingGlove.${handedness}`,
scale: GLOVE_SCALE,
});
const gloveScene = useMemo(() => {
const rootNode = gltf.scene.getObjectByName(
GLOVE_ROOT_NODE_NAMES[handedness],
);
if (!rootNode) {
throw new Error(
`Missing glove root node ${GLOVE_ROOT_NODE_NAMES[handedness]}`,
);
}
return clone(rootNode);
}, [gltf.scene, handedness]);
const hand = hands.find((candidate) =>
matchesHandedness(candidate.handedness, handedness),
);
useEffect(() => {
setGloveStatus(handedness, "loaded");
}, [handedness, setGloveStatus]);
useFrame((_, delta) => {
const group = groupRef.current;
const trackedHand = hands.find((candidate) =>
matchesHandedness(candidate.handedness, handedness),
);
if (!group || !trackedHand || trackedHand.landmarks.length < 21) {
if (group) group.visible = false;
return;
}
group.visible = true;
const wrist = trackedHand.landmarks[0];
const indexMcp = trackedHand.landmarks[5];
const middleMcp = trackedHand.landmarks[9];
const ringMcp = trackedHand.landmarks[13];
const pinkyMcp = trackedHand.landmarks[17];
if (!wrist || !indexMcp || !middleMcp || !ringMcp || !pinkyMcp) {
group.visible = false;
return;
}
landmarkToWorldPoint(wrist, camera, _wristPosition);
landmarkToWorldPoint(indexMcp, camera, _indexPosition);
landmarkToWorldPoint(middleMcp, camera, _middlePosition);
landmarkToWorldPoint(ringMcp, camera, _ringPosition);
landmarkToWorldPoint(pinkyMcp, camera, _pinkyPosition);
_targetPosition
.copy(_wristPosition)
.add(_indexPosition)
.add(_middlePosition)
.add(_ringPosition)
.add(_pinkyPosition)
.multiplyScalar(0.2);
_yAxis.copy(_middlePosition).sub(_wristPosition).normalize();
_xAxis.copy(_indexPosition).sub(_pinkyPosition).normalize();
_zAxis.crossVectors(_xAxis, _yAxis).normalize();
if (
_xAxis.lengthSq() === 0 ||
_yAxis.lengthSq() === 0 ||
_zAxis.lengthSq() === 0
) {
return;
}
_xAxis.crossVectors(_yAxis, _zAxis).normalize();
_matrix.makeBasis(_xAxis, _yAxis, _zAxis);
_targetQuaternion.setFromRotationMatrix(_matrix);
group.position.lerp(_targetPosition, Math.min(1, delta * 18));
group.quaternion.slerp(_targetQuaternion, Math.min(1, delta * 18));
const palmLength = _wristPosition.distanceTo(_middlePosition);
const scale = palmLength * GLOVE_SCALE;
group.scale.setScalar(scale);
});
if (!hand) return null;
return <primitive ref={groupRef} object={gloveScene} />;
}
export function HandTrackingGlove({
handedness,
}: HandTrackingGloveProps): React.JSX.Element {
const modelPath = GLOVE_MODEL_PATHS[handedness];
return (
<HandTrackingGloveErrorBoundary
handedness={handedness}
modelPath={modelPath}
>
<HandTrackingGloveModel handedness={handedness} />
</HandTrackingGloveErrorBoundary>
);
}
useGLTF.preload(GLOVE_MODEL_PATHS.left);
useGLTF.preload(GLOVE_MODEL_PATHS.right);
@@ -1,130 +0,0 @@
import { useMemo, useRef } from "react";
import { useFrame, useThree } from "@react-three/fiber";
import { useGLTF } from "@react-three/drei";
import * as THREE from "three";
import { clone } from "three/addons/utils/SkeletonUtils.js";
import { useHandTrackingSnapshot } from "@/hooks/handTracking/useHandTrackingSnapshot";
import { useLoggedGLTF } from "@/hooks/three/useLoggedGLTF";
import type { HandTrackingLandmark } from "@/types/handTracking/handTracking";
const LEFT_GLOVE_MODEL_URL = "/models/gant_l/model.gltf";
const HAND_SPACE_DISTANCE = 2.4;
const HAND_DEPTH_SCALE = 0.45;
const GLOVE_SCALE = 0.34;
const _cameraPosition = new THREE.Vector3();
const _direction = new THREE.Vector3();
const _xAxis = new THREE.Vector3();
const _yAxis = new THREE.Vector3();
const _zAxis = new THREE.Vector3();
const _matrix = new THREE.Matrix4();
const _targetQuaternion = new THREE.Quaternion();
const _targetPosition = new THREE.Vector3();
const _wristPosition = new THREE.Vector3();
const _indexPosition = new THREE.Vector3();
const _middlePosition = new THREE.Vector3();
const _ringPosition = new THREE.Vector3();
const _pinkyPosition = new THREE.Vector3();
function landmarkToWorldPoint(
landmark: HandTrackingLandmark,
camera: THREE.Camera,
target: THREE.Vector3,
): THREE.Vector3 {
_cameraPosition.setFromMatrixPosition(camera.matrixWorld);
target.set((1 - landmark.x) * 2 - 1, -landmark.y * 2 + 1, 0.5);
target.unproject(camera);
_direction.copy(target).sub(_cameraPosition).normalize();
target
.copy(_cameraPosition)
.addScaledVector(
_direction,
HAND_SPACE_DISTANCE - landmark.z * HAND_DEPTH_SCALE,
);
return target;
}
function isLeftHand(handedness: string): boolean {
return handedness.toLowerCase() === "left";
}
export function HandTrackingLeftGlove(): React.JSX.Element | null {
const groupRef = useRef<THREE.Group>(null);
const { camera } = useThree();
const { hands } = useHandTrackingSnapshot();
const gltf = useLoggedGLTF(LEFT_GLOVE_MODEL_URL, {
scope: "HandTrackingLeftGlove",
scale: GLOVE_SCALE,
});
const gloveScene = useMemo(() => clone(gltf.scene), [gltf.scene]);
const leftHand = hands.find((hand) => isLeftHand(hand.handedness));
useFrame((_, delta) => {
const group = groupRef.current;
const hand = hands.find((candidate) => isLeftHand(candidate.handedness));
if (!group || !hand || hand.landmarks.length < 21) {
if (group) group.visible = false;
return;
}
group.visible = true;
const wrist = hand.landmarks[0];
const indexMcp = hand.landmarks[5];
const middleMcp = hand.landmarks[9];
const ringMcp = hand.landmarks[13];
const pinkyMcp = hand.landmarks[17];
if (!wrist || !indexMcp || !middleMcp || !ringMcp || !pinkyMcp) {
group.visible = false;
return;
}
landmarkToWorldPoint(wrist, camera, _wristPosition);
landmarkToWorldPoint(indexMcp, camera, _indexPosition);
landmarkToWorldPoint(middleMcp, camera, _middlePosition);
landmarkToWorldPoint(ringMcp, camera, _ringPosition);
landmarkToWorldPoint(pinkyMcp, camera, _pinkyPosition);
_targetPosition
.copy(_wristPosition)
.add(_indexPosition)
.add(_middlePosition)
.add(_ringPosition)
.add(_pinkyPosition)
.multiplyScalar(0.2);
_yAxis.copy(_middlePosition).sub(_wristPosition).normalize();
_xAxis.copy(_indexPosition).sub(_pinkyPosition).normalize();
_zAxis.crossVectors(_xAxis, _yAxis).normalize();
if (
_xAxis.lengthSq() === 0 ||
_yAxis.lengthSq() === 0 ||
_zAxis.lengthSq() === 0
) {
return;
}
_xAxis.crossVectors(_yAxis, _zAxis).normalize();
_matrix.makeBasis(_xAxis, _yAxis, _zAxis);
_targetQuaternion.setFromRotationMatrix(_matrix);
group.position.lerp(_targetPosition, Math.min(1, delta * 18));
group.quaternion.slerp(_targetQuaternion, Math.min(1, delta * 18));
const palmLength = _wristPosition.distanceTo(_middlePosition);
const scale = palmLength * GLOVE_SCALE;
group.scale.setScalar(scale);
});
if (!leftHand) return null;
return <primitive ref={groupRef} object={gloveScene} />;
}
useGLTF.preload(LEFT_GLOVE_MODEL_URL);
+6 -1
View File
@@ -1,4 +1,5 @@
import { useHandTrackingSnapshot } from "@/hooks/handTracking/useHandTrackingSnapshot"; import { useHandTrackingSnapshot } from "@/hooks/handTracking/useHandTrackingSnapshot";
import { useHandTrackingGloveStatus } from "@/hooks/handTracking/useHandTrackingGloveStatus";
const HAND_CONNECTIONS: Array<[number, number]> = [ const HAND_CONNECTIONS: Array<[number, number]> = [
[0, 1], [0, 1],
@@ -26,8 +27,12 @@ const HAND_CONNECTIONS: Array<[number, number]> = [
export function HandTrackingVisualizer(): React.JSX.Element | null { export function HandTrackingVisualizer(): React.JSX.Element | null {
const { hands, status } = useHandTrackingSnapshot(); const { hands, status } = useHandTrackingSnapshot();
const gloves = useHandTrackingGloveStatus((state) => state.gloves);
const shouldShowSvgFallback = Object.values(gloves).some(
(gloveStatus) => gloveStatus === "error" || gloveStatus === "idle",
);
if (status === "idle" || hands.length === 0) { if (status === "idle" || hands.length === 0 || !shouldShowSvgFallback) {
return null; return null;
} }
@@ -1,4 +1,5 @@
import { useHandTrackingSnapshot } from "@/hooks/handTracking/useHandTrackingSnapshot"; import { useHandTrackingSnapshot } from "@/hooks/handTracking/useHandTrackingSnapshot";
import { useHandTrackingGloveStatus } from "@/hooks/handTracking/useHandTrackingGloveStatus";
import type { HandTrackingStatus } from "@/types/handTracking/handTracking"; import type { HandTrackingStatus } from "@/types/handTracking/handTracking";
const STATUS_LABELS: Record<HandTrackingStatus, string> = { const STATUS_LABELS: Record<HandTrackingStatus, string> = {
@@ -15,22 +16,23 @@ const STATUS_LABELS: Record<HandTrackingStatus, string> = {
export function HandTrackingDebugPanel(): React.JSX.Element | null { export function HandTrackingDebugPanel(): React.JSX.Element | null {
const { hands, status, usageStatus, serverStatus, error } = const { hands, status, usageStatus, serverStatus, error } =
useHandTrackingSnapshot(); useHandTrackingSnapshot();
const gloves = useHandTrackingGloveStatus((state) => state.gloves);
if (status === "idle") { if (status === "idle") {
return null; return null;
} }
const fist = hands.some((hand) => hand.isFist); const fist = hands.some((hand) => hand.isFist);
const hasLeftHand = hands.some(
(hand) => hand.handedness.toLowerCase() === "left",
);
const hasRightHand = hands.some(
(hand) => hand.handedness.toLowerCase() === "right",
);
const modelLoaded = const modelLoaded =
[hasLeftHand ? "gant_l" : null, hasRightHand ? "gant_r" : null] [
gloves.left === "loaded" ? "gant_l" : null,
gloves.right === "loaded" ? "gant_r" : null,
]
.filter(Boolean) .filter(Boolean)
.join(", ") || "none"; .join(", ") || "none";
const modelFallback = Object.values(gloves).some(
(gloveStatus) => gloveStatus === "error",
);
return ( return (
<section <section
@@ -51,6 +53,10 @@ export function HandTrackingDebugPanel(): React.JSX.Element | null {
<dt>Model loaded</dt> <dt>Model loaded</dt>
<dd>{modelLoaded}</dd> <dd>{modelLoaded}</dd>
</div> </div>
<div>
<dt>SVG fallback</dt>
<dd>{modelFallback ? "yes" : "no"}</dd>
</div>
{serverStatus ? ( {serverStatus ? (
<div> <div>
<dt>Server</dt> <dt>Server</dt>
+1 -1
View File
@@ -125,7 +125,7 @@ Ce document décrit le code réellement présent aujourd'hui dans le dépôt.
- \`src/components/ui/debug/DebugOverlayLayout.tsx\` monte l'overlay HTML debug compact quand il est activé depuis \`lil-gui\`. - \`src/components/ui/debug/DebugOverlayLayout.tsx\` monte l'overlay HTML debug compact quand il est activé depuis \`lil-gui\`.
- \`src/components/ui/debug/GameStateDebugPanel.tsx\` expose l'état de jeu courant, le changement de main/sub-state, les contrôles previous/next step et le reset. - \`src/components/ui/debug/GameStateDebugPanel.tsx\` expose l'état de jeu courant, le changement de main/sub-state, les contrôles previous/next step et le reset.
- \`src/components/ui/debug/HandTrackingDebugPanel.tsx\` affiche le statut hand tracking, l'usage, le modèle de gant chargé, le nombre de mains et l'état fist pendant l'activation du hand tracking. - \`src/components/ui/debug/HandTrackingDebugPanel.tsx\` affiche le statut hand tracking, l'usage, le modèle de gant chargé, le nombre de mains et l'état fist pendant l'activation du hand tracking.
- \`src/components/three/handTracking/HandTrackingLeftGlove.tsx\` place le modèle riggé \`gant_l\` sur la main gauche détectée dans la scène physics debug. - \`src/components/three/handTracking/HandTrackingGlove.tsx\` place les modèles riggés \`gant_l\` et \`gant_r\` sur les mains détectées dans la scène physics debug.
- \`src/components/debug/scene/DebugHelpers.tsx\` monte les helpers debug. - \`src/components/debug/scene/DebugHelpers.tsx\` monte les helpers debug.
- \`src/components/debug/scene/DebugCameraControls.tsx\` monte la caméra libre debug. - \`src/components/debug/scene/DebugCameraControls.tsx\` monte la caméra libre debug.
- Les contrôles globaux \`lil-gui\` incluent camera mode, scene mode, \`R3F Perf\` et \`Debug Overlay\`; les contrôles d'interaction vivent dans le dossier \`Interaction\`. - Les contrôles globaux \`lil-gui\` incluent camera mode, scene mode, \`R3F Perf\` et \`Debug Overlay\`; les contrôles d'interaction vivent dans le dossier \`Interaction\`.
@@ -0,0 +1,28 @@
import { create } from "zustand";
export type HandTrackingGloveHandedness = "left" | "right";
type HandTrackingGloveLoadState = "idle" | "loaded" | "error";
interface HandTrackingGloveStatusState {
gloves: Record<HandTrackingGloveHandedness, HandTrackingGloveLoadState>;
setGloveStatus: (
handedness: HandTrackingGloveHandedness,
status: HandTrackingGloveLoadState,
) => void;
}
export const useHandTrackingGloveStatus =
create<HandTrackingGloveStatusState>()((set) => ({
gloves: {
left: "idle",
right: "idle",
},
setGloveStatus: (handedness, status) =>
set((state) => ({
gloves: {
...state.gloves,
[handedness]: status,
},
})),
}));
+7 -2
View File
@@ -8,7 +8,7 @@ import { useCameraMode } from "@/hooks/debug/useCameraMode";
import { useSceneMode } from "@/hooks/debug/useSceneMode"; import { useSceneMode } from "@/hooks/debug/useSceneMode";
import { DebugCameraControls } from "@/components/debug/scene/DebugCameraControls"; import { DebugCameraControls } from "@/components/debug/scene/DebugCameraControls";
import { DebugHelpers } from "@/components/debug/scene/DebugHelpers"; import { DebugHelpers } from "@/components/debug/scene/DebugHelpers";
import { HandTrackingLeftGlove } from "@/components/three/handTracking/HandTrackingLeftGlove"; import { HandTrackingGlove } from "@/components/three/handTracking/HandTrackingGlove";
import { Environment } from "@/world/Environment"; import { Environment } from "@/world/Environment";
import { GameMusic } from "@/world/GameMusic"; import { GameMusic } from "@/world/GameMusic";
import { Lighting } from "@/world/Lighting"; import { Lighting } from "@/world/Lighting";
@@ -31,7 +31,12 @@ export function World(): React.JSX.Element {
<Environment /> <Environment />
<Lighting /> <Lighting />
<DebugHelpers /> <DebugHelpers />
{sceneMode === "physics" ? <HandTrackingLeftGlove /> : null} {sceneMode === "physics" ? (
<>
<HandTrackingGlove handedness="left" />
<HandTrackingGlove handedness="right" />
</>
) : null}
{cameraMode === "debug" ? <DebugCameraControls /> : null} {cameraMode === "debug" ? <DebugCameraControls /> : null}
{sceneMode === "game" ? ( {sceneMode === "game" ? (