refactor: clean upload pipeline and restore draco delivery

This commit is contained in:
Tom Boullay
2026-04-29 16:29:32 +02:00
parent 097b8f6486
commit 498765db61
32 changed files with 769 additions and 215 deletions
+1
View File
@@ -1,3 +1,4 @@
*.glb filter=lfs diff=lfs merge=lfs -text
*.gltf filter=lfs diff=lfs merge=lfs -text *.gltf filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text *.bin filter=lfs diff=lfs merge=lfs -text
*.png filter=lfs diff=lfs merge=lfs -text *.png filter=lfs diff=lfs merge=lfs -text
+7 -3
View File
@@ -1,6 +1,6 @@
# ============================================================================= # =============================================================================
# Upload GLTF — Dockerfile for Coolify # Upload GLTF — Dockerfile for Coolify
# Node 20 Debian · Multi-stage build # Node 20 Debian · Blender (headless) · Multi-stage build
# ============================================================================= # =============================================================================
# --- Stage 1: Dependencies --------------------------------------------------- # --- Stage 1: Dependencies ---------------------------------------------------
@@ -28,10 +28,11 @@ RUN npm run build
FROM node:20-slim AS runner FROM node:20-slim AS runner
LABEL maintainer="La Fabrik Durable" LABEL maintainer="La Fabrik Durable"
LABEL description="Secure GLTF upload interface with texture compression and GitHub push" LABEL description="Secure GLTF upload interface with Draco compression and GitHub push"
# Install runtime helpers # Install Blender (headless) + runtime helpers
RUN apt-get update && apt-get install -y --no-install-recommends \ RUN apt-get update && apt-get install -y --no-install-recommends \
blender \
tini \ tini \
curl \ curl \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
@@ -48,6 +49,9 @@ COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static COPY --from=builder /app/.next/static ./.next/static
COPY --from=builder /app/public ./public COPY --from=builder /app/public ./public
# Copy the Blender compression script
COPY --from=builder /app/scripts ./scripts
# Ensure tmp dir for uploads exists # Ensure tmp dir for uploads exists
RUN mkdir -p /tmp/assets RUN mkdir -p /tmp/assets
+14 -20
View File
@@ -3,7 +3,7 @@
A secure web interface for uploading `model.gltf` with its associated `.bin` file and textures with two outputs: A secure web interface for uploading `model.gltf` with its associated `.bin` file and textures with two outputs:
- **Nextcloud Drive** — Archives the original files with automatic versioning (VF/V1/V2...), so artists always have a history of past versions - **Nextcloud Drive** — Archives the original files with automatic versioning (VF/V1/V2...), so artists always have a history of past versions
- **GitHub** — Delivers GLTF assets and compressed textures to the dev team's repository, ready for integration - **GitHub** — Delivers Draco-compressed GLB assets by default, with an optional GLTF delivery mode for specific models
Built for La Fabrik Durable's internal use, but open-sourced for anyone looking for a similar solution. The app validates the upload locally, stages it server-side, then applies intelligent diffs to avoid unnecessary uploads and commits. The Drive upload serves as the source of truth and version history, while the GitHub upload delivers the prepared assets to developers Built for La Fabrik Durable's internal use, but open-sourced for anyone looking for a similar solution. The app validates the upload locally, stages it server-side, then applies intelligent diffs to avoid unnecessary uploads and commits. The Drive upload serves as the source of truth and version history, while the GitHub upload delivers the prepared assets to developers
@@ -34,7 +34,7 @@ The app runs on `http://localhost:3000` with hot reload. The upload API routes a
- Any associated binary buffer (`.bin`, for example `model.bin`) - Any associated binary buffer (`.bin`, for example `model.bin`)
- Any associated textures (`.png/.jpg/.jpeg/.webp`) - Any associated textures (`.png/.jpg/.jpeg/.webp`)
3. The folder is validated locally. `.glb` files are not accepted. 3. The folder is validated locally. `.glb` files are not accepted.
4. On clicking "Envoyer": 4. On clicking "Envoyer" or "Envoyer en GLTF":
- The app uploads the folder once to a temporary server-side staging area - The app uploads the folder once to a temporary server-side staging area
- The app prepares the final Git payload from this staging area - The app prepares the final Git payload from this staging area
- The app checks the remote Git repo for existing files and computes diffs - The app checks the remote Git repo for existing files and computes diffs
@@ -57,7 +57,7 @@ Invalid or unknown asset names still block the upload.
### Upload flow: Drive first, then Git ### Upload flow: Drive first, then Git
5. **Drive upload (archiving)** — Original files from the staging area are uploaded to the Nextcloud Drive with automatic versioning (see below). This serves as the artists' source of truth and version history. If the Drive upload fails, a modal asks the user whether to send to Git only or cancel entirely. 5. **Drive upload (archiving)** — Original files from the staging area are uploaded to the Nextcloud Drive with automatic versioning (see below). This serves as the artists' source of truth and version history. If the Drive upload fails, a modal asks the user whether to send to Git only or cancel entirely.
6. **Git upload (delivery to devs)** — The prepared Git payload is reused from staging: `model.gltf` and `.bin` files are preserved, textures are compressed server-side, then all changed files are pushed to GitHub in a single commit. This is what the dev team consumes in the application. 6. **Git upload (delivery to devs)** — The prepared Git payload is reused from staging. By default, Blender exports a single `model.glb` with Draco compression. For specific models, "Envoyer en GLTF" keeps the current separate `model.gltf` + `.bin` + compressed textures workflow.
### Drive versioning (Nextcloud WebDAV) ### Drive versioning (Nextcloud WebDAV)
@@ -99,16 +99,7 @@ All changes are pushed in a **single commit** with a grouped formatted message:
update: upload-gltf add a new model -> my-model update: upload-gltf add a new model -> my-model
📦 Model 📦 Model
✅ model.gltf ✅ model.glb (compressed)
🎨 Textures (color)
✅ color_porte.jpg (compressed)
🪶 Textures (roughness)
✅ roughness_tuyaux.png (compressed)
🧩 Assets
✅ model.bin
✅ opacity_fenetre.png (compressed)
``` ```
**Update (only one texture changed):** **Update (only one texture changed):**
@@ -116,10 +107,7 @@ update: upload-gltf add a new model -> my-model
update: upload-gltf update -> coffeetest update: upload-gltf update -> coffeetest
📦 Model 📦 Model
↔️ model.gltf 🔄 model.glb (compressed)
🎨 Textures (color)
🔄 color_tuyaux.jpg (compressed)
``` ```
Commit sections: Commit sections:
@@ -135,7 +123,7 @@ Commit sections:
Symbols: `✅` new — `🔄` modified — `↔️` unchanged (model always re-pushed) — `❌` deleted Symbols: `✅` new — `🔄` modified — `↔️` unchanged (model always re-pushed) — `❌` deleted
7. Orphan files (present on remote but not in the new upload) are deleted in the same commit 7. Orphan files (present on remote but not in the new upload) are deleted in the same commit
8. `model.gltf` is pushed as-is so companion files like `model.bin` remain valid 8. Default Git delivery pushes `model.glb` only; "Envoyer en GLTF" pushes separate `model.gltf`, `.bin`, and compressed textures.
Uploaded models are pushed to `public/models/<folderName>/` in the target repo. Uploaded models are pushed to `public/models/<folderName>/` in the target repo.
@@ -143,6 +131,7 @@ Uploaded models are pushed to `public/models/<folderName>/` in the target repo.
- Large uploads are faster than before because the folder is staged only once, but the Drive upload remains sequential. - Large uploads are faster than before because the folder is staged only once, but the Drive upload remains sequential.
- Git LFS batch uploads are sequential by batch. - Git LFS batch uploads are sequential by batch.
- Default GLB Draco delivery reduces Git LFS usage by replacing many support files with one compressed model file.
- Uploads expect a single `model.gltf` file plus optional flat support files (`.bin`, `.png`, `.jpg`, `.jpeg`, `.webp`). - Uploads expect a single `model.gltf` file plus optional flat support files (`.bin`, `.png`, `.jpg`, `.jpeg`, `.webp`).
## Project Structure ## Project Structure
@@ -193,11 +182,14 @@ lib/
├── upload-lock.ts # Lightweight in-memory per-folder upload lock ├── upload-lock.ts # Lightweight in-memory per-folder upload lock
├── asset-classification.ts # Group assets by family for commit messages ├── asset-classification.ts # Group assets by family for commit messages
├── asset-naming.ts # Allowed asset families and naming convention helpers ├── asset-naming.ts # Allowed asset families and naming convention helpers
├── blender.ts # Blender Draco compression helper
├── commit-message.ts # Commit message builder ├── commit-message.ts # Commit message builder
├── parse-upload.ts # FormData parser + validation ├── parse-upload.ts # FormData parser + validation
├── validate-folder.ts # Client-side folder validation (discriminated union) ├── validate-folder.ts # Client-side folder validation (discriminated union)
└── format-bytes.ts # Byte formatting utility └── format-bytes.ts # Byte formatting utility
Dockerfile # Multi-stage build: Node 20 slim + tini scripts/
└── compress.py # Blender Draco compression script
Dockerfile # Multi-stage build: Node 20 slim + Blender + tini
docker-entrypoint.sh # Startup check + launch docker-entrypoint.sh # Startup check + launch
``` ```
@@ -253,7 +245,7 @@ docker run -p 3000:3000 \
upload-gltf upload-gltf
``` ```
The Docker image runs the Next.js app and server-side asset preparation in a single container. The `docker-entrypoint.sh` script checks for required environment variables before launching the app The Docker image runs the Next.js app, Blender Draco compression, and server-side asset preparation in a single container. The `docker-entrypoint.sh` script checks for required environment variables before launching the app
## Supported Formats ## Supported Formats
@@ -263,6 +255,8 @@ The Docker image runs the Next.js app and server-side asset preparation in a sin
| Binary buffers | `.bin` | | Binary buffers | `.bin` |
| Textures | `.png`, `.jpg`, `.jpeg`, `.webp` | | Textures | `.png`, `.jpg`, `.jpeg`, `.webp` |
Git delivery outputs `.glb` by default, or keeps the source `.gltf` structure when "Envoyer en GLTF" is selected.
## License ## License
See [MIT](LICENSE) License See [MIT](LICENSE) License
+2 -1
View File
@@ -6,6 +6,7 @@ import { getModelFolderPath } from '@/lib/model-paths'
import { ensurePreparedStagingAssets } from '@/lib/upload-staging' import { ensurePreparedStagingAssets } from '@/lib/upload-staging'
import { parseStagingRequestBody } from '@/lib/upload-request' import { parseStagingRequestBody } from '@/lib/upload-request'
import { getErrorMessage } from '@/lib/guards' import { getErrorMessage } from '@/lib/guards'
import type { FileDiff } from '@/lib/types'
export const runtime = 'nodejs' export const runtime = 'nodejs'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
@@ -37,7 +38,7 @@ export async function POST(req: NextRequest) {
const remoteFileMap = new Map(files.map((file) => [file.name.toLowerCase(), file.size])) const remoteFileMap = new Map(files.map((file) => [file.name.toLowerCase(), file.size]))
const { fileChanges, deletedFileNames } = classifyFileChanges(filesToPush, remoteFileMap, folderPath) const { fileChanges, deletedFileNames } = classifyFileChanges(filesToPush, remoteFileMap, folderPath)
const diffs: Array<{ name: string; status: 'new' | 'changed' | 'deleted' }> = [] const diffs: FileDiff[] = []
for (const [name, status] of fileChanges.entries()) { for (const [name, status] of fileChanges.entries()) {
if (status === 'new' || status === 'changed') { if (status === 'new' || status === 'changed') {
+2 -1
View File
@@ -10,6 +10,7 @@ import {
import { acquireUploadLock, releaseUploadLock } from '@/lib/upload-lock' import { acquireUploadLock, releaseUploadLock } from '@/lib/upload-lock'
import { parseDriveRequestBody } from '@/lib/upload-request' import { parseDriveRequestBody } from '@/lib/upload-request'
import { getErrorMessage } from '@/lib/guards' import { getErrorMessage } from '@/lib/guards'
import type { DriveAction } from '@/lib/types'
export const runtime = 'nodejs' export const runtime = 'nodejs'
export const dynamic = 'force-dynamic' export const dynamic = 'force-dynamic'
@@ -27,7 +28,7 @@ export async function POST(req: NextRequest) {
let folderName: string let folderName: string
let parsedFiles: Awaited<ReturnType<typeof readStagedOriginalFiles>>['files'] let parsedFiles: Awaited<ReturnType<typeof readStagedOriginalFiles>>['files']
let action: 'new' | 'replace' let action: DriveAction
try { try {
const body: unknown = await req.json() const body: unknown = await req.json()
+1 -1
View File
@@ -13,7 +13,7 @@ export async function POST(req: NextRequest) {
try { try {
const parsed = await parseMultiUpload(req) const parsed = await parseMultiUpload(req)
const staged = await createStagingUpload(parsed.folderName, parsed.files) const staged = await createStagingUpload(parsed.folderName, parsed.files, parsed.gitModelMode)
return NextResponse.json({ success: true, ...staged }) return NextResponse.json({ success: true, ...staged })
} catch (err) { } catch (err) {
const message = getErrorMessage(err) const message = getErrorMessage(err)
+10 -3
View File
@@ -29,6 +29,8 @@ interface AlphaMapMaterial extends Material {
alphaTest: number alphaTest: number
} }
type AlphaImageSource = HTMLImageElement | HTMLCanvasElement | ImageBitmap
const alphaMapTextureCache = new WeakMap<Texture, Texture>() const alphaMapTextureCache = new WeakMap<Texture, Texture>()
function getRequestedFilename(requestedUrl: string) { function getRequestedFilename(requestedUrl: string) {
@@ -91,14 +93,19 @@ function supportsAlphaMap(material: Material): material is AlphaMapMaterial {
return 'alphaMap' in material return 'alphaMap' in material
} }
function isAlphaImageSource(image: object | null | undefined): image is AlphaImageSource {
return image instanceof HTMLImageElement
|| image instanceof HTMLCanvasElement
|| (typeof ImageBitmap !== 'undefined' && image instanceof ImageBitmap)
}
function createAlphaMapTexture(texture: Texture) { function createAlphaMapTexture(texture: Texture) {
const cachedTexture = alphaMapTextureCache.get(texture) const cachedTexture = alphaMapTextureCache.get(texture)
if (cachedTexture) return cachedTexture if (cachedTexture) return cachedTexture
const image = texture.image as unknown const image = texture.image as object | null | undefined
const isImageBitmap = typeof ImageBitmap !== 'undefined' && image instanceof ImageBitmap
if (!(image instanceof HTMLImageElement || image instanceof HTMLCanvasElement || isImageBitmap)) { if (!isAlphaImageSource(image)) {
texture.flipY = false texture.flipY = false
alphaMapTextureCache.set(texture, texture) alphaMapTextureCache.set(texture, texture)
return texture return texture
-9
View File
@@ -1,7 +1,3 @@
// ---------------------------------------------------------------------------
// Shared modal wrapper — handles overlay, centering, dialog role, aria
// ---------------------------------------------------------------------------
import type { ReactNode } from 'react' import type { ReactNode } from 'react'
interface ModalProps { interface ModalProps {
@@ -24,16 +20,11 @@ export default function Modal({ ariaLabelledBy, children }: ModalProps) {
) )
} }
// ---------------------------------------------------------------------------
// Shared modal footer with two buttons
// ---------------------------------------------------------------------------
interface ModalActionsProps { interface ModalActionsProps {
cancelLabel: string cancelLabel: string
confirmLabel: string confirmLabel: string
onCancel: () => void onCancel: () => void
onConfirm: () => void onConfirm: () => void
/** Tailwind classes for the confirm button (default: white bg) */
confirmClassName?: string confirmClassName?: string
disabled?: boolean disabled?: boolean
} }
-4
View File
@@ -1,7 +1,3 @@
// ---------------------------------------------------------------------------
// Shared SVG icon components
// ---------------------------------------------------------------------------
interface IconProps { interface IconProps {
className?: string className?: string
} }
+34 -15
View File
@@ -1,4 +1,5 @@
import { SpinnerIcon } from '@/components/ui/icons' import { SpinnerIcon, WarningIcon } from '@/components/ui/icons'
import type { GitModelMode } from '@/lib/types'
interface ActionButtonsProps { interface ActionButtonsProps {
isUploading: boolean isUploading: boolean
@@ -7,7 +8,7 @@ interface ActionButtonsProps {
hasPendingOrErrors: boolean hasPendingOrErrors: boolean
allDone: boolean allDone: boolean
hasErrors: boolean hasErrors: boolean
onUpload: () => void onUpload: (gitModelMode: GitModelMode) => void
onCancel: () => void onCancel: () => void
onReset: () => void onReset: () => void
} }
@@ -27,20 +28,38 @@ export default function ActionButtons({
const isBusy = isUploading || isChecking const isBusy = isUploading || isChecking
return ( return (
<div className="flex gap-3"> <div className="flex flex-col gap-3 sm:flex-row">
{!isBusy && hasPendingOrErrors && ( {!isBusy && hasPendingOrErrors && (
<button <>
onClick={onUpload} <button
disabled={cantUpload} onClick={() => onUpload('draco-glb')}
className={`flex-1 font-medium text-sm py-2.5 px-6 rounded-xl transition-all duration-150 disabled={cantUpload}
focus:outline-none focus:ring-2 focus:ring-white/50 border border-white/20 className={`flex-1 font-medium text-sm py-2.5 px-6 rounded-xl transition-all duration-150
${cantUpload focus:outline-none focus:ring-2 focus:ring-white/50 border border-white/20
? 'bg-white/30 text-gray-500 cursor-not-allowed' ${cantUpload
: 'bg-white text-[#000000] hover:bg-gray-200' ? 'bg-white/30 text-gray-500 cursor-not-allowed'
}`} : 'bg-white text-[#000000] hover:bg-gray-200'
> }`}
Envoyer >
</button> Envoyer
</button>
<button
onClick={() => onUpload('keep-gltf')}
disabled={cantUpload}
className={`flex-1 font-medium text-sm py-2.5 px-6 rounded-xl border transition-all duration-150
focus:outline-none focus:ring-2 focus:ring-white/30
${cantUpload
? 'bg-black-800 text-gray-600 border-white/10 cursor-not-allowed'
: 'bg-black-700 text-gray-300 border-black-600 hover:bg-black-600'
}`}
>
<span className="flex items-center justify-center gap-2">
<WarningIcon className="w-4 h-4 text-yellow-400" />
<span>Envoyer en GLTF</span>
</span>
</button>
</>
)} )}
{isBusy && ( {isBusy && (
-4
View File
@@ -1,7 +1,3 @@
// ---------------------------------------------------------------------------
// Drive/Git status sub-line for FolderCard
// ---------------------------------------------------------------------------
import { SpinnerIcon, XIcon, WarningIcon } from '@/components/ui/icons' import { SpinnerIcon, XIcon, WarningIcon } from '@/components/ui/icons'
import type { FolderEntry } from '@/lib/client-types' import type { FolderEntry } from '@/lib/client-types'
+1 -2
View File
@@ -59,7 +59,6 @@ export default function FolderCard({ entry, index, onToggleViewer, onRemove }: F
)} )}
</div> </div>
{/* Drive status sub-line (only during upload, not after success) */}
{entry.status !== 'success' && entry.driveStatus && entry.driveStatus !== 'pending' && ( {entry.status !== 'success' && entry.driveStatus && entry.driveStatus !== 'pending' && (
<DriveStatusLine driveStatus={entry.driveStatus} driveError={entry.driveError} /> <DriveStatusLine driveStatus={entry.driveStatus} driveError={entry.driveError} />
)} )}
@@ -97,7 +96,7 @@ export default function FolderCard({ entry, index, onToggleViewer, onRemove }: F
> >
<ModelViewer <ModelViewer
url={entry.modelUrl} url={entry.modelUrl}
assetUrls={entry.assetUrls || {}} assetUrls={entry.assetUrls}
filename={entry.modelFile.name} filename={entry.modelFile.name}
size={formatBytes(entry.modelFile.size)} size={formatBytes(entry.modelFile.size)}
/> />
+9
View File
@@ -6,6 +6,15 @@ echo "[upload-gltf] Starting Upload GLTF..."
# Ensure tmp directory for uploads exists # Ensure tmp directory for uploads exists
mkdir -p /tmp/assets mkdir -p /tmp/assets
# Check if Blender is available for Draco compression
if command -v blender > /dev/null 2>&1; then
BLENDER_VERSION=$(blender --version 2>/dev/null | head -n 1)
echo "[upload-gltf] Blender found: $BLENDER_VERSION"
echo "[upload-gltf] Draco compression is enabled."
else
echo "[upload-gltf] WARNING: Blender not found. GLB Draco uploads will fail; use 'Envoyer en GLTF' if needed."
fi
echo "[upload-gltf] Ready. Launching application..." echo "[upload-gltf] Ready. Launching application..."
exec "$@" exec "$@"
+11 -8
View File
@@ -3,20 +3,22 @@
import { useState, useRef, useCallback } from 'react' import { useState, useRef, useCallback } from 'react'
import { getErrorMessage } from '@/lib/guards' import { getErrorMessage } from '@/lib/guards'
import type { FolderEntry } from '@/lib/client-types' import type { FolderEntry } from '@/lib/client-types'
import type { FileDiff } from '@/lib/types' import type { DriveAction, FileDiff, GitModelMode } from '@/lib/types'
import { checkFolderDiffs, stageUpload, uploadDrive, uploadGit } from '@/lib/upload-api' import { checkFolderDiffs, stageUpload, uploadDrive, uploadGit } from '@/lib/upload-api'
import type { CheckResult } from '@/lib/upload-api' import type { CheckResult } from '@/lib/upload-api'
type UploadLogDetails = Record<string, string | number | boolean | undefined>
function formatElapsed(startedAt: number) { function formatElapsed(startedAt: number) {
return `${((performance.now() - startedAt) / 1000).toFixed(1)}s` return `${((performance.now() - startedAt) / 1000).toFixed(1)}s`
} }
function logUpload(level: 'INFO' | 'ERROR', step: string, action: string, startedAt: number, details?: Record<string, unknown>) { function logUpload(level: 'INFO' | 'ERROR', step: string, action: string, startedAt: number, details?: UploadLogDetails) {
const log = level === 'ERROR' ? console.error : console.info const log = level === 'ERROR' ? console.error : console.info
log(`[${level}] ${step} -> ${action} | Timer: ${formatElapsed(startedAt)}`, details || '') log(`[${level}] ${step} -> ${action} | Timer: ${formatElapsed(startedAt)}`, details || '')
} }
function startTimedLog(step: string, action: string, details?: Record<string, unknown>) { function startTimedLog(step: string, action: string, details?: UploadLogDetails) {
const startedAt = performance.now() const startedAt = performance.now()
logUpload('INFO', step, `${action} started`, startedAt, details) logUpload('INFO', step, `${action} started`, startedAt, details)
@@ -24,7 +26,7 @@ function startTimedLog(step: string, action: string, details?: Record<string, un
logUpload('INFO', step, `${action} running`, startedAt, details) logUpload('INFO', step, `${action} running`, startedAt, details)
}, 10_000) }, 10_000)
return (status: 'done' | 'failed' | 'cancelled' = 'done', extra?: Record<string, unknown>) => { return (status: 'done' | 'failed' | 'cancelled' = 'done', extra?: UploadLogDetails) => {
window.clearInterval(interval) window.clearInterval(interval)
logUpload(status === 'failed' ? 'ERROR' : 'INFO', step, `${action} ${status}`, startedAt, { ...details, ...extra }) logUpload(status === 'failed' ? 'ERROR' : 'INFO', step, `${action} ${status}`, startedAt, { ...details, ...extra })
} }
@@ -122,7 +124,7 @@ export function useUploadOrchestrator({
if (controller.signal.aborted) break if (controller.signal.aborted) break
const folderEntry = currentEntries[i] const folderEntry = currentEntries[i]
const driveAction = checkResultRef.current.exists ? 'replace' : 'new' const driveAction: DriveAction = checkResultRef.current.exists ? 'replace' : 'new'
const stagingId = stagingIdRef.current const stagingId = stagingIdRef.current
if (!stagingId) { if (!stagingId) {
updateEntry(i, { status: 'error', error: 'Preparation serveur introuvable' }) updateEntry(i, { status: 'error', error: 'Preparation serveur introuvable' })
@@ -148,7 +150,7 @@ export function useUploadOrchestrator({
driveResult = await uploadDrive( driveResult = await uploadDrive(
stagingId, stagingId,
secretRef.current, secretRef.current,
driveAction as 'new' | 'replace', driveAction,
controller.signal, controller.signal,
) )
endDriveLog(driveResult.success ? 'done' : 'failed', { error: driveResult.error }) endDriveLog(driveResult.success ? 'done' : 'failed', { error: driveResult.error })
@@ -176,7 +178,7 @@ export function useUploadOrchestrator({
} }
}, [updateEntry, pushGit]) }, [updateEntry, pushGit])
const handleUpload = useCallback(async () => { const handleUpload = useCallback(async (gitModelMode: GitModelMode) => {
if (uploadActionRef.current || isChecking || isUploading) return if (uploadActionRef.current || isChecking || isUploading) return
if (!secretRef.current.trim()) { if (!secretRef.current.trim()) {
@@ -199,11 +201,12 @@ export function useUploadOrchestrator({
folderName: folder.folderName, folderName: folder.folderName,
files: 1 + folder.textures.length, files: 1 + folder.textures.length,
modelSize: folder.modelFile.size, modelSize: folder.modelFile.size,
gitModelMode,
}) })
let staged: Awaited<ReturnType<typeof stageUpload>> let staged: Awaited<ReturnType<typeof stageUpload>>
try { try {
staged = await stageUpload(folder, secretRef.current, controller.signal) staged = await stageUpload(folder, gitModelMode, secretRef.current, controller.signal)
endStageLog('done', { stagingId: staged.stagingId, filesCount: staged.filesCount }) endStageLog('done', { stagingId: staged.stagingId, filesCount: staged.filesCount })
} catch (err) { } catch (err) {
endStageLog(controller.signal.aborted ? 'cancelled' : 'failed', { endStageLog(controller.signal.aborted ? 'cancelled' : 'failed', {
+2 -39
View File
@@ -1,46 +1,9 @@
import { getAssetFamily } from './asset-naming' import { getAssetFamily } from './asset-naming'
import type { AssetCategory } from './types'
export type AssetCategory = 'color' | 'diffuse' | 'roughness' | 'normal' | 'metalness' | 'height' | 'opacity' | 'orm' | 'ao' | 'assets'
export function classifyAssetCategory(filename: string): AssetCategory { export function classifyAssetCategory(filename: string): AssetCategory {
const name = filename.replace(/\.[^.]+$/, '') const name = filename.replace(/\.[^.]+$/, '')
const family = getAssetFamily(name.split('_')[0]) const family = getAssetFamily(name.split('_')[0])
if (family === 'color') { return family || 'assets'
return 'color'
}
if (family === 'diffuse') {
return 'diffuse'
}
if (family === 'roughness') {
return 'roughness'
}
if (family === 'normal') {
return 'normal'
}
if (family === 'metalness') {
return 'metalness'
}
if (family === 'height') {
return 'height'
}
if (family === 'opacity') {
return 'opacity'
}
if (family === 'orm') {
return 'orm'
}
if (family === 'ao') {
return 'ao'
}
return 'assets'
} }
+3 -3
View File
@@ -10,7 +10,7 @@ export const ASSET_FAMILIES = [
'ao', 'ao',
] as const ] as const
export type AssetFamily = typeof ASSET_FAMILIES[number] type AssetFamily = typeof ASSET_FAMILIES[number]
const ASSET_FAMILY_BY_KEY = new Map(ASSET_FAMILIES.map((family) => [family.toLowerCase(), family])) const ASSET_FAMILY_BY_KEY = new Map(ASSET_FAMILIES.map((family) => [family.toLowerCase(), family]))
const FORBIDDEN_ASSET_FAMILY_ALIASES: ReadonlyMap<string, AssetFamily> = new Map([ const FORBIDDEN_ASSET_FAMILY_ALIASES: ReadonlyMap<string, AssetFamily> = new Map([
@@ -44,7 +44,7 @@ export function getAssetFamily(value: string): AssetFamily | undefined {
return ASSET_FAMILY_BY_KEY.get(value.toLowerCase()) return ASSET_FAMILY_BY_KEY.get(value.toLowerCase())
} }
export function getForbiddenAssetFamilyAlias(value: string): AssetFamily | undefined { function getForbiddenAssetFamilyAlias(value: string): AssetFamily | undefined {
return FORBIDDEN_ASSET_FAMILY_ALIASES.get(value.toLowerCase()) return FORBIDDEN_ASSET_FAMILY_ALIASES.get(value.toLowerCase())
} }
@@ -143,6 +143,6 @@ export function getTextureNamingError(filename: string) {
return `Asset inconnu : ${filename}. Familles autorisees : ${formatAssetFamilies()}. Utilisez asset.png pour tout le modele ou asset_objet.png pour cibler un objet.` return `Asset inconnu : ${filename}. Familles autorisees : ${formatAssetFamilies()}. Utilisez asset.png pour tout le modele ou asset_objet.png pour cibler un objet.`
} }
export function formatAssetFamilies() { function formatAssetFamilies() {
return ASSET_FAMILIES.join(', ') return ASSET_FAMILIES.join(', ')
} }
+50
View File
@@ -0,0 +1,50 @@
import { join } from 'path'
import { existsSync } from 'fs'
import { execFile } from 'child_process'
import { promisify } from 'util'
const execFileAsync = promisify(execFile)
/**
* Compress a GLTF/GLB model using Blender's Draco compression.
* Returns { success: true } on success, or { success: false, error } on failure.
* GLB Draco is explicit: callers should fail instead of silently pushing heavy assets.
*/
export async function compressWithBlender(
inputPath: string,
outputPath: string,
): Promise<{ success: boolean; error?: string }> {
const blenderPath = process.env.BLENDER_PATH || 'blender'
const timeout = Number(process.env.BLENDER_TIMEOUT_MS || 600_000)
const scriptPath = join(process.cwd(), 'scripts', 'compress.py')
if (!existsSync(scriptPath)) {
return { success: false, error: 'scripts/compress.py introuvable' }
}
try {
await execFileAsync(
blenderPath,
[
'--background',
'--python', scriptPath,
'--',
'-i', inputPath,
'-o', outputPath,
'--draco-level', '7',
'--texture-size', '512',
'-q',
],
{ timeout },
)
if (!existsSync(outputPath)) {
return { success: false, error: "Blender n'a pas produit de fichier compresse" }
}
return { success: true }
} catch (err) {
const message = err instanceof Error ? err.message : String(err)
return { success: false, error: `Compression Blender echouee: ${message}` }
}
}
+1 -1
View File
@@ -4,6 +4,6 @@ export function revokeEntryUrls(entry: FolderEntry) {
const urls = new Set<string>() const urls = new Set<string>()
if (entry.modelUrl) urls.add(entry.modelUrl) if (entry.modelUrl) urls.add(entry.modelUrl)
Object.values(entry.assetUrls || {}).forEach((url) => urls.add(url)) Object.values(entry.assetUrls).forEach((url) => urls.add(url))
urls.forEach((url) => URL.revokeObjectURL(url)) urls.forEach((url) => URL.revokeObjectURL(url))
} }
+1 -5
View File
@@ -1,7 +1,3 @@
// ---------------------------------------------------------------------------
// Client-side types — used by components and hooks (no Node.js Buffer)
// ---------------------------------------------------------------------------
type FileStatus = 'pending' | 'uploading' | 'success' | 'error' type FileStatus = 'pending' | 'uploading' | 'success' | 'error'
export interface TextureFile { export interface TextureFile {
@@ -20,7 +16,7 @@ export interface FolderEntry {
error?: string error?: string
filename?: string filename?: string
modelUrl?: string modelUrl?: string
assetUrls?: Record<string, string> assetUrls: Record<string, string>
viewerOpen?: boolean viewerOpen?: boolean
warnings: string[] warnings: string[]
driveStatus?: DriveStatus driveStatus?: DriveStatus
+31 -14
View File
@@ -1,6 +1,23 @@
import type { AssetCategory } from './asset-classification' import { ASSET_FAMILIES } from './asset-naming'
import type { FileChange } from './types' import type { AssetCategory, FileChange, PreparedAssetSummary } from './types'
import type { PreparedAssetSummary } from './types'
const ASSET_SECTION_ORDER: AssetCategory[] = [...ASSET_FAMILIES, 'assets']
function getChangePrefix(change: FileChange) {
if (change === 'new') return '✅'
if (change === 'changed') return '🔄'
return null
}
function addGroupedAssetLine(
grouped: Map<AssetCategory, string[]>,
category: AssetCategory,
line: string,
) {
const current = grouped.get(category) || []
current.push(line)
grouped.set(category, current)
}
/** /**
* Build a formatted commit message based on the upload context. * Build a formatted commit message based on the upload context.
@@ -25,7 +42,6 @@ export function buildCommitMessage(
const lines: string[] = [title, ''] const lines: string[] = [title, '']
// Model section — show status for new, changed, or unchanged
const modelSummary = assetSummaries.find((asset) => asset.kind === 'model') const modelSummary = assetSummaries.find((asset) => asset.kind === 'model')
const modelChange = fileChanges.get(modelFilename.toLowerCase()) const modelChange = fileChanges.get(modelFilename.toLowerCase())
if (modelChange === 'new') { if (modelChange === 'new') {
@@ -45,15 +61,16 @@ export function buildCommitMessage(
if (asset.kind === 'model' || !asset.category) continue if (asset.kind === 'model' || !asset.category) continue
const change = fileChanges.get(asset.filename.toLowerCase()) const change = fileChanges.get(asset.filename.toLowerCase())
if (change === 'new') { if (!change) continue
const current = grouped.get(asset.category) || []
current.push(`${asset.filename}${asset.compressed ? ' (compressed)' : ''}`) const prefix = getChangePrefix(change)
grouped.set(asset.category, current) if (!prefix) continue
} else if (change === 'changed') {
const current = grouped.get(asset.category) || [] addGroupedAssetLine(
current.push(` 🔄 ${asset.filename}${asset.compressed ? ' (compressed)' : ''}`) grouped,
grouped.set(asset.category, current) asset.category,
} ` ${prefix} ${asset.filename}${asset.compressed ? ' (compressed)' : ''}`,
)
} }
const sectionTitles: Record<AssetCategory, string> = { const sectionTitles: Record<AssetCategory, string> = {
@@ -69,7 +86,7 @@ export function buildCommitMessage(
assets: '🧩 Assets', assets: '🧩 Assets',
} }
for (const category of ['color', 'diffuse', 'roughness', 'normal', 'metalness', 'height', 'opacity', 'orm', 'ao', 'assets'] as const) { for (const category of ASSET_SECTION_ORDER) {
const entries = grouped.get(category) const entries = grouped.get(category)
if (!entries || entries.length === 0) continue if (!entries || entries.length === 0) continue
lines.push('') lines.push('')
+1 -1
View File
@@ -4,7 +4,7 @@ export const ASSET_EXTENSIONS = new Set(['.bin'])
export const ALL_ALLOWED_EXTENSIONS = new Set([...MODEL_EXTENSIONS, ...TEXTURE_EXTENSIONS, ...ASSET_EXTENSIONS]) export const ALL_ALLOWED_EXTENSIONS = new Set([...MODEL_EXTENSIONS, ...TEXTURE_EXTENSIONS, ...ASSET_EXTENSIONS])
/** Extensions tracked by Git LFS (must match .gitattributes) */ /** Extensions tracked by Git LFS (must match .gitattributes) */
export const LFS_EXTENSIONS = new Set(['.gltf', '.bin', '.png', '.jpg', '.jpeg', '.webp']) export const LFS_EXTENSIONS = new Set(['.glb', '.gltf', '.bin', '.png', '.jpg', '.jpeg', '.webp'])
export const TMP_DIR = '/tmp/assets' export const TMP_DIR = '/tmp/assets'
+1 -3
View File
@@ -13,9 +13,7 @@ interface DiffResult {
* the remote file map. * the remote file map.
* *
* Rules: * Rules:
* - Models: always re-pushed, * - Models: always re-pushed, but marked as unchanged when the remote file exists.
* but marked as 'unchanged' in the commit message when the folder already
* exists (we keep the current behavior of always delivering the model file).
* - Textures: compared by size (not compressed, reliable). * - Textures: compared by size (not compressed, reliable).
* - Orphan remote files: classified as deletions. * - Orphan remote files: classified as deletions.
*/ */
-4
View File
@@ -1,7 +1,3 @@
// ---------------------------------------------------------------------------
// Format bytes to human-readable string
// ---------------------------------------------------------------------------
export function formatBytes(bytes: number): string { export function formatBytes(bytes: number): string {
if (bytes <= 0) return '0 B' if (bytes <= 0) return '0 B'
const k = 1024 const k = 1024
+4 -2
View File
@@ -6,8 +6,10 @@ import type { PushFile, RemoteFile } from './types'
const LFS_BATCH_SIZE = 100 const LFS_BATCH_SIZE = 100
type LogDetails = Record<string, string | number | boolean | undefined>
function isHttpError(err: unknown): err is { status: number } { function isHttpError(err: unknown): err is { status: number } {
return typeof err === 'object' && err !== null && 'status' in err && typeof (err as Record<string, unknown>).status === 'number' return isRecord(err) && typeof err.status === 'number'
} }
function getOctokit(): Octokit { function getOctokit(): Octokit {
@@ -51,7 +53,7 @@ function formatElapsed(startedAt: number) {
return `${((performance.now() - startedAt) / 1000).toFixed(1)}s` return `${((performance.now() - startedAt) / 1000).toFixed(1)}s`
} }
function logInfo(step: string, action: string, startedAt: number, details?: Record<string, unknown>) { function logInfo(step: string, action: string, startedAt: number, details?: LogDetails) {
console.info(`[INFO] ${step} -> ${action} | Timer: ${formatElapsed(startedAt)}`, details || '') console.info(`[INFO] ${step} -> ${action} | Timer: ${formatElapsed(startedAt)}`, details || '')
} }
+9 -2
View File
@@ -3,11 +3,17 @@ import { NextRequest } from 'next/server'
import { sanitizeFilename } from './sanitize' import { sanitizeFilename } from './sanitize'
import { ALL_ALLOWED_EXTENSIONS, MODEL_EXTENSIONS, MAX_FILE_SIZE, TEXTURE_EXTENSIONS } from './constants' import { ALL_ALLOWED_EXTENSIONS, MODEL_EXTENSIONS, MAX_FILE_SIZE, TEXTURE_EXTENSIONS } from './constants'
import { getTextureNamingError } from './asset-naming' import { getTextureNamingError } from './asset-naming'
import type { ParsedFile } from './types' import type { GitModelMode, ParsedFile } from './types'
interface ParsedUpload { interface ParsedUpload {
folderName: string folderName: string
files: ParsedFile[] files: ParsedFile[]
gitModelMode: GitModelMode
}
function parseGitModelMode(value: FormDataEntryValue | null): GitModelMode {
if (value === 'draco-glb' || value === 'keep-gltf') return value
throw new Error('Mode Git invalide')
} }
export async function parseMultiUpload(req: NextRequest): Promise<ParsedUpload> { export async function parseMultiUpload(req: NextRequest): Promise<ParsedUpload> {
@@ -15,6 +21,7 @@ export async function parseMultiUpload(req: NextRequest): Promise<ParsedUpload>
const folderValue = formData.get('folderName') const folderValue = formData.get('folderName')
const folderName = typeof folderValue === 'string' ? folderValue.trim() || 'assets' : 'assets' const folderName = typeof folderValue === 'string' ? folderValue.trim() || 'assets' : 'assets'
const safeFolderName = sanitizeFilename(folderName).replace(/[^a-zA-Z0-9-_]/g, '-') const safeFolderName = sanitizeFilename(folderName).replace(/[^a-zA-Z0-9-_]/g, '-')
const gitModelMode = parseGitModelMode(formData.get('gitModelMode'))
const rawFiles = formData.getAll('files') const rawFiles = formData.getAll('files')
const fileTypes = formData.getAll('fileTypes').filter((value): value is string => typeof value === 'string') const fileTypes = formData.getAll('fileTypes').filter((value): value is string => typeof value === 'string')
@@ -95,5 +102,5 @@ export async function parseMultiUpload(req: NextRequest): Promise<ParsedUpload>
throw new Error('Un seul fichier model.gltf est autorise') throw new Error('Un seul fichier model.gltf est autorise')
} }
return { folderName: safeFolderName, files: parsed } return { folderName: safeFolderName, files: parsed, gitModelMode }
} }
+80 -24
View File
@@ -1,23 +1,22 @@
import { extname } from 'path' import { randomUUID } from 'crypto'
import { existsSync } from 'fs'
import { mkdir, readFile, rm, writeFile } from 'fs/promises'
import { extname, join } from 'path'
import { compressWithBlender } from '@/lib/blender'
import { compressTextureBuffer } from '@/lib/texture-compression' import { compressTextureBuffer } from '@/lib/texture-compression'
import { classifyAssetCategory } from '@/lib/asset-classification' import { classifyAssetCategory } from '@/lib/asset-classification'
import { normalizeTextureFilename } from '@/lib/asset-naming' import { normalizeTextureFilename } from '@/lib/asset-naming'
import { TEXTURE_EXTENSIONS } from '@/lib/constants' import { TEXTURE_EXTENSIONS, TMP_DIR } from '@/lib/constants'
import { getModelAssetPath } from '@/lib/model-paths' import { getModelAssetPath } from '@/lib/model-paths'
import type { ParsedFile, PreparedAssetSummary, PushFile } from '@/lib/types' import type { GitModelMode, ParsedFile, PreparedAssetSummary, PreparedGitAssetsResult, PushFile } from '@/lib/types'
interface PrepareGitAssetsParams { interface PrepareGitAssetsParams {
folderName: string folderName: string
parsedFiles: ParsedFile[] parsedFiles: ParsedFile[]
gitModelMode: GitModelMode
} }
interface PrepareGitAssetsResult { type JsonValue = string | number | boolean | null | JsonValue[] | { [key: string]: JsonValue }
filesToPush: PushFile[]
modelFilename: string
assetSummaries: PreparedAssetSummary[]
compressed: boolean
compressionError?: string
}
function getTextureFilenameMap(parsedFiles: ParsedFile[]) { function getTextureFilenameMap(parsedFiles: ParsedFile[]) {
const filenameMap = new Map<string, string>() const filenameMap = new Map<string, string>()
@@ -51,14 +50,14 @@ function getReferencedFilename(uri: string) {
return cleanUri.split(/[\\/]/).pop()?.toLowerCase() return cleanUri.split(/[\\/]/).pop()?.toLowerCase()
} }
function rewriteGltfUris(value: unknown, filenameMap: Map<string, string>): unknown { function rewriteGltfUris(value: JsonValue, filenameMap: Map<string, string>): JsonValue {
if (Array.isArray(value)) { if (Array.isArray(value)) {
return value.map((entry) => rewriteGltfUris(entry, filenameMap)) return value.map((entry) => rewriteGltfUris(entry, filenameMap))
} }
if (!value || typeof value !== 'object') return value if (!value || typeof value !== 'object') return value
const rewritten: Record<string, unknown> = {} const rewritten: Record<string, JsonValue> = {}
for (const [key, entry] of Object.entries(value)) { for (const [key, entry] of Object.entries(value)) {
if (key === 'uri' && typeof entry === 'string') { if (key === 'uri' && typeof entry === 'string') {
@@ -76,20 +75,20 @@ function rewriteGltfUris(value: unknown, filenameMap: Map<string, string>): unkn
function prepareModelBuffer(buffer: Buffer, filenameMap: Map<string, string>) { function prepareModelBuffer(buffer: Buffer, filenameMap: Map<string, string>) {
if (filenameMap.size === 0) return buffer if (filenameMap.size === 0) return buffer
const parsed: unknown = JSON.parse(buffer.toString('utf-8')) const parsed = JSON.parse(buffer.toString('utf-8')) as JsonValue
return Buffer.from(JSON.stringify(rewriteGltfUris(parsed, filenameMap), null, 2), 'utf-8') return Buffer.from(JSON.stringify(rewriteGltfUris(parsed, filenameMap), null, 2), 'utf-8')
} }
export async function prepareGitAssets({ async function prepareSeparateFiles(
folderName, folderName: string,
parsedFiles, parsedFiles: ParsedFile[],
}: PrepareGitAssetsParams): Promise<PrepareGitAssetsResult> { textureFilenameMap: Map<string, string>,
) {
const filesToPush: PushFile[] = [] const filesToPush: PushFile[] = []
const assetSummaries: PreparedAssetSummary[] = [] const assetSummaries: PreparedAssetSummary[] = []
let modelFilename = '' let modelFilename = ''
let compressed = false let compressed = false
let compressionError: string | undefined let compressionError: string | undefined
const textureFilenameMap = getTextureFilenameMap(parsedFiles)
for (const pf of parsedFiles) { for (const pf of parsedFiles) {
let content = pf.buffer let content = pf.buffer
@@ -131,11 +130,68 @@ export async function prepareGitAssets({
}) })
} }
return { return { filesToPush, modelFilename, assetSummaries, compressed, compressionError }
filesToPush, }
modelFilename,
assetSummaries, async function prepareDracoGlb(
compressed, folderName: string,
compressionError, parsedFiles: ParsedFile[],
textureFilenameMap: Map<string, string>,
): Promise<PreparedGitAssetsResult> {
const tmpFolder = join(TMP_DIR, 'blender', `${folderName}-${randomUUID()}`)
const inputModelPath = join(tmpFolder, 'model.gltf')
const outputModelPath = join(tmpFolder, 'model.glb')
await mkdir(tmpFolder, { recursive: true })
try {
for (const pf of parsedFiles) {
if (pf.isModel) {
await writeFile(inputModelPath, prepareModelBuffer(pf.buffer, textureFilenameMap))
continue
}
const filename = textureFilenameMap.get(pf.filename.toLowerCase()) || pf.filename
const ext = extname(filename).toLowerCase()
const content = TEXTURE_EXTENSIONS.has(ext)
? (await compressTextureBuffer(filename, pf.buffer)).buffer
: pf.buffer
await writeFile(join(tmpFolder, filename), content)
}
const result = await compressWithBlender(inputModelPath, outputModelPath)
if (!result.success || !existsSync(outputModelPath)) {
throw new Error(result.error || 'Compression Blender echouee')
}
const content = await readFile(outputModelPath)
const modelFilename = 'model.glb'
return {
filesToPush: [{
path: getModelAssetPath(folderName, modelFilename),
contentBase64: content.toString('base64'),
}],
modelFilename,
assetSummaries: [{ filename: modelFilename, kind: 'model', compressed: true }],
compressed: true,
}
} finally {
await rm(tmpFolder, { recursive: true, force: true }).catch(() => {})
} }
} }
export async function prepareGitAssets({
folderName,
parsedFiles,
gitModelMode,
}: PrepareGitAssetsParams): Promise<PreparedGitAssetsResult> {
const textureFilenameMap = getTextureFilenameMap(parsedFiles)
if (gitModelMode === 'keep-gltf') {
return prepareSeparateFiles(folderName, parsedFiles, textureFilenameMap)
}
return prepareDracoGlb(folderName, parsedFiles, textureFilenameMap)
}
+24 -2
View File
@@ -1,5 +1,3 @@
import type { AssetCategory } from './asset-classification'
export interface ParsedFile { export interface ParsedFile {
filename: string filename: string
buffer: Buffer buffer: Buffer
@@ -11,8 +9,14 @@ export interface PushFile {
contentBase64: string contentBase64: string
} }
export type GitModelMode = 'draco-glb' | 'keep-gltf'
export type DriveAction = 'new' | 'replace'
export type FileChange = 'new' | 'changed' | 'unchanged' export type FileChange = 'new' | 'changed' | 'unchanged'
export type AssetCategory = 'color' | 'diffuse' | 'roughness' | 'normal' | 'metalness' | 'height' | 'opacity' | 'orm' | 'ao' | 'assets'
export interface FileDiff { export interface FileDiff {
name: string name: string
status: 'changed' | 'new' | 'deleted' status: 'changed' | 'new' | 'deleted'
@@ -29,3 +33,21 @@ export interface PreparedAssetSummary {
category?: AssetCategory category?: AssetCategory
compressed: boolean compressed: boolean
} }
export interface StagingUploadResult {
stagingId: string
folderName: string
filesCount: number
}
export interface PreparedGitAssetsResult {
filesToPush: PushFile[]
modelFilename: string
assetSummaries: PreparedAssetSummary[]
compressed: boolean
compressionError?: string
}
export interface PreparedStageAssetsResult extends PreparedGitAssetsResult {
folderName: string
}
+23 -25
View File
@@ -1,31 +1,37 @@
import { isRecord } from './guards' import { isRecord } from './guards'
import type { FolderEntry } from './client-types' import type { FolderEntry } from './client-types'
import type { FileDiff } from './types' import type { DriveAction, FileDiff, GitModelMode, StagingUploadResult } from './types'
export interface CheckResult { export interface CheckResult {
exists: boolean exists: boolean
diffs: FileDiff[] diffs: FileDiff[]
} }
interface StageResult {
stagingId: string
folderName: string
filesCount: number
}
function getApiError(data: unknown, fallback: string) { function getApiError(data: unknown, fallback: string) {
return isRecord(data) && typeof data.error === 'string' ? data.error : fallback return isRecord(data) && typeof data.error === 'string' ? data.error : fallback
} }
function getUploadJsonHeaders(secret: string) {
return {
'Content-Type': 'application/json',
'x-upload-secret': secret.trim(),
}
}
function isAbortError(err: unknown) {
return err instanceof DOMException && err.name === 'AbortError'
}
function isFileDiff(value: unknown): value is FileDiff { function isFileDiff(value: unknown): value is FileDiff {
return isRecord(value) return isRecord(value)
&& typeof value.name === 'string' && typeof value.name === 'string'
&& (value.status === 'new' || value.status === 'changed' || value.status === 'deleted') && (value.status === 'new' || value.status === 'changed' || value.status === 'deleted')
} }
function buildUploadFormData(folder: FolderEntry): FormData { function buildUploadFormData(folder: FolderEntry, gitModelMode: GitModelMode): FormData {
const formData = new FormData() const formData = new FormData()
formData.append('folderName', folder.folderName) formData.append('folderName', folder.folderName)
formData.append('gitModelMode', gitModelMode)
formData.append('files', folder.modelFile) formData.append('files', folder.modelFile)
formData.append('fileTypes', 'model') formData.append('fileTypes', 'model')
@@ -47,10 +53,7 @@ export async function checkFolderDiffs(
): Promise<CheckResult> { ): Promise<CheckResult> {
const res = await fetch('/api/upload/check', { const res = await fetch('/api/upload/check', {
method: 'POST', method: 'POST',
headers: { headers: getUploadJsonHeaders(secret),
'Content-Type': 'application/json',
'x-upload-secret': secret.trim(),
},
body: JSON.stringify({ stagingId }), body: JSON.stringify({ stagingId }),
signal, signal,
}) })
@@ -72,10 +75,11 @@ export async function checkFolderDiffs(
export async function stageUpload( export async function stageUpload(
folder: FolderEntry, folder: FolderEntry,
gitModelMode: GitModelMode,
secret: string, secret: string,
signal?: AbortSignal, signal?: AbortSignal,
): Promise<StageResult> { ): Promise<StagingUploadResult> {
const formData = buildUploadFormData(folder) const formData = buildUploadFormData(folder, gitModelMode)
const res = await fetch('/api/upload/stage', { const res = await fetch('/api/upload/stage', {
method: 'POST', method: 'POST',
headers: { 'x-upload-secret': secret.trim() }, headers: { 'x-upload-secret': secret.trim() },
@@ -103,16 +107,13 @@ export async function stageUpload(
export async function uploadDrive( export async function uploadDrive(
stagingId: string, stagingId: string,
secret: string, secret: string,
action: 'new' | 'replace', action: DriveAction,
signal?: AbortSignal, signal?: AbortSignal,
): Promise<{ success: boolean; error?: string }> { ): Promise<{ success: boolean; error?: string }> {
try { try {
const res = await fetch('/api/upload/drive', { const res = await fetch('/api/upload/drive', {
method: 'POST', method: 'POST',
headers: { headers: getUploadJsonHeaders(secret),
'Content-Type': 'application/json',
'x-upload-secret': secret.trim(),
},
body: JSON.stringify({ stagingId, action }), body: JSON.stringify({ stagingId, action }),
signal, signal,
}) })
@@ -122,7 +123,7 @@ export async function uploadDrive(
} }
return { success: true } return { success: true }
} catch (err) { } catch (err) {
if (err instanceof DOMException && err.name === 'AbortError') { if (isAbortError(err)) {
return { success: false, error: 'Upload annule' } return { success: false, error: 'Upload annule' }
} }
return { success: false, error: 'Erreur reseau (Drive)' } return { success: false, error: 'Erreur reseau (Drive)' }
@@ -140,10 +141,7 @@ export async function uploadGit(
try { try {
const res = await fetch('/api/upload/git', { const res = await fetch('/api/upload/git', {
method: 'POST', method: 'POST',
headers: { headers: getUploadJsonHeaders(secret),
'Content-Type': 'application/json',
'x-upload-secret': secret.trim(),
},
body: JSON.stringify({ stagingId }), body: JSON.stringify({ stagingId }),
signal, signal,
}) })
@@ -158,7 +156,7 @@ export async function uploadGit(
onProgress(100) onProgress(100)
return { success: true, filename: typeof data.folderName === 'string' ? data.folderName : undefined } return { success: true, filename: typeof data.folderName === 'string' ? data.folderName : undefined }
} catch (err) { } catch (err) {
if (err instanceof DOMException && err.name === 'AbortError') { if (isAbortError(err)) {
return { success: false, error: 'Upload annule' } return { success: false, error: 'Upload annule' }
} }
return { success: false, error: 'Erreur reseau' } return { success: false, error: 'Erreur reseau' }
+1 -2
View File
@@ -1,6 +1,5 @@
import { isRecord } from './guards' import { isRecord } from './guards'
import type { DriveAction } from './types'
type DriveAction = 'new' | 'replace'
interface StagingRequestBody { interface StagingRequestBody {
stagingId: string stagingId: string
+23 -16
View File
@@ -5,7 +5,14 @@ import { existsSync } from 'fs'
import { TMP_DIR } from '@/lib/constants' import { TMP_DIR } from '@/lib/constants'
import { getModelAssetPath } from '@/lib/model-paths' import { getModelAssetPath } from '@/lib/model-paths'
import { prepareGitAssets } from '@/lib/prepare-git-assets' import { prepareGitAssets } from '@/lib/prepare-git-assets'
import type { ParsedFile, PreparedAssetSummary, PushFile } from '@/lib/types' import type {
GitModelMode,
ParsedFile,
PreparedAssetSummary,
PreparedStageAssetsResult,
PushFile,
StagingUploadResult,
} from '@/lib/types'
const STAGING_ROOT = join(TMP_DIR, 'staging') const STAGING_ROOT = join(TMP_DIR, 'staging')
const STAGING_TTL_MS = 60 * 60 * 1000 const STAGING_TTL_MS = 60 * 60 * 1000
@@ -26,20 +33,12 @@ interface StagedPreparedData {
interface StagingManifest { interface StagingManifest {
stagingId: string stagingId: string
folderName: string folderName: string
gitModelMode: GitModelMode
createdAt: number createdAt: number
originals: StagedOriginalFile[] originals: StagedOriginalFile[]
prepared?: StagedPreparedData prepared?: StagedPreparedData
} }
interface PreparedStageAssetsResult {
folderName: string
filesToPush: PushFile[]
modelFilename: string
assetSummaries: PreparedAssetSummary[]
compressed: boolean
compressionError?: string
}
function getStageDir(stagingId: string) { function getStageDir(stagingId: string) {
return join(STAGING_ROOT, stagingId) return join(STAGING_ROOT, stagingId)
} }
@@ -87,7 +86,11 @@ async function cleanupExpiredStagingUploads() {
} }
} }
export async function createStagingUpload(folderName: string, parsedFiles: ParsedFile[]) { export async function createStagingUpload(
folderName: string,
parsedFiles: ParsedFile[],
gitModelMode: GitModelMode,
): Promise<StagingUploadResult> {
await cleanupExpiredStagingUploads() await cleanupExpiredStagingUploads()
const stagingId = randomUUID() const stagingId = randomUUID()
@@ -106,6 +109,7 @@ export async function createStagingUpload(folderName: string, parsedFiles: Parse
const manifest: StagingManifest = { const manifest: StagingManifest = {
stagingId, stagingId,
folderName, folderName,
gitModelMode,
createdAt: Date.now(), createdAt: Date.now(),
originals, originals,
} }
@@ -137,12 +141,11 @@ async function readOriginalParsedFiles(stagingId: string, manifest: StagingManif
) )
} }
async function buildPreparedPushFiles(stagingId: string, manifest: StagingManifest): Promise<PushFile[]> { async function buildPreparedPushFiles(stagingId: string, manifest: StagingManifest, prepared: StagedPreparedData): Promise<PushFile[]> {
const preparedDir = getPreparedDir(stagingId) const preparedDir = getPreparedDir(stagingId)
const preparedFiles = manifest.prepared?.assetSummaries || []
return Promise.all( return Promise.all(
preparedFiles.map(async (file) => { prepared.assetSummaries.map(async (file) => {
const buffer = await readFile(join(preparedDir, file.filename)) const buffer = await readFile(join(preparedDir, file.filename))
return { return {
path: getModelAssetPath(manifest.folderName, file.filename), path: getModelAssetPath(manifest.folderName, file.filename),
@@ -157,7 +160,11 @@ export async function ensurePreparedStagingAssets(stagingId: string): Promise<Pr
if (!manifest.prepared) { if (!manifest.prepared) {
const parsedFiles = await readOriginalParsedFiles(stagingId, manifest) const parsedFiles = await readOriginalParsedFiles(stagingId, manifest)
const prepared = await prepareGitAssets({ folderName: manifest.folderName, parsedFiles }) const prepared = await prepareGitAssets({
folderName: manifest.folderName,
parsedFiles,
gitModelMode: manifest.gitModelMode,
})
const preparedDir = getPreparedDir(stagingId) const preparedDir = getPreparedDir(stagingId)
await mkdir(preparedDir, { recursive: true }) await mkdir(preparedDir, { recursive: true })
@@ -181,7 +188,7 @@ export async function ensurePreparedStagingAssets(stagingId: string): Promise<Pr
return { return {
folderName: manifest.folderName, folderName: manifest.folderName,
filesToPush: await buildPreparedPushFiles(stagingId, manifest), filesToPush: await buildPreparedPushFiles(stagingId, manifest, manifest.prepared),
modelFilename: manifest.prepared.modelFilename, modelFilename: manifest.prepared.modelFilename,
assetSummaries: manifest.prepared.assetSummaries, assetSummaries: manifest.prepared.assetSummaries,
compressed: manifest.prepared.compressed, compressed: manifest.prepared.compressed,
+1 -1
View File
@@ -14,7 +14,7 @@ interface GltfJson {
} }
/** Discriminated union: either valid (with model) or invalid (with errors). */ /** Discriminated union: either valid (with model) or invalid (with errors). */
export type ValidationResult = type ValidationResult =
| { ok: true; model: File; textures: TextureFile[]; warnings: string[] } | { ok: true; model: File; textures: TextureFile[]; warnings: string[] }
| { ok: false; errors: string[] } | { ok: false; errors: string[] }
+422
View File
@@ -0,0 +1,422 @@
#!/usr/bin/env python3
"""
Blender Draco Compression Script
CLI tool to compress 3D meshes with Draco compression using Blender
Usage:
blender --background --python compress.py -- [options]
Options:
-i, --input FILE Input file (required in advanced mode)
-o, --output FILE Output file (default: input_compressed.glb)
--draco-level LEVEL Draco compression level 0-10 (default: 7)
--resize-textures / --no-resize Enable/disable texture resizing (default: enabled)
--texture-size SIZE Max texture size in pixels (default: 512)
--batch Batch mode: input is a directory
--output-dir DIR Output directory for batch mode
--format FORMAT Output format: glb or gltf (default: glb)
-q, --quiet Quiet mode (less output)
-h, --help Show this help message
Examples:
# Simple mode (all defaults)
blender --background --python compress.py -- input.glb
# Advanced mode
blender --background --python compress.py -- -i input.glb -o output.glb --draco-level 10
# Batch mode
blender --background --python compress.py -- --batch ./models/ --output-dir ./compressed/
"""
import os
import sys
import io
import argparse
from contextlib import redirect_stdout
from pathlib import Path
import bpy
try:
import bpy_types
except ImportError:
bpy_types = None
SUPPORTED_IMPORT_FORMATS = {
'.glb': 'gltf',
'.gltf': 'gltf',
'.obj': 'obj',
'.ply': 'ply',
'.stl': 'stl',
'.x3d': 'x3d',
'.wrl': 'x3d',
'.3ds': '3ds',
'.fbx': 'fbx',
'.dae': 'dae',
}
SUPPORTED_OUTPUT_FORMATS = ['glb', 'gltf']
def file_name(filepath):
return os.path.split(filepath)[1]
def file_suffix(filepath):
return os.path.splitext(file_name(filepath))[1].lower()
def dir_path(filepath):
return os.path.split(filepath)[0]
def get_import_operator(suffix):
operators = {
'gltf': bpy.ops.import_scene.gltf,
'obj': bpy.ops.import_scene.obj,
'ply': bpy.ops.import_mesh.ply,
'stl': bpy.ops.import_mesh.stl,
'x3d': bpy.ops.import_scene.x3d,
'3ds': bpy.ops.import_scene.fbx,
'fbx': bpy.ops.import_scene.fbx,
'dae': bpy.ops.import_scene.dae,
}
return operators.get(suffix)
def get_output_extension(format_type):
return '.glb' if format_type == 'glb' else '.gltf'
def import_mesh(filepath):
suffix = file_suffix(filepath)
if suffix not in SUPPORTED_IMPORT_FORMATS:
raise ValueError(f"Unsupported input format: {suffix}")
format_type = SUPPORTED_IMPORT_FORMATS[suffix]
import_op = get_import_operator(format_type)
if import_op is None:
raise ValueError(f"Cannot import {suffix} format")
stdout_buffer = io.StringIO()
with redirect_stdout(stdout_buffer):
import_op(filepath=str(filepath))
output = stdout_buffer.getvalue()
return output
def clear_scene():
bpy.ops.object.select_all(action='SELECT')
bpy.ops.object.delete()
return len(bpy.data.objects) == 0
def resize_textures(target_size):
resized_count = 0
for image in bpy.data.images:
if image.size[0] > target_size or image.size[1] > target_size:
old_width = image.size[0]
old_height = image.size[1]
scale = min(target_size / old_width, target_size / old_height)
new_width = int(old_width * scale)
new_height = int(old_height * scale)
image.scale(new_width, new_height)
resized_count += 1
print(f" Resized '{image.name}': {old_width}x{old_height} -> {new_width}x{new_height}")
return resized_count
def export_mesh(filepath, draco_level=7, format_type='glb'):
export_kwargs = {
'filepath': str(filepath),
'export_draco_mesh_compression_enable': True,
'export_draco_mesh_compression_level': draco_level,
'export_format': 'GLB' if format_type == 'glb' else 'GLTF_SEPARATE',
}
stdout_buffer = io.StringIO()
with redirect_stdout(stdout_buffer):
bpy.ops.export_scene.gltf(**export_kwargs)
return stdout_buffer.getvalue()
def get_default_output(input_path, format_type='glb'):
input_file = Path(input_path)
suffix = get_output_extension(format_type)
return str(input_file.parent / f"{input_file.stem}_compressed{suffix}")
def process_file(input_path, output_path=None, draco_level=7,
resize_textures_flag=True, texture_size=512,
format_type='glb', quiet=False):
if not quiet:
print(f"\n{'='*50}")
print(f"Processing: {input_path}")
if not os.path.exists(input_path):
raise FileNotFoundError(f"Input file not found: {input_path}")
if not quiet:
original_size = os.path.getsize(input_path)
print(f"Original size: {original_size / 1024:.2f} KB")
if not clear_scene():
raise RuntimeError("Failed to clear Blender scene")
if not quiet:
print("Importing mesh...")
import_mesh(input_path)
if len(bpy.data.objects) == 0:
raise RuntimeError(f"No objects imported from {input_path}")
if not quiet:
mesh_count = sum(1 for obj in bpy.data.objects if isinstance(obj.data, bpy.types.Mesh))
print(f"Imported {mesh_count} mesh(es)")
if resize_textures_flag:
if not quiet:
print(f"Resizing textures (max: {texture_size}px)...")
resized = resize_textures(texture_size)
if not quiet and resized > 0:
print(f"Resized {resized} texture(s)")
if output_path is None:
output_path = get_default_output(input_path, format_type)
os.makedirs(os.path.dirname(output_path) or '.', exist_ok=True)
if not quiet:
print(f"Exporting with Draco compression (level={draco_level})...")
export_mesh(output_path, draco_level, format_type)
if not os.path.exists(output_path):
raise RuntimeError(f"Export failed: {output_path} not created")
final_size = os.path.getsize(output_path)
if not quiet:
original_size = os.path.getsize(input_path) if os.path.exists(input_path) else 0
reduction = ((original_size - final_size) / original_size * 100) if original_size > 0 else 0
print(f"\nOutput: {output_path}")
print(f"Final size: {final_size / 1024:.2f} KB")
if original_size > 0:
print(f"Reduction: {reduction:.1f}%")
print("Compression complete!")
return output_path, final_size
def process_batch(input_dir, output_dir=None, draco_level=7,
resize_textures_flag=True, texture_size=512,
format_type='glb', quiet=False):
if not os.path.exists(input_dir):
raise FileNotFoundError(f"Input directory not found: {input_dir}")
if output_dir:
os.makedirs(output_dir, exist_ok=True)
files_found = []
for ext in SUPPORTED_IMPORT_FORMATS.keys():
files_found.extend(Path(input_dir).glob(f"*{ext}"))
files_found.extend(Path(input_dir).glob(f"*{ext.upper()}"))
files_found = sorted(set(files_found))
if not files_found:
print(f"No supported files found in {input_dir}")
return []
if not quiet:
print(f"\n{'='*50}")
print(f"BATCH MODE")
print(f"Input directory: {input_dir}")
print(f"Files found: {len(files_found)}")
results = []
for i, file_path in enumerate(files_found, 1):
if output_dir:
input_file = Path(file_path)
suffix = get_output_extension(format_type)
output_path = os.path.join(output_dir, f"{input_file.stem}_compressed{suffix}")
else:
output_path = None
if not quiet:
print(f"\n[{i}/{len(files_found)}]")
try:
result_path, _ = process_file(
str(file_path),
output_path=output_path,
draco_level=draco_level,
resize_textures_flag=resize_textures_flag,
texture_size=texture_size,
format_type=format_type,
quiet=quiet
)
results.append((str(file_path), result_path, True, None))
except Exception as e:
error_msg = str(e)
if not quiet:
print(f"ERROR: {error_msg}")
results.append((str(file_path), None, False, error_msg))
success_count = sum(1 for _, _, success, _ in results if success)
fail_count = len(results) - success_count
if not quiet:
print(f"\n{'='*50}")
print(f"BATCH COMPLETE")
print(f"Total files: {len(results)}")
print(f"Success: {success_count}")
print(f"Failed: {fail_count}")
return results
def main():
argv = sys.argv
if "--" not in argv:
argv = []
else:
argv = argv[argv.index("--") + 1:]
parser = argparse.ArgumentParser(
description='Compress 3D meshes with Draco compression using Blender',
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Examples:
# Simple mode (all defaults)
blender --background --python compress.py -- input.glb
# With options
blender --background --python compress.py -- -i input.glb -o output.glb --draco-level 10
# Batch mode
blender --background --python compress.py -- --batch ./models/ --output-dir ./compressed/
"""
)
parser.add_argument(
'input',
nargs='?',
help='Input file or directory (for batch mode)'
)
parser.add_argument(
'-i', '--input',
dest='input_file',
help='Input file (alternative to positional argument)'
)
parser.add_argument(
'-o', '--output',
dest='output',
help='Output file (default: input_compressed.glb)'
)
parser.add_argument(
'--draco-level',
type=int,
default=7,
choices=range(0, 11),
help='Draco compression level 0-10 (default: 7)'
)
resize_group = parser.add_mutually_exclusive_group()
resize_group.add_argument(
'--resize-textures',
action='store_true',
default=True,
help='Enable texture resizing (default: enabled)'
)
resize_group.add_argument(
'--no-resize',
action='store_false',
dest='resize_textures',
help='Disable texture resizing'
)
parser.add_argument(
'--texture-size',
type=int,
default=512,
help='Max texture size in pixels (default: 512)'
)
parser.add_argument(
'--batch',
action='store_true',
help='Batch mode: process all files in input directory'
)
parser.add_argument(
'--output-dir', '-d',
dest='output_dir',
help='Output directory for batch mode'
)
parser.add_argument(
'--format', '-f',
choices=SUPPORTED_OUTPUT_FORMATS,
default='glb',
help='Output format (default: glb)'
)
parser.add_argument(
'-q', '--quiet',
action='store_true',
help='Quiet mode (less output)'
)
args = parser.parse_args(argv)
input_path = args.input or args.input_file
if not input_path:
parser.print_help()
print("\nError: Input file or directory is required")
sys.exit(1)
if args.batch or os.path.isdir(input_path):
results = process_batch(
input_path,
output_dir=args.output_dir,
draco_level=args.draco_level,
resize_textures_flag=args.resize_textures,
texture_size=args.texture_size,
format_type=args.format,
quiet=args.quiet
)
failed = [r for r in results if not r[2]]
if failed:
sys.exit(1)
else:
try:
process_file(
input_path,
output_path=args.output,
draco_level=args.draco_level,
resize_textures_flag=args.resize_textures,
texture_size=args.texture_size,
format_type=args.format,
quiet=args.quiet
)
except Exception as e:
print(f"Error: {e}")
sys.exit(1)
if __name__ == "__main__":
main()