feat: add ktx2 texture fallback flow

This commit is contained in:
Tom Boullay
2026-05-17 16:18:17 +02:00
parent 81c513ee1f
commit 3cfb3a21a9
8 changed files with 390 additions and 30 deletions
+18 -9
View File
@@ -3,7 +3,7 @@
A secure web interface for uploading `model.gltf` with its associated `.bin` file and textures with two outputs:
- **Nextcloud Drive** — Archives the original files with automatic versioning (VF/V1/V2...), so artists always have a history of past versions
- **Git remote** — Delivers Draco-compressed GLB assets by default, with an optional GLTF delivery mode for specific models
- **Git remote** — Delivers Draco-compressed GLB assets by default, with an optional GLTF delivery mode using KTX2 textures, then WebP/original fallbacks for specific models
Built for La Fabrik Durable's internal use, but open-sourced for anyone looking for a similar solution. The app validates the upload locally, stages it server-side, then compares file diffs to avoid unnecessary uploads and commits. The Drive upload serves as the source of truth and version history, while the Git upload delivers the prepared assets to developers.
@@ -14,7 +14,8 @@ Built for La Fabrik Durable's internal use, but open-sourced for anyone looking
- [**Tailwind CSS**](https://v3.tailwindcss.com/docs/installation) for styling
- [**Octokit**](https://github.com/octokit/rest.js/#readme) + provider adapters for GitHub and Gitea uploads
- [**Nextcloud WebDAV**](https://docs.nextcloud.com/server/latest/developer_manual/client_apis/WebDAV/index.html) for Drive archiving with automatic versioning
- [**Sharp**](https://sharp.pixelplumbing.com/install/) for server-side texture compression
- [**Sharp**](https://sharp.pixelplumbing.com/install/) for server-side WebP texture fallback
- [**KTX-Software**](https://github.com/KhronosGroup/KTX-Software) (`toktx`) for optional KTX2 texture delivery
- [**npm lockfile + Coolify** (Docker)](https://coolify.io/docs/applications/build-packs/dockerfile) for hosting
## Usage
@@ -87,7 +88,7 @@ Invalid or unknown asset names still block the upload.
### Upload flow: Drive first, then Git
5. **Drive upload (archiving)** — Original files from the staging area are uploaded to the Nextcloud Drive with automatic versioning (see below). This serves as the artists' source of truth and version history. If the Drive upload fails, a modal asks the user whether to send to Git only or cancel entirely.
6. **Git upload (delivery to devs)** — The prepared Git payload is reused from staging. By default, Blender exports a single `model.glb` with Draco compression. If Blender cannot process a specific model, the upload falls back to the separate `model.gltf` + `.bin` + compressed textures workflow and shows a warning. For specific models, "Envoyer en GLTF" keeps that separate GLTF delivery mode from the start.
6. **Git upload (delivery to devs)** — The prepared Git payload is reused from staging. By default, Blender exports a single `model.glb` with Draco compression. If Blender cannot process a specific model, the upload falls back to the separate `model.gltf` + `.bin` + optimized textures workflow and shows a warning. In separate GLTF delivery, each texture tries one optimized output in order: KTX2 with `KHR_texture_basisu`, then WebP through Sharp, then the original texture if all optimization fails. For specific models, "Envoyer en GLTF" keeps that separate GLTF delivery mode from the start.
### Drive versioning (Nextcloud WebDAV)
@@ -153,7 +154,7 @@ Commit sections:
Symbols: `✅` new — `🔄` modified — `↔️` unchanged (model always re-pushed) — `❌` deleted
7. Orphan files (present on remote but not in the new upload) are deleted in the same commit
8. Default Git delivery pushes `model.glb` when Blender compression succeeds. If Blender fails, the app falls back to separate `model.gltf`, `.bin`, and compressed textures with a warning. "Envoyer en GLTF" always uses separate GLTF delivery.
8. Default Git delivery pushes `model.glb` when Blender compression succeeds. If Blender fails, the app falls back to separate `model.gltf`, `.bin`, and one delivered texture per source: `.ktx2` when possible, `.webp` when KTX2 fails, or the original file when both optimizations fail. "Envoyer en GLTF" always uses separate GLTF delivery.
Uploaded models are pushed to `public/models/<folderName>/` in the target repo.
@@ -162,6 +163,7 @@ Uploaded models are pushed to `public/models/<folderName>/` in the target repo.
- Large uploads are staged once, but the Drive upload remains sequential.
- Git LFS batch uploads are sequential by batch.
- Default GLB Draco delivery reduces Git LFS usage by replacing many support files with one compressed model file.
- KTX2 is currently applied to separate GLTF delivery only. Embedding KTX2 inside the default GLB would require a glTF-aware optimizer step after Blender, not just image preprocessing before Blender.
- Uploads expect a single `model.gltf` file plus optional flat support files (`.bin`, `.png`, `.jpg`, `.jpeg`, `.webp`).
## Project Structure
@@ -221,14 +223,15 @@ lib/
├── asset-classification.ts # Group assets by family for commit messages
├── asset-naming.ts # Allowed asset families and naming convention helpers
├── blender.ts # Blender Draco compression helper
├── texture-compression.ts # KTX2/WebP texture delivery helpers
├── commit-message.ts # Commit message builder
├── parse-upload.ts # FormData parser + validation
├── validate-folder.ts # Client-side folder validation (discriminated union)
└── format-bytes.ts # Byte formatting utility
scripts/
└── compress.py # Blender Draco compression script
Dockerfile # Multi-stage build: Node 20 slim + Blender + tini
docker-entrypoint.sh # Upload temp setup + Blender availability check
Dockerfile # Multi-stage build: Node 20 slim + Blender + KTX-Software + tini
docker-entrypoint.sh # Upload temp setup + Blender/toktx availability check
```
## Installation
@@ -250,6 +253,9 @@ GIT_USERNAME=your-gitea-username
GIT_TOKEN=your-git-provider-token
GIT_BRANCH=main
GIT_REPO_URL=https://git.example.com/your-org/your-repo
# Optional texture optimization
TOKTX_PATH=toktx
TEXTURE_MAX_SIZE=1024
# Nextcloud Drive (public share WebDAV)
NEXTCLOUD_URL=https://cloud.example.com
NEXTCLOUD_SHARE_TOKEN=your-public-share-token
@@ -265,6 +271,8 @@ NEXTCLOUD_BASE_PATH=Models
| `GIT_TOKEN` | Git provider token with repository read/write access. `GITHUB_TOKEN` is still accepted for backward compatibility. | Yes |
| `GIT_BRANCH` | Target branch (default: main) | No |
| `GIT_REPO_URL` | Target GitHub or Gitea repository URL (`owner/repo`, HTTPS, or SSH) | Yes |
| `TOKTX_PATH` | Path to the `toktx` binary used for KTX2 texture generation (default: `toktx`). | No |
| `TEXTURE_MAX_SIZE` | Maximum texture dimension used for KTX2/WebP separate GLTF delivery (default: `1024`). | No |
| `NEXTCLOUD_URL` | Nextcloud instance URL | Yes |
| `NEXTCLOUD_SHARE_TOKEN` | Public share token (the part after `/s/` in the share link) | Yes |
| `NEXTCLOUD_SHARE_PASSWORD` | Public share password (empty if none) | No |
@@ -327,7 +335,7 @@ docker run -p 3000:3000 \
upload-gltf
```
The Docker image runs the Next.js app, Blender Draco compression, and server-side asset preparation in a single container. The `docker-entrypoint.sh` script creates the upload temp directory and reports Blender availability before launching the app.
The Docker image runs the Next.js app, Blender Draco compression, KTX2 texture generation, and server-side asset preparation in a single container. The `docker-entrypoint.sh` script creates the upload temp directory and reports Blender/toktx availability before launching the app.
### Secret rotation
@@ -366,9 +374,10 @@ git push origin main && git push gitea main
|------|------------|
| 3D Models | `.gltf` |
| Binary buffers | `.bin` |
| Textures | `.png`, `.jpg`, `.jpeg`, `.webp` |
| Input textures | `.png`, `.jpg`, `.jpeg`, `.webp` |
| Generated Git textures | `.ktx2`, `.webp`, or original source extension as fallback |
Git delivery outputs `.glb` by default, or keeps the source `.gltf` structure when "Envoyer en GLTF" is selected.
Git delivery outputs `.glb` by default, or keeps the source `.gltf` structure when "Envoyer en GLTF" is selected. Separate GLTF delivery tries KTX2 first, then WebP, then the original texture. When KTX2 is selected, `model.gltf` references it through `KHR_texture_basisu`.
## License