refactor: stage uploads before drive and git delivery

This commit is contained in:
Tom Boullay
2026-04-24 17:41:38 +02:00
parent 53c4c0ed60
commit 71bcf2b36d
8 changed files with 405 additions and 68 deletions
+55 -20
View File
@@ -1,9 +1,9 @@
# upload-GLTF
A secure web interface for uploading 3D assets (GLTF/GLB + textures) with two outputs:
A secure web interface for uploading `model.glb` and its associated textures with two outputs:
- **Nextcloud Drive** — Archives the original files with automatic versioning (VF/V1/V2...), so artists always have a history of past versions.
- **GitHub** — Delivers compressed models (Draco via Blender) to the dev team's repository, ready for integration.
- **GitHub** — Delivers compressed models (Draco via Blender) and compressed textures to the dev team's repository, ready for integration.
Built for La Fabrik Durable.
@@ -68,6 +68,8 @@ npm run dev
Access the app at `http://localhost:3000`
> **Note:** Draco compression requires Blender installed locally. On macOS, Blender is typically at `/Applications/Blender.app/Contents/MacOS/Blender`. Set `BLENDER_PATH` in `.env.local` accordingly. If Blender is not available, models are pushed to GitHub without compression.
>
> Local 3D preview is currently supported for `.glb` files only.
### Production (Coolify / Docker)
@@ -90,17 +92,19 @@ The Docker image includes Blender headless (installed once at build time). On st
2. They select a folder containing:
- `model.glb` (**required**)
- Any associated textures (`.png/.jpg/.jpeg/.webp`)
4. The model is displayed in a 3D preview
3. The `.glb` model is displayed in a local 3D preview
5. On clicking "Envoyer":
- The app checks the remote Git repo for existing files and computes diffs (textures by size, models always re-pushed)
- The app uploads the folder once to a temporary server-side staging area
- The app prepares the final Git payload from this staging area
- The app checks the remote Git repo for existing files and computes diffs
- If the folder doesn't exist, upload proceeds directly
- If the folder exists and files differ, a confirmation dialog shows **only the actual changes**
- If nothing changed, the upload is skipped entirely
### Upload flow: Drive first, then Git
6. **Drive upload (archiving)** — Original files (before Blender compression) are uploaded to the Nextcloud Drive with automatic versioning (see below). This serves as the artists' source of truth and version history. If the Drive upload fails, a modal asks the user whether to send to Git only or cancel entirely.
7. **Git upload (delivery to devs)**Models are compressed with Blender Draco, then all changed files are pushed to GitHub in a single commit. This is what the dev team consumes in the application.
6. **Drive upload (archiving)** — Original files from the staging area are uploaded to the Nextcloud Drive with automatic versioning (see below). This serves as the artists' source of truth and version history. If the Drive upload fails, a modal asks the user whether to send to Git only or cancel entirely.
7. **Git upload (delivery to devs)**The prepared Git payload is reused from staging: models are compressed with Blender Draco, textures are compressed server-side, then all changed files are pushed to GitHub in a single commit. This is what the dev team consumes in the application.
### Drive versioning (Nextcloud WebDAV)
@@ -124,9 +128,15 @@ Models/
All files are uploaded to `VF/` (not just diffs), because the move operation empties the previous folder.
### Upload safeguards
- The upload flow prevents duplicate submissions on the client (`Envoyer`, overwrite confirmation, and "Git only" confirmation are locked while processing)
- The server applies a lightweight per-folder lock on Drive and Git routes to avoid duplicate commits and concurrent writes
- The folder is staged server-side so the browser sends the payload only once during the full upload flow
### Commit messages
All changes are pushed in a **single commit** with a formatted message:
All changes are pushed in a **single commit** with a grouped formatted message:
**New folder:**
```
@@ -134,8 +144,14 @@ update: upload-gltf add a new model -> my-model
📦 Model
✅ model.glb (compressed)
🎨 Textures
✅ color.jpg
🎨 Textures (color)
✅ color_porte.jpg (compressed)
🪶 Textures (roughness)
✅ roughness_tuyaux.png (compressed)
🧩 Assets
✅ opacity_fenetre.png (compressed)
```
**Update (only one texture changed):**
@@ -143,28 +159,44 @@ update: upload-gltf add a new model -> my-model
update: upload-gltf update -> coffeetest
📦 Model
↔️ model.glb (inchange)
🎨 Textures
🔄 color_tuyaux.jpg
↔️ model.glb (compressed)
🎨 Textures (color)
🔄 color_tuyaux.jpg (compressed)
```
Sections currently used:
- `📦 Model`
- `🎨 Textures (color)`
- `🪶 Textures (roughness)`
- `🧭 Textures (normal)`
- `🔩 Textures (metalness)`
- `🧩 Assets`
- `🗑 Deleted`
Symbols: `✅` new — `🔄` modified — `↔️` unchanged (model always re-pushed) — `❌` deleted
8. Orphan files (present on remote but not in the new upload) are deleted in the same commit
9. If Blender is unavailable, the original model is pushed as-is (graceful fallback)
## Destinations
Uploaded models are pushed to `public/models/<folderName>/` in the target repo.
## Current Limitations
- Large uploads are faster than before because the folder is staged only once, but the Drive upload remains sequential.
- Git LFS uploads are still sequential.
- The current upload contract still expects a single `model.glb` file and a flat texture set.
## Project Structure
```
app/
├── api/upload/
│ ├── check/route.ts # GET: check remote folder + file sizes for diff
│ ├── drive/route.ts # POST: upload originals to Nextcloud Drive (VF/Vx versioning)
── git/route.ts # POST: compress with Blender + push to GitHub
│ ├── stage/route.ts # POST: upload folder once to temporary staging
│ ├── check/route.ts # POST: prepare staged Git assets and compare with remote files
── drive/route.ts # POST: upload staged originals to Nextcloud Drive (VF/Vx versioning)
│ └── git/route.ts # POST: push staged prepared assets to GitHub
├── globals.css # Tailwind + CSS variable fonts
├── layout.tsx # Root layout (next/font/google)
└── page.tsx # Home page
@@ -191,14 +223,17 @@ hooks/
└── useUploadOrchestrator.ts # Upload pipeline orchestration (Drive → Git)
lib/
├── constants.ts # Shared constants and extensions
├── types.ts # Server types (ParsedFile, FileDiff, etc.)
├── types.ts # Server types (ParsedFile, FileDiff, staged asset metadata, etc.)
├── client-types.ts # Client types (FolderEntry, DriveStatus, etc.)
├── upload-api.ts # Client-side API helpers (check, uploadDrive, uploadGit)
├── upload-api.ts # Client-side API helpers (stage, check, uploadDrive, uploadGit)
├── diff-files.ts # File diff classification (new/changed/unchanged/deleted)
├── sanitize.ts # Filename sanitization
├── auth.ts # Upload secret validation (timing-safe)
├── github.ts # Octokit helpers (getRemoteFolder, pushAllToGitHub)
├── nextcloud.ts # Nextcloud WebDAV client (native fetch, cached config)
├── upload-staging.ts # Temporary server-side staging and prepared asset reuse
├── upload-lock.ts # Lightweight in-memory per-folder upload lock
├── asset-classification.ts # Group assets by family for commit messages
├── blender.ts # Blender Draco compression
├── commit-message.ts # Commit message builder
├── parse-upload.ts # FormData parser + validation
@@ -214,7 +249,7 @@ docker-entrypoint.sh # Startup: Blender check + launch
| Type | Extensions |
|------|------------|
| 3D Models | `.glb`, `.gltf` |
| 3D Models | `.glb` |
| Textures | `.png`, `.jpg`, `.jpeg`, `.webp` |
## License
+9 -10
View File
@@ -1,9 +1,8 @@
import { NextRequest, NextResponse } from 'next/server'
import { validateUploadSecret } from '@/lib/auth'
import { parseMultiUpload } from '@/lib/parse-upload'
import { getRemoteFolder } from '@/lib/github'
import { classifyFileChanges } from '@/lib/diff-files'
import { prepareGitAssets } from '@/lib/prepare-git-assets'
import { ensurePreparedStagingAssets } from '@/lib/upload-staging'
export const runtime = 'nodejs'
export const dynamic = 'force-dynamic'
@@ -16,22 +15,22 @@ export async function POST(req: NextRequest) {
const authError = validateUploadSecret(req)
if (authError) return authError
let folderName: string
let parsedFiles: Awaited<ReturnType<typeof parseMultiUpload>>['files']
let stagingId: string
try {
const parsed = await parseMultiUpload(req)
folderName = parsed.folderName
parsedFiles = parsed.files
const body = await req.json()
stagingId = body.stagingId
if (!stagingId || typeof stagingId !== 'string') {
throw new Error('stagingId manquant')
}
} catch (err) {
const message = err instanceof Error ? err.message : 'Erreur inconnue'
return NextResponse.json({ success: false, error: message }, { status: 400 })
}
const folderPath = `public/models/${folderName}`
try {
const { filesToPush } = await prepareGitAssets({ folderName, parsedFiles })
const { folderName, filesToPush } = await ensurePreparedStagingAssets(stagingId)
const folderPath = `public/models/${folderName}`
const { exists, files } = await getRemoteFolder(folderPath)
if (exists) {
+14 -9
View File
@@ -1,6 +1,6 @@
import { NextRequest, NextResponse } from 'next/server'
import { validateUploadSecret } from '@/lib/auth'
import { parseMultiUpload } from '@/lib/parse-upload'
import { readStagedOriginalFiles } from '@/lib/upload-staging'
import {
mkdirRecursive,
moveFolder,
@@ -17,8 +17,8 @@ export const dynamic = 'force-dynamic'
//
// Upload **original** files (no Blender compression) to Nextcloud Drive.
//
// FormData fields:
// - folderName, files[], fileTypes[], textureNames[] (same as /api/upload/git)
// JSON body:
// - stagingId
// - action: "new" | "replace"
//
// Versioning logic:
@@ -42,16 +42,21 @@ export async function POST(req: NextRequest) {
)
}
// --- Parse files (includes extra fields like "action") ---
// --- Parse staging request ---
let folderName: string
let parsedFiles: Awaited<ReturnType<typeof parseMultiUpload>>['files']
let parsedFiles: Awaited<ReturnType<typeof readStagedOriginalFiles>>['files']
let action: string
try {
const parsed = await parseMultiUpload(req)
folderName = parsed.folderName
parsedFiles = parsed.files
action = parsed.extra.action?.trim() || 'new'
const body = await req.json()
const stagingId = body.stagingId
if (!stagingId || typeof stagingId !== 'string') {
throw new Error('stagingId manquant')
}
action = typeof body.action === 'string' ? body.action.trim() || 'new' : 'new'
const staged = await readStagedOriginalFiles(stagingId)
folderName = staged.folderName
parsedFiles = staged.files
} catch (err) {
const message = err instanceof Error ? err.message : 'Erreur inconnue'
return NextResponse.json({ success: false, error: message }, { status: 400 })
+12 -8
View File
@@ -1,10 +1,9 @@
import { NextRequest, NextResponse } from 'next/server'
import { validateUploadSecret } from '@/lib/auth'
import { parseMultiUpload } from '@/lib/parse-upload'
import { getRemoteFolder, pushAllToGitHub } from '@/lib/github'
import { buildCommitMessage } from '@/lib/commit-message'
import { classifyFileChanges } from '@/lib/diff-files'
import { prepareGitAssets } from '@/lib/prepare-git-assets'
import { cleanupStagingUpload, ensurePreparedStagingAssets, readStagedManifest } from '@/lib/upload-staging'
import { acquireUploadLock, releaseUploadLock } from '@/lib/upload-lock'
export const runtime = 'nodejs'
@@ -19,14 +18,17 @@ export async function POST(req: NextRequest) {
const authError = validateUploadSecret(req)
if (authError) return authError
// --- Parse all files ---
let folderName: string
let parsedFiles: Awaited<ReturnType<typeof parseMultiUpload>>['files']
let stagingId: string
try {
const parsed = await parseMultiUpload(req)
folderName = parsed.folderName
parsedFiles = parsed.files
const body = await req.json()
stagingId = body.stagingId
if (!stagingId || typeof stagingId !== 'string') {
throw new Error('stagingId manquant')
}
const manifest = await readStagedManifest(stagingId)
folderName = manifest.folderName
} catch (err) {
const message = err instanceof Error ? err.message : 'Erreur inconnue'
return NextResponse.json({ success: false, error: message }, { status: 400 })
@@ -47,7 +49,7 @@ export async function POST(req: NextRequest) {
compressed,
compressionError,
assetSummaries,
} = await prepareGitAssets({ folderName, parsedFiles })
} = await ensurePreparedStagingAssets(stagingId)
// --- Detect existing files and classify changes ---
const folderPath = `public/models/${folderName}`
@@ -67,6 +69,7 @@ export async function POST(req: NextRequest) {
// If nothing changed, don't create an empty commit
if (changedFilesToPush.length === 0 && deletePaths.length === 0) {
await cleanupStagingUpload(stagingId).catch(() => {})
return NextResponse.json({
success: true,
folderName,
@@ -90,6 +93,7 @@ export async function POST(req: NextRequest) {
// --- Push all in one commit ---
try {
const { commitUrl } = await pushAllToGitHub(changedFilesToPush, deletePaths, commitMessage)
await cleanupStagingUpload(stagingId).catch(() => {})
return NextResponse.json({
success: true,
+21
View File
@@ -0,0 +1,21 @@
import { NextRequest, NextResponse } from 'next/server'
import { validateUploadSecret } from '@/lib/auth'
import { parseMultiUpload } from '@/lib/parse-upload'
import { createStagingUpload } from '@/lib/upload-staging'
export const runtime = 'nodejs'
export const dynamic = 'force-dynamic'
export async function POST(req: NextRequest) {
const authError = validateUploadSecret(req)
if (authError) return authError
try {
const parsed = await parseMultiUpload(req)
const staged = await createStagingUpload(parsed.folderName, parsed.files)
return NextResponse.json({ success: true, ...staged })
} catch (err) {
const message = err instanceof Error ? err.message : 'Erreur inconnue'
return NextResponse.json({ success: false, error: message }, { status: 400 })
}
}
+29 -6
View File
@@ -7,7 +7,7 @@
import { useState, useRef, useCallback } from 'react'
import type { FolderEntry } from '@/lib/client-types'
import type { FileDiff } from '@/lib/types'
import { checkFolderDiffs, uploadDrive, uploadGit } from '@/lib/upload-api'
import { checkFolderDiffs, stageUpload, uploadDrive, uploadGit } from '@/lib/upload-api'
import type { CheckResult } from '@/lib/upload-api'
interface UseUploadOrchestratorParams {
@@ -42,6 +42,7 @@ export function useUploadOrchestrator({
const abortRef = useRef<AbortController | null>(null)
const checkResultRef = useRef<CheckResult>({ exists: false, diffs: [] })
const uploadActionRef = useRef(false)
const stagingIdRef = useRef<string | null>(null)
// Refs for values used inside callbacks to avoid stale closures
const secretRef = useRef(secret)
@@ -51,10 +52,14 @@ export function useUploadOrchestrator({
// ---- Internal: push a single folder to Git ----
const pushGit = useCallback(async (index: number, signal?: AbortSignal) => {
const folderEntry = entriesRef.current[index]
const stagingId = stagingIdRef.current
if (!stagingId) {
updateEntry(index, { status: 'error', error: 'Preparation serveur introuvable' })
return
}
const gitResult = await uploadGit(
folderEntry,
stagingId,
secretRef.current,
(pct) => updateEntry(index, { progress: 50 + Math.round(pct / 2) }),
signal,
@@ -89,6 +94,11 @@ export function useUploadOrchestrator({
const folderEntry = currentEntries[i]
const driveAction = checkResultRef.current.exists ? 'replace' : 'new'
const stagingId = stagingIdRef.current
if (!stagingId) {
updateEntry(i, { status: 'error', error: 'Preparation serveur introuvable' })
return
}
// ---- Step 1: Drive upload ----
updateEntry(i, {
@@ -100,7 +110,7 @@ export function useUploadOrchestrator({
})
const driveResult = await uploadDrive(
folderEntry,
stagingId,
secretRef.current,
driveAction as 'new' | 'replace',
controller.signal,
@@ -141,12 +151,17 @@ export function useUploadOrchestrator({
setGlobalError(null)
const folder = entriesRef.current[0]
const controller = new AbortController()
abortRef.current = controller
try {
const staged = await stageUpload(folder, secretRef.current, controller.signal)
stagingIdRef.current = staged.stagingId
const check = await checkFolderDiffs(
folder,
staged.stagingId,
secretRef.current,
abortRef.current?.signal,
controller.signal,
)
checkResultRef.current = check
@@ -155,11 +170,13 @@ export function useUploadOrchestrator({
setNoChangesFolder(folder.folderName)
uploadActionRef.current = false
setIsChecking(false)
abortRef.current = null
return
}
setOverwriteConfirm({ folderName: folder.folderName, diffs: check.diffs })
uploadActionRef.current = false
setIsChecking(false)
abortRef.current = null
return
}
} catch (err) {
@@ -167,10 +184,12 @@ export function useUploadOrchestrator({
setGlobalError(message)
uploadActionRef.current = false
setIsChecking(false)
abortRef.current = null
return
}
uploadActionRef.current = false
abortRef.current = null
await proceedUpload()
}, [setSecretError, proceedUpload, isChecking, isUploading])
@@ -231,6 +250,8 @@ export function useUploadOrchestrator({
const handleCancel = useCallback(() => {
if (isChecking) {
abortRef.current?.abort()
abortRef.current = null
uploadActionRef.current = false
setIsChecking(false)
return
@@ -240,6 +261,7 @@ export function useUploadOrchestrator({
uploadActionRef.current = false
setIsResolvingDriveError(false)
setIsUploading(false)
stagingIdRef.current = null
}, [isChecking])
const handleReset = useCallback(() => {
@@ -251,6 +273,7 @@ export function useUploadOrchestrator({
setDriveError(null)
checkResultRef.current = { exists: false, diffs: [] }
uploadActionRef.current = false
stagingIdRef.current = null
}, [resetEntries])
return {
+51 -15
View File
@@ -10,6 +10,12 @@ export interface CheckResult {
diffs: FileDiff[]
}
export interface StageResult {
stagingId: string
folderName: string
filesCount: number
}
// ---------------------------------------------------------------------------
// Shared FormData builder
// ---------------------------------------------------------------------------
@@ -49,15 +55,17 @@ function buildUploadFormData(
* Throws on auth/network errors so callers can surface them to the user.
*/
export async function checkFolderDiffs(
folder: FolderEntry,
stagingId: string,
secret: string,
signal?: AbortSignal,
): Promise<CheckResult> {
const formData = buildUploadFormData(folder)
const res = await fetch('/api/upload/check', {
method: 'POST',
headers: { 'x-upload-secret': secret.trim() },
body: formData,
headers: {
'Content-Type': 'application/json',
'x-upload-secret': secret.trim(),
},
body: JSON.stringify({ stagingId }),
signal,
})
@@ -75,24 +83,51 @@ export async function checkFolderDiffs(
return { exists: true, diffs: (data.diffs || []) as FileDiff[] }
}
export async function stageUpload(
folder: FolderEntry,
secret: string,
signal?: AbortSignal,
): Promise<StageResult> {
const formData = buildUploadFormData(folder)
const res = await fetch('/api/upload/stage', {
method: 'POST',
headers: { 'x-upload-secret': secret.trim() },
body: formData,
signal,
})
const data = await res.json()
if (!res.ok || !data.success) {
throw new Error(data.error || `Erreur serveur (${res.status})`)
}
return {
stagingId: data.stagingId,
folderName: data.folderName,
filesCount: data.filesCount,
}
}
// ---------------------------------------------------------------------------
// Upload original files to Nextcloud Drive
// ---------------------------------------------------------------------------
/** Upload original files to Nextcloud Drive (no Blender compression). */
export async function uploadDrive(
folder: FolderEntry,
stagingId: string,
secret: string,
action: 'new' | 'replace',
signal?: AbortSignal,
): Promise<{ success: boolean; error?: string }> {
const formData = buildUploadFormData(folder, { action })
try {
const res = await fetch('/api/upload/drive', {
method: 'POST',
headers: { 'x-upload-secret': secret.trim() },
body: formData,
headers: {
'Content-Type': 'application/json',
'x-upload-secret': secret.trim(),
},
body: JSON.stringify({ stagingId, action }),
signal,
})
const data = await res.json()
@@ -112,20 +147,21 @@ export async function uploadDrive(
/** Upload files to GitHub (with Blender compression). */
export async function uploadGit(
folder: FolderEntry,
stagingId: string,
secret: string,
onProgress: (pct: number) => void,
signal?: AbortSignal,
): Promise<{ success: boolean; filename?: string; error?: string }> {
const formData = buildUploadFormData(folder)
onProgress(10)
try {
const res = await fetch('/api/upload/git', {
method: 'POST',
headers: { 'x-upload-secret': secret.trim() },
body: formData,
headers: {
'Content-Type': 'application/json',
'x-upload-secret': secret.trim(),
},
body: JSON.stringify({ stagingId }),
signal,
})
@@ -137,7 +173,7 @@ export async function uploadGit(
}
onProgress(100)
return { success: true, filename: folder.folderName }
return { success: true, filename: data.folderName }
} catch (err) {
if (err instanceof DOMException && err.name === 'AbortError') {
return { success: false, error: 'Upload annule' }
+214
View File
@@ -0,0 +1,214 @@
import { randomUUID } from 'crypto'
import { dirname, join } from 'path'
import { mkdir, readdir, readFile, rm, stat, writeFile } from 'fs/promises'
import { existsSync } from 'fs'
import { TMP_DIR } from '@/lib/constants'
import { prepareGitAssets } from '@/lib/prepare-git-assets'
import type { ParsedFile, PreparedAssetSummary } from '@/lib/types'
const STAGING_ROOT = join(TMP_DIR, 'staging')
const STAGING_TTL_MS = 60 * 60 * 1000
interface StagedOriginalFile {
filename: string
size: number
isModel: boolean
}
interface StagedPreparedData {
modelFilename: string
compressed: boolean
compressionError?: string
assetSummaries: PreparedAssetSummary[]
}
interface StagingManifest {
stagingId: string
folderName: string
createdAt: number
originals: StagedOriginalFile[]
prepared?: StagedPreparedData
}
interface PushFile {
path: string
contentBase64: string
}
interface PreparedStageAssetsResult {
folderName: string
filesToPush: PushFile[]
modelFilename: string
assetSummaries: PreparedAssetSummary[]
compressed: boolean
compressionError?: string
}
function getStageDir(stagingId: string) {
return join(STAGING_ROOT, stagingId)
}
function getOriginalDir(stagingId: string) {
return join(getStageDir(stagingId), 'original')
}
function getPreparedDir(stagingId: string) {
return join(getStageDir(stagingId), 'prepared')
}
function getManifestPath(stagingId: string) {
return join(getStageDir(stagingId), 'manifest.json')
}
async function ensureParentDir(filePath: string) {
await mkdir(dirname(filePath), { recursive: true })
}
async function writeManifest(manifest: StagingManifest) {
await ensureParentDir(getManifestPath(manifest.stagingId))
await writeFile(getManifestPath(manifest.stagingId), JSON.stringify(manifest, null, 2), 'utf-8')
}
export async function cleanupExpiredStagingUploads() {
if (!existsSync(STAGING_ROOT)) return
const entries = await readdir(STAGING_ROOT, { withFileTypes: true })
const now = Date.now()
for (const entry of entries) {
if (!entry.isDirectory()) continue
const stagingId = entry.name
try {
const manifest = await readStagedManifest(stagingId)
if (now - manifest.createdAt > STAGING_TTL_MS) {
await cleanupStagingUpload(stagingId)
}
} catch {
await cleanupStagingUpload(stagingId).catch(() => {})
}
}
}
export async function createStagingUpload(folderName: string, parsedFiles: ParsedFile[]) {
await cleanupExpiredStagingUploads()
const stagingId = randomUUID()
const originalDir = getOriginalDir(stagingId)
await mkdir(originalDir, { recursive: true })
const originals: StagedOriginalFile[] = []
for (const file of parsedFiles) {
const filePath = join(originalDir, file.filename)
await ensureParentDir(filePath)
await writeFile(filePath, file.buffer)
originals.push({ filename: file.filename, size: file.buffer.length, isModel: file.isModel })
}
const manifest: StagingManifest = {
stagingId,
folderName,
createdAt: Date.now(),
originals,
}
await writeManifest(manifest)
return {
stagingId,
folderName,
filesCount: originals.length,
}
}
export async function readStagedManifest(stagingId: string): Promise<StagingManifest> {
const manifestPath = getManifestPath(stagingId)
const content = await readFile(manifestPath, 'utf-8')
return JSON.parse(content) as StagingManifest
}
async function readOriginalParsedFiles(stagingId: string, manifest: StagingManifest): Promise<ParsedFile[]> {
const originalDir = getOriginalDir(stagingId)
return Promise.all(
manifest.originals.map(async (file) => ({
filename: file.filename,
buffer: await readFile(join(originalDir, file.filename)),
isModel: file.isModel,
})),
)
}
async function buildPreparedPushFiles(stagingId: string, manifest: StagingManifest): Promise<PushFile[]> {
const preparedDir = getPreparedDir(stagingId)
return Promise.all(
manifest.originals.map(async (file) => {
const buffer = await readFile(join(preparedDir, file.filename))
return {
path: `public/models/${manifest.folderName}/${file.filename}`,
contentBase64: buffer.toString('base64'),
}
}),
)
}
export async function ensurePreparedStagingAssets(stagingId: string): Promise<PreparedStageAssetsResult> {
const manifest = await readStagedManifest(stagingId)
if (!manifest.prepared) {
const parsedFiles = await readOriginalParsedFiles(stagingId, manifest)
const prepared = await prepareGitAssets({ folderName: manifest.folderName, parsedFiles })
const preparedDir = getPreparedDir(stagingId)
await mkdir(preparedDir, { recursive: true })
for (const file of prepared.filesToPush) {
const filename = file.path.split('/').pop()
if (!filename) continue
const outputPath = join(preparedDir, filename)
await ensureParentDir(outputPath)
await writeFile(outputPath, Buffer.from(file.contentBase64, 'base64'))
}
manifest.prepared = {
modelFilename: prepared.modelFilename,
compressed: prepared.compressed,
compressionError: prepared.compressionError,
assetSummaries: prepared.assetSummaries,
}
await writeManifest(manifest)
}
return {
folderName: manifest.folderName,
filesToPush: await buildPreparedPushFiles(stagingId, manifest),
modelFilename: manifest.prepared.modelFilename,
assetSummaries: manifest.prepared.assetSummaries,
compressed: manifest.prepared.compressed,
compressionError: manifest.prepared.compressionError,
}
}
export async function readStagedOriginalFiles(stagingId: string): Promise<{ folderName: string; files: ParsedFile[] }> {
const manifest = await readStagedManifest(stagingId)
return {
folderName: manifest.folderName,
files: await readOriginalParsedFiles(stagingId, manifest),
}
}
export async function cleanupStagingUpload(stagingId: string) {
await rm(getStageDir(stagingId), { recursive: true, force: true })
}
export async function stagingExists(stagingId: string): Promise<boolean> {
try {
const info = await stat(getStageDir(stagingId))
return info.isDirectory()
} catch {
return false
}
}