Uploading beyond the proxy limit without bypassing the proxy

01

The application limit was not the real limit

OpenMake accepts large files as inputs to autonomous agent tasks. The application can allow a file well beyond 100 MB, but that does not matter when a proxied public request is rejected before it reaches the API.

Cloudflare documents a 100 MB maximum upload size for Free and Pro zones. A single multipart request therefore failed at the edge with HTTP 413 before OpenMake could apply its own validation or show useful progress.

02

The constraints shaped the protocol

The quickest workaround would have been an unproxied upload host. We kept the protected public route and changed the request shape instead. That decision preserved the existing DNS, TLS, authentication, and deployment boundary.

• Keep small uploads on the existing multipart path.
• Authenticate every upload operation and bind each session to one user.
• Make a repeated chunk write safe so the browser can retry an interrupted part.
• Reuse the existing storedPath contract after assembly so extraction, sandbox injection, and cleanup do not fork.

03

A four-step protocol with a one-time claim

The browser first declares the file name, MIME type, byte size, and expected chunk count. It sends raw application/octet-stream chunks, asks the server to complete the session, and finally creates the task with an uploadId reference.

Completion is idempotent. Claiming is intentionally not: once the assembled file moves into the task directory, the temporary upload session is removed and cannot be attached to a second task.

POST /api/agent-task-uploads
PUT  /api/agent-task-uploads/:id/chunks/0
PUT  /api/agent-task-uploads/:id/chunks/1
POST /api/agent-task-uploads/:id/complete
POST /api/agent-tasks  { files: [{ uploadId: id }] }

04

Ownership and integrity live at the storage boundary

Each server-issued UUID maps to a directory containing meta.json, numbered chunks, and the assembled file. The metadata records the owner and declaration. Every write, complete, claim, and abort operation reloads that metadata and compares the authenticated user.

The store rejects path-shaped IDs, empty or oversized chunks, out-of-range indexes, missing parts, and totals that differ from the declared size. A partial assembly is renamed only after all chunks have been appended. Unclaimed sessions are removed opportunistically after 24 hours.

05

The browser switches transport, not product behavior

When the combined binary payload exceeds 60 MB, the web client uploads each file in 24 MB pieces. The 24 MB client size stays below the API's 32 MB raw-parser ceiling and leaves room for transport overhead. Smaller inputs continue through multipart without paying the extra round trips.

After completion, the task creation request is small JSON containing upload references. From that point on, document extraction and agent execution follow the same path as before.

06

Verification crossed unit, API, and browser boundaries

The implementation commit records seven chunk-store unit tests and 130 passing tests in the agent-task suite, with TypeScript and ESLint clean. The unit cases cover restoration, idempotent completion, owner isolation, UUID path defense, missing chunks, size mismatch, invalid indexes, and claiming before completion.

The live API check sent a 130 MB file in six chunks through chat.openmake.cc, created a task, compared the stored file's SHA-256 with the original, and confirmed cleanup after deletion. The browser check attached a 65 MB PDF and observed three chunk requests, a successful upload reference, and an accepted task execution request.

07

Trade-offs we accepted

The disk-backed design is simple and survives an application restart, but horizontal scaling requires a shared upload volume or upload-session affinity. Assembly is deliberately serial and bounded by one server chunk at a time, favoring predictable memory use over maximum throughput.

Declared size detects missing or extra bytes, but the public protocol does not yet carry a per-file checksum. The live test proved byte identity externally; a future protocol revision should make that guarantee part of completion itself.

08

What comes next

The next useful layer is a status endpoint that lets the browser discover already received indexes after a refresh. File checksums, object storage for multi-instance deployments, and clearer pause/resume progress would turn the current retryable transport into a fully resumable upload system.