Engineering / Field notes
Designing video uploads that survive unreliable networks
Large uploads need more than a progress bar: resumable parts, measured concurrency, persisted state, and a finalization path that can recover after a tab closes.
01
A large upload is a distributed workflow
A browser upload looks simple until the file is several gigabytes, the connection changes from office Wi-Fi to a hotspot, or the tab disappears one part before completion. At that point the upload is no longer one request. It is a workflow shared by the browser, an application API, object storage, and the background services that prepare playback.
Treating the workflow explicitly changes the design. The browser should be able to ask for an upload session, send independent parts directly to storage, persist enough local state to resume, and submit the completed part inventory for finalization.
02
Split the file into independently retryable work
Multipart upload turns one fragile transfer into many bounded transfers. If part 37 fails, the browser retries part 37 instead of restarting the entire source. Each successful part produces a receipt that belongs in the completion request.
Part size is a tradeoff. Tiny parts increase request and bookkeeping overhead. Very large parts make each retry expensive. A good implementation centralizes these limits so the browser, API, and storage policy agree on the protocol.
- Generate part work from a single source of truth.
- Keep successful part receipts until finalization is acknowledged.
- Make completion idempotent so retrying it is safe.
03
Concurrency should respond to the network
Fixed concurrency is tempting: always upload four parts at once and call it done. But four parallel transfers can be conservative on a strong wired connection and destructive on a weak mobile one.
An additive-increase, multiplicative-decrease strategy gives the browser a practical feedback loop. Increase concurrency gradually while parts complete cleanly. When errors or timeouts appear, reduce concurrency decisively. This is not a perfect model of the network, but it is substantially kinder than either one-at-a-time uploads or an unbounded queue.
04
Resume is a product feature, not a retry button
A resumable upload needs durable browser state: which local file the session represents, which parts completed, the upload identifier, and when the server-side session expires. IndexedDB is a better home for that record than an in-memory component state.
The product must also explain what happened. A user should be able to distinguish paused, reconnecting, awaiting finalization, and expired. If the original file is no longer available to the browser, the interface should say so rather than displaying a progress bar that can never move.
- Persist session and receipt state locally.
- Reconcile local parts with server truth after reconnecting.
- Give expired sessions a clear restart path.
05
Completion can outlive the page
The last byte reaching storage is not the same as the video being ready. The object must be completed, recorded, inspected, and handed to the playback pipeline. A small browser outbox can preserve the final completion intent if navigation interrupts that handoff.
From there, durable background jobs should own provider ingest, polling, derivative generation, and publication readiness. The browser can observe the job; it should not be the job runner.
06
The calm upload experience is built from explicit states
The best upload UI feels uneventful because the complicated states are accounted for. Progress moves, temporary failures recover, refresh does not erase completed work, and processing continues after the browser leaves.
That calm experience is not created by animation. It comes from a protocol that expects networks, tabs, and downstream services to be unreliable—and gives every stage a safe way forward.