Building a web application that accepts video is easy until the first real file arrives.
A conventional web application is built around short exchanges. The browser sends a request, the server reads or changes some data, and a response comes back quickly. Video breaks that shape. Files are large. Processing can take minutes. Different stages need different runtimes. A user may close the tab halfway through, while the work still needs to finish safely.
When I started building CutCaption, I wanted the product to feel like a browser editor. The architecture behind that experience, however, looks less like a normal form-based application and more like a small media-processing system.
This article explains the model I used: direct uploads to object storage, a deliberately boring application control plane, durable job records, disposable serverless media workers, derived editing assets, and asynchronous exports. It is a case study, but the patterns apply to many young products that process video, audio, documents, or other large files.
Video changes the shape of a web application
A video product has four pressures that an ordinary form-based web application can often ignore.
Trying to solve all four pressures inside the main web server produces a fragile system. A large upload competes with normal API traffic. One slow media command occupies a process that should be serving users. A deployment can terminate work that was already in progress. Scaling the application also scales expensive media dependencies that most requests never use.
The architectural change is to stop treating media processing as a request and start treating it as durable work.
The system at a glance
I divide the system into two broad parts.
The control plane owns product state. It knows about users, projects, permissions, the editable caption document, and the lifecycle of each job. It decides what may happen and records what has happened.
The media plane performs expensive work. Its workers download source material, run media tools or models, upload results, and report an outcome. A worker is not the authority on the project. It is a replaceable executor for one durable instruction.
This split matters more than the number of services. A startup does not need a large microservice estate to get the boundary right. One application, one relational database, one object store, and a few independently packaged worker types can already provide a strong separation of concerns.
The database carries decisions and lifecycle state. The object store carries bytes. Workers carry computation. The browser carries interaction.
Let the object store carry the upload
The first design decision is where the video bytes go.
The application server still needs to authorize the operation and record what is about to be uploaded. It does not need to relay every byte. Instead, the browser asks the application for narrowly scoped upload permission and sends the file directly to object storage.
After the object store has accepted the file, the browser asks the application to finalize the upload. Finalization verifies enough information to commit the project and its first job. Only then is background processing allowed to begin.
This has several useful consequences.
- The web server is not sized around the largest accepted video.
- Upload progress belongs to the browser, where the user can see it directly.
- An interrupted upload can be recovered without holding an application request open.
- The same storage boundary works whether the eventual worker runs beside the application or on a serverless platform.
There is an important distinction here: storing a file is not the same as accepting it into the product. The application should not create a finished-looking project merely because some bytes exist. Finalization is the transaction that joins the uploaded object, the project record, and the first processing obligation.
Long-running work is state, not a request
Once the upload is finalized, the application creates a durable job record. That record is the coordination point between the product and whichever worker eventually performs the work.
A minimal job needs an identity, a type, a current state, references to its inputs, and enough timestamps to support recovery. It may also carry progress and a sanitized failure reason. The specific schema matters less than one rule: the job must exist independently of the process trying to execute it.
A worker claims one pending job, obtains temporary ownership, and begins work. While it is active, it renews that ownership. If the worker disappears, a recovery process can see that ownership has expired and decide whether the job should be retried or failed.
The ownership mechanism is commonly called a lease. It prevents two workers from believing they still own the same active job indefinitely. A late worker may finish its local command, but it should not be allowed to overwrite a newer authoritative result after losing the lease.
Completion must also be idempotent. If a worker uploads the output but loses the network response while reporting success, it cannot know whether the application committed the result. Repeating the success report should return the same outcome rather than creating a second export or reversing a completed job into a failure.
This is one of the practical differences between a demo pipeline and a deployed product. The happy path is easy. The system becomes trustworthy when duplicate messages, missing responses, expired workers, and user cancellation have defined meanings.
Why serverless workers fit an early video product
Media demand is usually uneven. A worker may be idle for a while and then receive several expensive jobs close together. Different stages may also need incompatible system packages or hardware.
Serverless workers fit this shape well. Each worker type can have a small purpose-built image. Ingest needs media inspection and conversion tools. Transcription needs a speech runtime or access to a speech service. Export needs a renderer and may benefit from a different compute class. The platform can start instances when work arrives and release them when demand falls.
This keeps the application server small and predictable. It also keeps the expensive runtime dependencies out of ordinary web requests.
But serverless does not remove the distributed-systems work. It merely changes who operates the machines.
- Cold starts exist.
- Invocations can be delayed or repeated.
- Execution environments have limits.
- Temporary disks disappear.
- A provider may report an ambiguous result.
- Scaling without product-level limits can become expensive.
The application therefore keeps its own durable view of the job. The provider's invocation is a way to obtain compute, not the source of truth for the product.
I split workers along runtime and failure boundaries rather than creating a service for every verb. Media preparation, speech processing, and final export are meaningfully different workloads. Splitting a tiny metadata step from the rest merely because it has a different function name would add coordination without buying useful isolation.
Keep the original; edit with derivatives
An uploaded video is both an input and an asset that may be needed again. It should be treated differently from files that can be regenerated.
The original is the source of truth. The ingest worker derives smaller, purpose-built representations from it. A lightweight proxy makes browser playback and seeking easier. An extracted audio track feeds speech processing. Waveform data supports the timeline. Preview images make scrubbing and project lists feel immediate. Timed captions become an editable document rather than pixels baked into the source.
This is a small data lineage graph. It answers useful operational questions:
- Which assets must be retained?
- Which assets may be deleted and regenerated?
- Which stage produced a corrupt output?
- Can the editor reopen if the original is temporarily unavailable?
- Can a final export still reach the quality the user expects?
The database should store identities, relationships, state, and metadata. It should not become a warehouse for large binary payloads. Workers exchange object references through the job record and move the actual media through object storage.
Object storage therefore acts as the common data plane between compute environments. A worker does not need a shared local filesystem with the application. It only needs permission to read its declared inputs and write its declared outputs.
The browser observes work instead of waiting for it
The browser should never keep one request open for the life of a media job.
After finalization, the application can immediately return the project and its current state. The interface shows that processing is active and periodically asks for fresh status. The same state can appear in a project list, an activity view, or a notification surface. Closing the editor does not cancel the job unless the user explicitly asks for cancellation.
This sounds like a small interface detail, but it is another architectural boundary. The browser is a client of durable state, not the owner of the process.
Polling is often good enough for an early product. It is simple, observable, and easy to recover after a tab sleeps or reconnects. Push updates can be added later when their operational cost is justified. The underlying job model does not need to change.
Once the required derived assets are ready, the editor loads the proxy, waveform, preview images, and caption document. Most editing now happens against assets designed for interaction rather than the heavyweight original.
Each stage has a visible state. The user is not asked to infer whether a spinner means “still running,” “stuck,” or “failed three minutes ago.”
Export is another durable job
Export follows the same architecture in the opposite direction.
The editor first saves the current subtitle document. It then produces a rendering description — a compact, deterministic account of what each caption should look like and when it should appear — for that specific revision, and asks the application to create an export job. The export worker downloads the chosen source media and the rendering inputs, produces the final video, uploads it to object storage, and reports the result.
The application remains responsible for project state, policy, and the durable relationship between the request and the resulting file. The worker is responsible for rendering exactly one claimed job.
This is also where preview consistency becomes relevant. A browser preview and a final video run in different environments. Recreating styling rules independently in the export worker would create two renderers that slowly drift apart. CutCaption instead treats the generated rendering description as a contract shared by preview and export. The fast preview may optimize some updates, but a canonical full build remains the correctness path.
The useful lesson is not the subtitle format or browser library. It is the boundary: visual decisions should be settled before the preview and export environments diverge.
Failure is part of the design
A media pipeline should assume that every boundary can fail independently.
Cancellation is also cooperative. Removing a project or cancelling a job should revoke the worker's right to commit future progress. A well-behaved worker notices the lost lease and stops expensive child processes. The interface should not promise that every external operation disappears instantaneously.
Observability should follow the same boundary. Record state transitions, durations, attempt counts, and sanitized failure categories. Avoid copying private media content or full caption payloads into logs merely because logging is convenient.
Most recovery work can run periodically. It looks for jobs that should have progressed but did not, repairs safe cases, and makes terminal failures explicit. This is less glamorous than a new processing feature, but it is what turns a pipeline into an operable product.
What I would not build first
It is easy to draw this architecture and conclude that it needs a large message broker, a container orchestrator, event streaming, and a separate service for every stage.
That may eventually be appropriate. It is not the starting requirement.
For an early product, a relational database can hold durable job records. Workers can claim from that record set through a narrow application interface. A periodic recovery process can handle stale work. Object storage can be the media data plane. A small number of worker images can separate the genuinely different runtimes.
This design is intentionally conservative. It uses the database transaction model already required by the product and avoids introducing infrastructure whose failure modes the product does not yet need.
As demand grows, individual boundaries can evolve:
- A database-backed claim path can move behind a dedicated queue.
- Polling can gain event-driven wakeups.
- One worker pool can split by priority or hardware.
- Derived assets can gain regional caches.
- Recovery can become more specialized as real failure evidence accumulates.
The important part is that these changes replace a boundary rather than rewrite the product. The browser still observes project state. The application still owns the job. The worker still consumes object references and reports an outcome.
The practical starting point
If I were building another media startup, I would begin with the following rules:
- Keep the control plane boring. Put product decisions and durable state in the application and relational database.
- Send large files directly to object storage. The application grants and finalizes the operation but does not relay the payload.
- Create the job before dispatching it. Compute is replaceable; the obligation must already be durable.
- Make workers stateless and narrowly responsible. Inputs and outputs live in object storage, not on one machine.
- Model job ownership explicitly. Leases, idempotent completion, and stale recovery should exist before scale makes their absence painful.
- Separate originals from derivatives. Know what is irreplaceable, what is optimized for interaction, and what can be regenerated.
- Let the interface observe state. Do not tie the user's tab to the lifetime of remote compute.
- Add infrastructure in response to evidence. Preserve the boundaries and replace the mechanism behind them when the simpler version becomes the bottleneck.
CutCaption now feels like one browser application to the person using it. Behind that surface, uploads, media preparation, speech processing, editing, and export happen across different lifetimes and compute environments.
The system works because those parts do not need to share a process. They share contracts: an object reference, a durable job state, an ownership rule, and a defined result.
That is the architecture I would carry into the next media product.