Finding things in an archive

What is the cheapest way to generate embeddings from videos?

The cheapest video embedding pipeline is the one that processes each file once. Whatever you spend per hour of footage, you spend again every time the archive gets reprocessed, and reprocessing is what usually turns a tolerable number into an ugly one. So the first decision is not which model or which host. It is how many passes over your library you are willing to pay for.

The levers that actually move the number

Video embedding stacks four costs on top of each other. Decoding comes first, because the file has to be read and turned into frames and audio, and long high-bitrate files are slow to decode before any model runs at all. Sampling comes second, and it is the biggest lever anyone has. A pipeline that embeds one frame per second does roughly ten times the model work of one that embeds a frame every ten seconds, and on interview or presentation footage the sparse version loses almost nothing. Third is inference, which is what people usually mean when they ask this question. Fourth is the vector store, which is cheap per vector and stops being cheap somewhere past a few hundred million of them.

The cost nobody budgets for is the second pass. A better model ships, someone decides the sampling was too sparse, another team wants audio embeddings as well as visual ones, and the whole library goes through again. Every route below is really a bet about how many times that will happen.

Run an open model on hardware you already have

If you have a machine with a decent GPU sitting under a desk, this is the floor. You supply the electricity and your own time, and there is no per-hour anything. The real price is operational: you own the decode pipeline, the batching, the retries, the checkpointing so a crash at hour 300 does not cost you hours 1 through 299, and the storage layout. For a one-time backfill of an archive that is not growing much, this is usually the right answer and people talk themselves out of it too quickly. The models are there for the taking, and the tradeoffs of that route are laid out in more detail in running open models instead of a managed service.

Rent GPUs for a burst, then stop

A backfill is a burst workload. Renting capacity for a weekend and shutting it down afterward tends to beat both buying hardware and streaming footage through a service continuously, because your library has a beginning and an end. This route punishes disorganization: if you have not deduplicated the archive first, you will pay to embed the same wedding-in-a-warehouse promo four times under four filenames.

Send the footage to a hosted embedding API

Here you are renting somebody else's compute plus their operational work, which is genuinely worth something if nobody on your team wants to own a decode pipeline. The thing to check before committing is what happens on the second pass, since that is where the arithmetic changes, and whether the interface you are buying is an embedding endpoint or an actual search system. Those are different products with different downstream work, and what an indexing API hands back is worth understanding before you pick.

Embed the words instead of the pixels

If your footage is people talking, text embeddings over a transcript will get you most of the retrieval quality at a fraction of the compute, because you run speech recognition once and then embed a few thousand tokens per hour instead of thousands of frames. Transcription has its own price, and what an hour of speech-to-text costs is the number to compare against. This route goes blind exactly where the talking stops.

Let ingest do it

The last shape is a retrieval layer where indexing is part of storing the material rather than a project you run. This is the shape that removes the second-pass problem structurally instead of by discipline. Vivu works this way: each file is indexed once as part of being uploaded to a project, so after a busy shoot week the only thing to remember is the upload, and the embedding comes with it.

When you do not need embeddings at all

If your library is a few hundred clips, filenames and a spreadsheet beat a vector index on every dimension including how fast you can find something. If everything you search for is a spoken phrase, plain keyword search over transcripts is close to free and takes an afternoon. Embeddings earn their cost when the questions are fuzzy, the library is large, and neither of those is going to reverse.

The decision comes down to one question: is this a finite archive you index once, or a library that keeps growing? If it is finite, rent compute for a burst and own the output. If it keeps growing, the cheap-looking option that requires a human to remember to run it will quietly become the expensive one.

FAQ

How many frames per second do I need to embed for video search?

For most footage, one frame every two to ten seconds is enough. Shot length is the thing to match: if your average shot runs five seconds, sampling every ten seconds will miss shots entirely, and sampling every half second mostly buys you near-duplicate vectors of the same shot.

Fast-moving material is the exception. Sports, gameplay, and anything with rapid cuts need denser sampling because the meaningful moment can be under a second long. A common compromise is to sample sparsely by default and densely on segments flagged by audio energy or scene-change detection, which keeps the frame count low without going blind on the busy parts.

Do I have to re-embed everything when a better model comes out?

Only if you want the new model's results on old footage, and you cannot mix them. Embeddings from two different models live in different vector spaces, so a query embedded with the new model will return nonsense when compared against vectors made by the old one.

In practice teams either accept a split library, where new material is searchable with the new model and the archive stays on the old index, or they schedule a full re-embed. Keeping the original files and the decode pipeline reproducible is what makes the second option affordable later, so store the source media in a way you can re-read, not just the vectors.

Can I generate video embeddings on a laptop?

Yes, for a small library or an overnight job. A recent laptop with a decent GPU or unified memory will process footage at a workable rate when you are sampling frames sparsely rather than densely.

The limit is throughput, not capability. A few hundred hours of footage on a laptop turns into days of wall-clock time, and the machine is unusable for anything else while it runs. That is the point where renting compute for one weekend stops being an indulgence.

Does embedding cost depend on file size or video length?

Length drives inference cost, since the number of vectors comes from how many frames you sample per minute. A 4K file and a 720p file of the same duration produce the same number of embeddings, because frames get resized down to the model's input resolution anyway.

File size still costs you elsewhere. Large files are slower to decode and more expensive to move across a network, so a library of high-bitrate camera originals can spend more on decode and transfer than on the model itself.