Video in AI assistants and agents

How to build an agent workflow around footage you already have

An agent workflow around existing video has three moving parts: a way for the assistant to look inside the videos, a review step where a person decides what is actually usable, and a handoff to whoever does the next job. The first part decides whether the other two exist at all. An assistant that can only see file names will never answer "find the part where someone admits they were nervous", however the prompt is written.

Start from what the agent can reach

Four arrangements are common, and they differ mainly in how much of the video the assistant can see.

Pasting a clip into the conversation works when you already know which file and roughly where in it. You did the finding; the assistant is doing the describing. Past a handful of files this is not a workflow, it is manual work with a chat window attached.

A filesystem connector lets an assistant list folders, read names and dates, and move things around. It reads the shell of a library and never the contents. This is the arrangement people build first by accident, and then wonder why the assistant cannot answer a question about what happens in the footage.

Building your own retrieval layer means transcription, chunking, embeddings, a store, and a tool the assistant can call. You get control over how matching works, and you own the thing afterwards. A transcript-only index also stops at anything that was shown rather than said, which matters if your archive is lectures, performances, or demos. The trade is worked through in building the index yourself versus connecting one that already exists.

The fourth is a hosted retrieval connector. The footage goes into a project, it is indexed once, and the assistant queries it by description from inside a normal conversation. The cost is that the video has to be uploaded and the index has to be built before any of it works.

Ask for a moment, then look at what came back

Vivu connects to any assistant that supports remote MCP, including Claude and ChatGPT, and the footage lives in a Vivu project you upload to first. That project is indexed in the cloud once, so a question asked in a new conversation weeks later reads from the same index rather than starting over. In the chat you describe the moment instead of naming a file, something like find the part where someone talks about nerves before going on stage. What comes back is a set of time ranges drawn from different videos in that project, each with a line saying why it was picked, and each one opens in a results page where you can preview them in turn. Results you keep can be exported as the original clip and passed to whoever is cutting.

The step people skip

The review step is not optional, and it is where most of the value of the loop shows up. A description-based search returns things that are related in ways you did not specify. Ask for moments about stage nerves and you may get a passage that is really about how many years it took to relax in front of an audience, which is either exactly what you wanted or not what you wanted at all. No model settles that. A person watches the range and decides, which is why the review pass has its own shape rather than being a rubber stamp.

The handoff deserves the same attention. A list of time ranges is only useful to an editor if it says which recording, which range, and why it was chosen, which is the same discipline as briefing an editor from footage you already own.

When you do not need an agent in the loop

If your archive is small enough that you remember what is in it, the workflow is scrubbing, and it is faster. If everything you need is in one recording you shot last week, open it. If the job is tagging and filing rather than finding specific moments, a filesystem connector and a naming convention will take you further than any retrieval layer. Agents earn their place when the question is about content you cannot remember the location of.

Where this route stops

A search runs inside one project, so the project is the unit you plan around rather than the whole archive. Video has to be uploaded and indexed before it can be searched, which makes ingest a real step in your workflow rather than a background detail. Searching consumes an allowance, so a loop that fires dozens of speculative queries costs something. And what comes back is a range you can open and preview, not a frame-accurate in and out point, so anything going to an edit gets padded on both sides.

If your footage is already uploaded and your questions are about content, the loop is worth building. If your questions are really about organisation, build the naming convention instead.

FAQ

Can an AI agent search video files on my computer?

Not directly. An assistant with a filesystem connector can read file names, sizes, and folder structure on a machine it has access to, but it cannot see the picture or hear the audio inside those files. Searching by content requires the video to have been indexed somewhere first, which means the footage sits in a service that did that work.

Do I have to rebuild the index every time I start a new chat?

No. Indexing happens when the video is ingested, not when you ask a question. Once a project has been indexed, a new conversation with the same account queries the existing index. What resets between chats is the conversation, including any list of results you were working from, so save the ranges you care about somewhere durable before you close the window.

How do I tell an agent workflow is worth the setup?

Count the questions you currently answer by scrubbing. If a week goes by with three or four requests along the lines of "we talked about that somewhere, find it", the index pays for itself. If those requests come up twice a year, the setup will decay before you use it again.

What happens when the search returns nothing?

An empty result is ambiguous, and it is worth treating as a signal rather than an answer. It can mean the moment is not in the footage, or that the description was too narrow for the index to match anything. The practical response is to widen the wording once before concluding the footage does not contain it.