Give the agent a video search tool over the episodes, not the raw files. Export each episode's camera stream to a video file, upload and index the batch in a system that matches plain-language descriptions to what's on screen, then have the agent ask for the behavior you want ("the gripper closes but the object slips out") and collect the matching episodes and time ranges into a list a person reviews. The agent does the asking and the bookkeeping. Your team still decides what counts as a failure.
Why episode video is awkward to search
A robot dataset is usually a large number of short recordings. The file names say which task, which setup and when, but not how the episode went. A success flag tells you the episode failed, not what the failure looked like: a slip in the grasp, an arm hovering and hesitating, an object knocked out of reach, the arm sitting idle while it waits for a command. Those are visible in the video and rarely written down anywhere.
The usual answers to "AI video management" point at asset managers, which store, tag and share files. That helps you find the episode. It doesn't help you find the moment inside the episode where the object slips.
Four routes
Paste frames into a chat. Useful for asking about one episode you already picked. It doesn't work across a batch.
A filesystem connector. The agent sees episode names and folders and can filter by task or date. Nothing about the content comes through.
Build your own index. Sample frames from each episode, embed them, store the vectors with timestamps, and expose a search tool to the agent. You control the models and where the data lives, and you maintain all of it. Extracting embeddings with timestamps is where most of the cost sits. For the general mechanics, see how AI search inside footage works.
A hosted video search connector. You upload the episodes to a service that indexes them and exposes search over MCP. Less to build and run, but the video has to live in that service.
Ask your agent for failed grasps across a batch of episodes
Vivu connects to any assistant or agent that supports MCP, such as Claude or ChatGPT. Episodes recorded as ordinary video files upload directly to a Vivu project (camera streams inside ROS bag or MCAP need exporting to video first), and each one is indexed once in the cloud, so the same batch can be searched again next week from a new conversation. Because episodes are short, a fast search mostly tells the agent which episodes are likely relevant, returning whole recordings, while a precise search, which the agent has to wait on, narrows to the stretch inside each episode and adds a one-line reason. From there the agent can build a review sheet with episode, time range and reason, and a person previews each segment on the results page before deciding whether it goes into the eval or training set.
A good split is fast search first to pick the episodes, then precise search for the ones you'll actually review.
Where this route stops
Vivu searches one project at a time and only finds what's visible in camera video that has been uploaded and indexed in the cloud. There's a search allowance, with a free tier and paid tiers and the limits listed on Vivu's pricing page, and results are openable time ranges, not frame-exact boundaries, so whoever cuts training clips still sets the exact in and out points. Why a grasp failed stays an engineering question. The video shows that it failed, and your logs and your team work out the cause.
Skip the search layer entirely if your episode outcomes are already logged reliably and you only need counts, or if the batch is small enough to watch in one sitting.
Picking a route
If the failures you care about show up in your logs, query the logs. If you only ever notice them by watching, give the agent something that has already watched the episodes. Then choose between building that index and uploading to one based on whether your team wants to maintain it.
FAQ
Can an agent tell a failed grasp from a successful one in video?
It can find segments that match a description, like "the object slips out of the gripper after lifting". Whether that counts as a failure under your definition is a reviewer's call. Write queries around what's visible, not around the label you want.
What does an empty search result mean?
Either the behavior isn't in those episodes or the query was too strict. Loosen the wording once, or search for an episode you know contains the behavior, before concluding it isn't there.
Can this be used on teleoperation recordings?
Yes, if the camera footage is exported as video. It's useful for finding moments like the arm sitting idle between commands. It isn't a way to grade operators, and it shouldn't be used as one.