Live shopping product index

---
name: live-shopping-product-index
description: "Index where the host pitched each product in a live shopping replay with Vivu: times, pitch line and price card read from frames, as a CSV for the editor. Use after a stream."
---

Product segment index from live shopping replays

This skill takes a live shopping replay that the user downloaded from their own stream's back office (a two or three hour mp4) and the list of products in the order they were pinned, and returns a product index for the editor. Each row is one stretch where the host pitched one product: the product, the source file, start and end as MM:SS, the host's selling line copied from the transcript, the price card text read from a full frame, that frame's path, and an empty "editor took" column. Claude cuts an eight minute window around each product's pin time, prices the indexing against the user's Vivu plan, indexes the windows in a private Vivu project, runs one speech search and one on screen search per product, checks every candidate against the transcript and against frames, and writes the CSV. The editor cuts the clips by hand.

The value is in what the pin log and the transcript each miss. The back office only records when a product link was pinned, and hosts often start talking about a product before it is pinned, keep talking after the next card goes up, or come back to it twenty minutes later. The price card and discount badge exist only in the picture: the transcript says "the discount ends when the stream ends", never the price or the code. Two models in the same family (a 5 quart and a 7 quart fryer) sound alike in speech and are told apart by the card. So the skill searches speech for the pitch, searches the screen for the card, rejects pitches that only mention a product in passing, and reads every price from a full frame of the user's own file.

When to use

Use when a live commerce operator or stream producer says "make a product index of last night's live for the editor", "where did she talk about the air fryer in the replay", "pull every product pitch and its price card from the stream", or "the pin times are wrong, find when each product was actually pitched". For one question about one product in one replay, search Vivu directly. This skill works after the stream on the user's own replay; it does not process a stream while it is live, does not cut or publish clips by itself, and does not judge sales, GMV or which pitch worked better.

Working principles

1. Report measured numbers, not estimates. When a number is an estimate, say so.
2. Vivu's results are candidates. A row is verified only after the transcript shows the host pitching that product (not comparing it in passing) and a full frame shows that product's card. Rejected candidates are counted and kept in rejected.csv with the reason.
3. Stop and tell the user when a required capability or file is missing (no replay file, no pin list, no transcript, no ffmpeg, no write access in Vivu). Do not guess around it.
4. Ask the user before anything that is expensive to redo or that acts on their behalf: the cut windows and the indexing cost before upload, and the sample rows before the full index.
5. Prices, codes and badge text come from frames only, never from Vivu's reason text or from memory of the product page.

What you need before starting

Check each item at the start of the run and tell the user plainly what is missing.

The skill needs a shell and local files, so it runs in Claude Code on the user's computer (terminal or the Code tab of Claude Desktop). It downloads nothing from YouTube, so no residential IP is needed. Nothing is scheduled and nothing is written outside the working folder.

Inputs to collect

Ask for anything missing, most important first.

1. The replay file and its transcript (REPLAY_FOLDER). Required.
2. The pin list: product name and pin time for each product. Required.
3. Which products to index. Default: every product on the pin list, up to the number the plan covers (Step 3).
4. Window size. Default: 3 minutes before the pin time to 5 minutes after it, about 8 minutes per product.
5. The Vivu project name. Default: "Live replay STREAM_DATE", private.

Files and state

Keep everything in one working folder next to the replay:

live-index/
  windows.csv        product, pin_time, cut file, CUT_START, CUT_END, minutes
  cut/               one window per product, the files that get uploaded
  results/           raw search results, one JSON file per search
  frames/            contact sheets and full frames read for each row
  product_index.csv  one row per verified or candidate pitch
  rejected.csv       candidates that failed a check, with the reason
  state.json         uploaded files and video ids, searches run with job ids, every candidate with its verdict, rows written

state.json is updated after every step. A rerun reads it first: windows already uploaded are not uploaded again, searches already run are not rerun, and checked candidates keep their verdicts, so an interrupted run resumes where it stopped.

Step 1: Check the Vivu connector and the setup

Goal: every row of What you need is confirmed before anything is cut or uploaded.

1. Call vivu_get_account. If the tool is missing, tell the user to add the Vivu connector in Claude (https://mcp.vivu.ai/mcp) and stop. If can_create_projects is not true, or a write call later fails with "has not granted vivu.write", ask the user to reconnect Vivu with write access and stop until they have.
2. Run ffmpeg -version and ffprobe -version. List REPLAY_FOLDER and check that the replay has a transcript file.
3. Create the working folders; ffmpeg does not create an output folder:

mkdir -p live-index/cut live-index/results live-index/frames

Run every later command from inside live-index/ (cd live-index).

Done when vivu_get_account shows can_create_projects: true, ffmpeg and ffprobe print versions, and the working folders exist.

Step 2: Confirm consent and collect the inputs

Goal: the user has confirmed the replay may be used, and Claude has the pin list and the transcript.

1. Ask Compliance items 1 to 3 before anything else in this step. Stop if the replay fails them.
2. Collect the pin list as product name plus pin time (HH:MM:SS into the replay). If the back office shows clock times instead, ask for the stream start time and subtract it.
3. Confirm the transcript's times are offsets into the same file (the first caption should fall near the host's first words).

In our test run the pin list and transcript came from the synthetic fixture's own script, so this step was not exercised in our test run.

Done when the user has confirmed Compliance items 1 to 3 and the pin list is written into windows.csv.

Step 3: Cut one window per product

Goal: one short file per product, so index minutes go only where the pitches are.

1. For each product, set CUT_START to 3 minutes before its pin time and CUT_END to 5 minutes after it. Hosts often start a pitch before the pin and keep talking after the next pin, so the margin goes on both sides. Merge windows that overlap into one file.
2. Show the windows to the user (product, CUT_START, CUT_END, minutes) and let them adjust.
3. Cut with re-encoding, so the cut starts exactly at CUT_START and every Vivu time maps back to the replay by adding CUT_START (REPLAY_FILE is the original file, N is the product's number in the pin list):

ffmpeg -v error -ss CUT_START -to CUT_END -i REPLAY_FILE -c:v libx264 -c:a aac cut/window_N.mp4

A stream copy (-c copy) is faster but starts at the nearest keyframe, a few seconds early, and then every time in the index is off by an unknown amount.

In our test run each synthetic file stood in for one cut window; the cut command itself was run once on a test window and came out exact.

Done when cut/ holds one file per window and windows.csv records each CUT_START.

Step 4: Price the indexing and get approval

Goal: the user sees the cost before anything is uploaded.

1. Measure each cut file and sum the minutes. ffprobe takes one input file per call:

ffprobe -v error -show_entries format=duration -of csv=p=0 cut/window_N.mp4

2. Call vivu_get_usage for the plan and what remains this month.
3. Search credits: two precise searches per product (pitch and card), 5 credits each, plus 5 for each rewording if a first page is not clean. Label the total as an estimate.
4. Show one table and wait for approval:

For reference, the Free plan has 20 index minutes a month and 50 search credits a month; Premium is $30 a month with 180 index minutes and 500 search credits. As an estimate, six products at about 8 minutes each come to about 48 index minutes and 60 search credits per stream; Free covers about two products, and indexing a whole three hour replay would use all 180 Premium minutes for the month. If the windows do not fit, offer these levers in order: narrow each window around the pin time, index the products the editor needs first (say which ones are left out), and only then a larger plan.

In our test run the account was an admin account whose vivu_get_usage shows no remaining allowance, so the comparison against a real plan and the approval were not exercised in our test run.

Done when the user has approved the minutes and the credit estimate.

Step 5: Upload and index

Goal: every window is ready in a private Vivu project.

1. Call vivu_list_projects and reuse the project named in the inputs if it exists. Otherwise call vivu_create_project with that name and visibility "private". Replays can show prices and promotions that are not public yet; the default visibility is the whole organization.
2. Call vivu_open_upload_page with the project ID right before the upload. The link expires in 180 seconds and is a sign in link, so never paste it into a message or a file. Give it to the user to open in their own browser and choose the files in cut/, or attach them with a browser tool that can upload local files. Claude in Chrome accepts at most 10 MB per upload call; an 8 minute window at 1080p is usually larger, so the user adds those files in the Vivu web app. Never split or recompress a window to fit.
3. Poll vivu_list_videos about every 30 seconds until every file shows ready. Match each video back to windows.csv by file name; Vivu turns spaces and punctuation in names into "_". Record the video IDs in state.json.

In our test run the files went up through a script, not through a user's browser or Claude in Chrome; that upload path was not exercised in our test run.

Done when vivu_list_videos shows every window in windows.csv as ready.

Step 6: Search the pitches and the cards

Goal: two raw result files per product.

For each product, run both queries with vivu_search_videos (project_id, query, mode "precise", maximum_results 10). PRODUCT is the product name from the pin list, written the way the host says it ("Breeze 5-quart air fryer", not the SKU). Each call returns a job ID. Call vivu_get_search_results until complete is true; each status call can wait up to 45 seconds, so a pending search is not a stalled one. Save each completed result as results/pitch_N.json and results/card_N.json. Show the result page link in the live reply only; it expires after four hours, so it never goes into the index or any file.

Both are precise because the index needs time ranges; a fast search returns only whole files with no reason, and the windows are already chosen. maximum_results 10 is more than one window usually holds for one product, and it is the ceiling on what comes back: a list that stops at exactly 10 means the cap was hit, so raise it and rerun that search.

Why this chain: the pitch search finds when the host talked about the product, which is the clip the editor wants; the card search flips from speech to the screen to find when that product's card was up, which is where the price and badge are, and which model was really on sale. The card query names the kind of thing on screen (a price or a badge), not the price itself, so a wrong guess at the price cannot steer the search.

Done when results/ holds two result files per product and state.json records the job IDs.

Step 7: Check every candidate

Goal: every candidate has a verdict, every pitch row has a line from the transcript, and every price is read from a full frame.

Vivu returns a time range that contains the moment, not the exact second, and the reason text is a paraphrase. Pitch and card windows often run past the moment into the next product. Two risks drive this step. Speech searches by topic have, in other tests on interviews and customer story videos, returned passages that only mention the topic in passing, which here would be a comparison with an earlier product. On screen text searches have, in other tests, come back with a reason that invents small text and digits. Neither happened on our clean test replay, but real streams are messier, so the transcript and the frame decide every row.

1. Pitch candidates: read the transcript lines inside the window. Keep the candidate only if the host is presenting this product: its features, its price, why to buy it. Reject it when the product is only mentioned while the host sells something else ("compared with the five quart I showed you earlier"), when the host is answering shipping or general questions with the card still up, or when the lines are about a sibling model. Set the row's start and end to the first and last transcript line of the pitch, not the window edges. Copy one or two selling lines word for word into pitch_quote.
2. Card candidates: make a sheet of one frame every half second across the window (START is start_ms / 1000, DURATION is (end_ms - start_ms) / 1000; the 10x11 grid holds 55 seconds, so for a longer window raise the second number):

ffmpeg -v error -ss START -t DURATION -i cut/window_N.mp4 -vf "fps=2,scale=320:-2,tile=10x11" -frames:v 1 frames/card_N_sheet.png

Keep the quotes around the filter. Find the cells where this product's card is up and pick one in the middle. Extract that moment as a full frame (SECONDS is the cell's time: START plus half a second per cell before it) and read the product name, price and badge from it:

ffmpeg -v error -ss SECONDS -i cut/window_N.mp4 -frames:v 1 -q:v 3 frames/card_N_read.png

Never take the price from the first or last cell: windows often start before the card is pinned and end after the next card is up. Reject the candidate if the card in the frame is a different product or a sibling model. 3. Match each verified pitch to the card that was up during it. When the sheet shows another product's card under the pitch (the host started before switching the pin), write CARD_MISMATCH with the times in card_note, and take price_card_text from the first frame where this product's own card is up. 4. A product whose card was found but whose pitch search found nothing gets one row with status card_only: it was pinned but, as far as the search shows, not pitched. An empty pitch result does not prove the host never mentioned it; say so in card_note. 5. Read vivu_get_video_summary with include_segments, start_ms and end_ms around a window only as a second signal. Rejected candidates stay in results/ and go into rejected.csv with the reason; they are counted, not dropped silently.

Worked example from our test run

In our test run we used 2 synthetic replay windows, 8.5 minutes in total, made for the test: a live room with a host silhouette, a chat panel and a pinned product card (name, large price, yellow badge), and a synthetic voice for the host. Six products were set up to trip the weak spots: a smaller and a larger fryer from the same family, a pair of standard and Pro earbuds, a kettle whose card was pinned while the host only read comments, a pitch that started before its card was pinned, a pitch under the previous product's card, a product the host came back to later, and pitches that compared back to an earlier product. The host never said a price, a badge or a code. We wrote down every pitch and every card interval before searching. Both files were ready about 2.3 minutes after the upload page was opened, and the run used 12 precise searches.

For the products the host pitched, the pitch searches returned 6 windows: 6 real, 0 false and 0 missed, including the fryer pitch that began before its card and the later return to it. None of the comparison passages came back as a pitch of the earlier product, and the kettle pitch search returned 0, so the card on screen did not leak into the speech search. The card searches returned 7 windows: 7 real, 0 false and 0 missed, and neither sibling model's card was returned for the other. This was the first wording of each query on this corpus. Pitch windows were 24 to 64 seconds wide and card windows 34 to 64 seconds, wider than the moments: the smaller fryer's card window ended on the larger fryer's card and its higher price, and the Aero Pro card window ended on the next product's card, which is why the price comes from a frame in the middle of the card's cells. The Aero Pro pitch window started while the standard Aero card and its price were still pinned. The video summary for the earbuds file put the Aero Pro pitch inside a segment titled "Aero Wireless Earbuds" and said the host shares pricing, although prices were only on screen. These were clean synthetic screens and one clear voice with pauses between products; real streams with continuous talk, background music, a moving camera, small or animated price stickers and a full length replay were not exercised in our test run, and the transcript check used the fixture script instead of a speech to text tool.

Done when every candidate has a verdict in state.json and the user has seen the real and rejected counts per product.

Step 8: Assemble the product index

Goal: one CSV the editor can work from.

1. For each row, convert times to the full replay: start_mmss and end_mmss are CUT_START plus the transcript times in the cut, written as minutes and seconds (hours too for long replays).
2. Write the rows in time order:

product,source_file,start_mmss,end_mmss,pitch_quote,quote_source,price_card_text,frame_path,card_note,status,editor_took
Breeze 5-Quart Air Fryer,REPLAY_FILE,00:20,01:05,"Let's start with the Breeze five quart air fryer. This is the compact one",platform captions,Breeze 5-Quart Air Fryer | $59.99 | 15% OFF TODAY,frames/card_1_read.png,no card on screen 00:20-00:50,verified,

3. Field mapping:

4. Show two or three sample rows and the mapping and wait for a yes before writing the full index. Say which parts are inferred: where a pitch starts and ends is Claude's reading of the transcript, and a product with no pitch row was not found, not proven absent.
5. Write product_index.csv and rejected.csv. Give the user the counts per product and list every CARD_MISMATCH, because those are the clips where the price on screen is not the product being sold.

In our test run the index had a row for each verified pitch plus one card_only row for the kettle. The user's approval of the sample rows was not exercised in our test run.

Done when product_index.csv exists with a row for every verified or candidate pitch and every card_only product, and the user has approved the sample rows.

Compliance

1. Use only the user's own stream replays (their store or brand, or a stream they have the rights to). Do not process another seller's stream recording. Ask before Step 2.
2. Hosts and guests on camera: check that their contracts or release cover reuse of the recording for clips. If minors appear on camera (kids' clothing try ons, family guests), the skill needs a guardian's consent for them or it does not process that replay. Ask before Step 2.
3. Chat on screen shows viewers' handles. The index never copies chat text or handles. Ask before Step 2 whether the replay may be uploaded to a third party service.
4. The windows stay in the user's Vivu project until the user deletes them. Create the project as private, because replays can show unreleased prices. The skill never calls vivu_delete_video or vivu_delete_project unless the user asks, and confirms first.
5. The skill works after the stream on a downloaded replay. It does not cut or publish clips, does not post anywhere, and does not rate sales, GMV or how well a pitch worked. No face, logo or product image recognition: a product is identified only by what the host says and what the card says.

Known failure modes