Recipe card from cooking videos

---
name: recipe-card-step-frames
description: "Turn your own cooking videos into a recipe card with Vivu: amounts read from the on-screen captions, one photo per step, a finished-dish shot and timestamped links. Use before writing the blog post."
---

Recipe card with step frames from your cooking videos

This skill takes the cooking videos a creator has on their own drive, plus a rough list of the recipe's steps, and returns a recipe card draft to paste into a blog post or a video description: one row per step with the source file, the time, the amount caption read from the picture (for example "3 TBSP | SOY SAUCE"), what the creator said in that step when there is narration, a photo of the step taken from the footage, a finished-dish photo, and a link to the creator's public video at that second. Claude measures the videos, prices the job against the user's Vivu plan, uploads them to a private Vivu project, runs one search for every amount caption and a few searches for the steps that have none, checks every window on full frames, sweeps each video with a cheap local contact sheet for captions the search missed, and writes recipe_card.csv and recipe_card.md.

The value is in what only the picture holds. Many recipe videos print the amounts on screen and never say them; plenty have no narration at all, only music. Auto captions and transcripts miss every one of those numbers, and the step photos can only come from the footage. The risk is a wrong amount in a published recipe, so every amount in the card is copied from a full frame, never from Vivu's reason text, and anything Claude could not read is marked UNCERTAIN for the creator to fill in.

When to use

Use when a creator says "make a recipe card from my salmon video", "pull the ingredient amounts and step photos from these three cooking videos for my blog", "write the recipe for my video description with timestamps", or "I need step by step photos from my recipe footage".

Not for these:

1. One question about one video ("how long did I bake it for?"): search Vivu directly and look at the frame.
2. Subtitles, captions files, edits, short clips or social cuts. This skill reads the footage and writes a card; it does not cut or deliver video.
3. Someone else's videos. Use only footage the creator owns or has the rights to.
4. Nutrition facts, scaling a recipe, or converting units. The card copies what the video shows; Claude can help with those separately, labeled as its own calculation.

Working principles

1. Report measured numbers, not estimates. When a number is an estimate, say so.
2. Nothing is verified until it has been checked against the source. A Vivu window is a candidate until a full frame shows the caption; the amount in the card is copied from that frame. Spoken words come from the creator's own captions file or a local transcript, never from the reason text, which paraphrases.
3. Stop and tell the user when a required capability or tool is missing. Do not guess around it.
4. Ask the user before anything that is expensive to redo (the index minutes, before upload) and before anything that acts on their behalf. This skill writes local files only; publishing the card is the creator's own step.
5. A step with no amount caption gets "NO AMOUNT ON SCREEN", and an amount Claude cannot read gets UNCERTAIN. Never fill an amount from memory, from a similar recipe or from the reason text.

What you need before starting

Check each item at the start of the run and tell the user plainly what is missing before doing anything else.

This skill needs Claude Code on the user's computer (the terminal or the Code tab of Claude Desktop), because it reads local video files and runs ffmpeg. Claude on the web and cloud sessions cannot do that. Nothing recurs, so no scheduler is involved; run it once per recipe. The only connector it needs is Vivu.

Inputs to collect

Ask for anything missing, most important first.

1. VIDEO_FOLDER and the files that belong to this recipe. Required.
2. The step list: the creator's draft, one line per step. Default: Claude drafts it from vivu_get_video_summary and the creator corrects it in Step 2.
3. The public link for each video (YouTube or other). Default: none; the card then gives the file name and the time instead of a link.
4. WORK_NAME: a short name for the working folder and the Vivu project. Default: the recipe name, for example maple-salmon.
5. Where the card goes. Default: local files only; the creator pastes the card into their blog or description themselves.

Files and state

Keep everything in one working folder:

recipe-WORK_NAME/
  config.json         VIDEO_FOLDER, Vivu project id, public links, query wording
  videos.csv          file, duration_s, video_id, public_url
  steps.csv           the step list: step, step_text
  results/            one JSON per search, without the result page link
  sheets/             contact sheets of windows and whole videos
  read/               full frames the amounts were read from
  steps/              the step photos and finished.jpg
  candidates.csv      every window Vivu returned or the sweep found, with the verdict
  recipe_card.csv     one row per step
  recipe_card.md      the card to paste
  state.json          steps done, files uploaded, searches run with job ids, windows checked

Run every command from the folder that contains recipe-WORK_NAME/. A rerun reads state.json first: a file marked uploaded is not uploaded again, a search with a saved result is not run again, and a window with a verdict in candidates.csv is not checked again.

Step 1: Check the Vivu connector and the setup

Goal: confirm every requirement before spending anything.

1. Call vivu_get_account. If the tool does not exist, tell the user to add the Vivu connector in Claude (https://mcp.vivu.ai/mcp) and stop. If the account does not show can_create_projects: true, or a later write call fails with "has not granted vivu.write", ask the user to reconnect Vivu and allow write access, then stop until they have.
2. Run ffmpeg -version and ffprobe -version, and list VIDEO_FOLDER.

Done when vivu_get_account shows can_create_projects: true, both commands print a version, and the video files are listed.

Step 2: Collect the videos, the step list and the links

Goal: the files, the step list and the public links written down.

1. Confirm the Compliance items with the user.
2. Write videos.csv with each file and its public link. Rename files that contain spaces or brackets to plain names (for example salmon.mp4), because Vivu replaces spaces and punctuation with underscores and plain names make results easy to match.
3. Write steps.csv from the creator's draft. When there is no draft, write one after Step 4 from vivu_get_video_summary and ask the creator to correct it.
4. Only if the creator has no local copy: download their own published video at 720p from a machine with a residential IP (YouTube blocks cloud IPs). 720p keeps caption text readable. VIDEO_ID is the 11 character ID in the creator's link:

yt-dlp --js-runtimes node --restrict-filenames \
  -f "bv*[height<=720]+ba/b[height<=720]" --merge-output-format mp4 \
  --match-filter "duration<=3600" \
  --download-archive recipe-WORK_NAME/archive.txt \
  -P "home:VIDEO_FOLDER" -o "%(id)s.%(ext)s" --retries 5 \
  "https://www.youtube.com/watch?v=VIDEO_ID"

The youtube-competitor-watch skill (https://vivu.ai/skills/youtube-competitor-watch) covers channel verification and download troubleshooting in more depth. If yt-dlp fails with "unable to download video data", retry once with --extractor-args "youtube:player_client=web_embedded".

In our test run the videos were three public Creative Commons recipe videos downloaded this way, standing in for a creator's own files, so collecting a creator's draft step list was not exercised in our test run.

Done when videos.csv lists every file and steps.csv has one row per step (or the user has agreed to draft it after Step 4).

Step 3: Measure and price

Goal: the job priced against the user's plan and approved before upload.

1. Measure each file and write duration_s:

ffprobe -v error -show_entries format=duration -of csv=p=0 VIDEO_FOLDER/FILE.mp4

FILE is the video's file name without .mp4, as written in videos.csv (for example salmon).

2. Call vivu_get_usage and show one table:

Plan facts: Free is $0 a month with 20 index minutes a month and 50 search credits a month; Premium is $30 a month with 180 index minutes and 500 search credits. A precise search uses 5 credits in total. A fast search uses 1 credit; this skill does not use fast. A 12 minute recipe video uses 12 index minutes, so Free covers one such video a month (estimate).

3. If the job does not fit, offer levers in this order: index only the final cut, not the raw takes; index only the videos for this recipe; split the work across two months; and only then move to a larger plan. Never trim or recompress a video to save minutes.

The account in our test run returns no allowance figures, so the plan table and the user's approval were not exercised in our test run.

Done when every file has a duration_s and the user has approved the minutes and credits.

Step 4: Upload and index

Goal: every video indexed in a private Vivu project.

1. Call vivu_list_projects and reuse a project named "Recipe WORK_NAME" if one exists. Otherwise call vivu_create_project with that name and visibility "private". The default visibility is organization, which shows the project to everyone in the user's Vivu organization.
2. Call vivu_open_upload_page with the project ID immediately before uploading. The link expires in 180 seconds and works once, so never post or store it. Give it to the user to open in their own browser and select the files, or open it in a browser tool that can attach local files. Claude in Chrome accepts at most 10 MB per upload call, and recipe videos are usually larger, so most users will open the link themselves or add the files in the Vivu web app. Never split or recompress a file to fit.
3. Poll vivu_list_videos until every file shows ready, then record each video_id in videos.csv. Match listed names to local names; Vivu replaces spaces and punctuation with underscores.

In our test run the files went through the upload page with an automated browser, so opening the link in the user's own browser was not exercised in our test run.

Done when every row of videos.csv has a video_id and every file shows ready.

Step 5: Search for amount captions

Goal: candidate windows for every amount printed on screen.

One precise search covers all the videos in the project. The query describes the kind of caption, not its words: a query that quotes the caption itself has led Vivu to attach that caption to windows that do not show it.

maximum_results 30 fits three to five recipe videos with up to about ten amount captions each; it is also the recall ceiling, so raise it for more videos. fast is not used: it returns whole files with an empty reason, and the files are already known.

Run vivu_search_videos (project_id, query, mode "precise", maximum_results 30). It returns a job ID; call vivu_get_search_results until complete is true. Each status call can wait up to 45 seconds, so a pending search is not a stalled one. Save the result to results/amount_caption.json without the result page link, and show the result page link only in the live reply: it expires after four hours, so it never goes into the card or any file.

Done when results/amount_caption.json is saved and its job id is in state.json.

Step 6: Read every caption from full frames

Goal: each window labeled kept or rejected, with the amount copied from a full frame.

1. Make a contact sheet across the window, one tile every half second. A window often holds several captions in a row:

ffmpeg -v error -y -ss START -to END -i VIDEO_FOLDER/FILE.mp4 -vf "fps=2,scale=320:-1,tile=8x6" -frames:v 1 recipe-WORK_NAME/sheets/FILE_START.png

START and END are the window in seconds (start_ms and end_ms divided by 1000). Tile k is at START plus k/2 seconds, counting from 0 at the top left; one sheet covers 24 seconds, so make another sheet from START plus 24 for a longer window. 2. For each caption on the sheet, extract a full frame where it is fully shown and read it:

ffmpeg -v error -y -ss SECONDS -i VIDEO_FOLDER/FILE.mp4 -frames:v 1 -q:v 3 recipe-WORK_NAME/read/FILE_SECONDS.png

3. Copy the caption exactly as written, units and symbols included (210°C/410°F, 8'-10'). Kept: a caption with a number and a unit. Rejected: a window with no amount caption on any frame; count it as a false positive. Text that is not legible is UNCERTAIN.
4. Write every window into candidates.csv, one line per caption found in it.

Worked example from our test run

The test corpus was three public recipe videos (15.95 minutes): a salmon meal prep with amount labels in the lower left and music only during the steps, a vertical phone video of french fries with instruction text at the top and music only, and a banana bread video whose voiceover is burned in as full captions, so its amounts are both said and written. Before searching we went through every video on a contact sheet and wrote down every amount caption, the captions without numbers ("GRAB A BOWL", "SPRINKLE SALT & PEPPER") and the finished-dish shots. All files were ready about 11 minutes after the upload started.

The amount caption search returned 8 windows: 8 real, 0 false, and 6 captions missed. No caption without a number was returned, and the reason text matched the frames. The median window was 14 seconds wide, but the widest held the whole marinade sequence, several captions in a row, while the reason named only the first; the half second sheet is what turned it into separate rows. Every caption in the vertical phone video was missed although each is legible on a full frame, and the remaining misses were short caption lines in the banana bread video. Step 7 found all of them.

Done when every window has a verdict in candidates.csv and each kept caption has a full frame in read/.

Step 7: Sweep each video for captions the search missed

Goal: no amount caption left out because the search did not return it.

Recall is the weak side of the search, especially on vertical or low resolution video. Sweep every video with a contact sheet of the whole file, one tile every 2 seconds. This is local and costs no credits:

ffmpeg -v error -y -i VIDEO_FOLDER/FILE.mp4 -vf "fps=1/2,scale=400:-1,tile=6x8" recipe-WORK_NAME/sheets/FILE_sweep_%02d.png

Use scale=270:-1 and tile=8x4 for a vertical video. Sheet n (counting from 1) starts at (n - 1) times 96 seconds for the 6x8 layout, or 64 seconds for 8x4; tile k is 2 times k seconds after that. Captions shorter than 2 seconds can fall between tiles, so check the 2 seconds around each caption the search returned and around each step boundary with the Step 6 half second command. For each caption on the sweep that is not already in candidates.csv, extract a full frame with the Step 6 command, read it, and add it with the verdict "added from sweep".

In our test run the sweep sheets showed all 6 captions the search missed, and the vertical video's captions were legible on full frames at its low resolution.

Done when every video has sweep sheets and every amount caption on them is in candidates.csv.

Step 8: Find the steps that have no amount caption

Goal: a time and a check for every step in steps.csv.

Map each kept caption to its step in steps.csv by time and by what the frame shows. For each step still without a time, run one precise search with maximum_results 5 (one step happens once or twice in a video, and 5 keeps a little room). Write the query in the creator's words for that step. When the video has narration, describe what is said; when it has music only, describe what is done, which turns the same step from said to shown:

Check a step_said window with vivu_get_video_summary (project_id, video_id, include_segments true, start_ms and end_ms around the window), then copy the words from the creator's captions file or a local transcript into the said column; mark it NOT VERIFIED when neither exists. Check a step_shown window on a half second contact sheet (Step 6 command): kept only when the frames show that step being done. Finding an action from the picture alone is a weak Vivu capability, so a step_shown window is never kept without that check.

In our test run the step_said search returned 1 window, 1 real, and the summary segment agreed with the burned-in captions. The step_shown search on the music-only video returned 2 windows: 1 real and 1 false, the false one being the opening montage of fries already frying. A control search for a step none of the videos contain (adding vanilla extract) returned 0 windows.

Done when every step in steps.csv has a time with a kept window, or "not found in the video".

Step 9: Pick the step photos and the finished shot

Goal: one photo per step and one finished-dish photo.

1. Run one precise search for the finished dish:

maximum_results 10 leaves room for two or three hero shots per video. 2. For each step and for the finished dish, pick the clearest tile on the window's half second sheet: food in focus, no motion blur, no hands covering it. Skip frames with people in them; windows often start or end on the host, and the creator decides whether faces go on the card. Save a full frame:

ffmpeg -v error -y -ss SECONDS -i VIDEO_FOLDER/FILE.mp4 -frames:v 1 -q:v 2 recipe-WORK_NAME/steps/NN.jpg

NN is the step number from steps.csv with two digits (01, 02, ...); for the finished dish save it as recipe-WORK_NAME/steps/finished.jpg instead.

In our test run the finished-dish search returned 5 windows, all 5 real, at least one in every video; some of them ended on the host at the counter, so the photo came from earlier tiles.

Done when steps/ has a photo for every step that has a time, plus finished.jpg.

Step 10: Assemble the recipe card

Goal: a card the creator can paste after checking it.

1. Show the creator one sample row and the field mapping, and wait for a yes before writing the rest:

step,step_text,source_file,time,amount_caption,amount_source,said,said_checked,photo,video_link
2,Add the salmon and marinate,salmon.mp4,01:48,"170G/6OZ | SALMON FILLETS, SKIN ON OR OFF; MARINATE | 30 MINS - OVERNIGHT",search,,no narration,steps/02.jpg,https://www.youtube.com/watch?v=VIDEO_ID&t=108

2. Write recipe_card.csv and recipe_card.md. The markdown has the finished photo, an ingredient list built from the kept captions, the numbered steps with photos and time links, and a "Check before publishing" list of every NO AMOUNT ON SCREEN, UNCERTAIN and NOT VERIFIED item.
3. Tell the creator the counts: steps with an amount, steps without, captions found only by the sweep, windows rejected.

The creator's approval of the sample row was not exercised in our test run; the card was written for the salmon video and nothing was published.

Done when recipe_card.csv has one row per step in steps.csv, recipe_card.md exists, and the creator has approved the sample row.

Compliance

1. Use only videos the creator made or has the rights to use. Confirm this before Step 2. When the creator downloads their own published video, it is for this card only.
2. People on camera: step photos show food and hands. Leave out frames with faces unless the creator wants them, and leave out anyone who did not agree to appear, including children; a child's face never goes on the card.
3. The videos stay in the user's Vivu project until the user deletes them. Create the project as private. Delete a project or video only when the user asks, and confirm first.
4. The skill writes local files only. Publishing the card to a blog, a description or social media is the creator's own action after checking it.
5. No face or logo recognition. Brand names on packaging are copied only when they are part of an amount caption the creator wrote.

Known failure modes