Sea and light condition shortlists

---
name: sea-light-condition-shortlists
description: "Sort a batch of vessel camera videos into one candidate CSV per sea and light condition (fog, rough sea, sun glare, rain, dark) with Vivu. Use when picking robustness eval samples."
---

Sea and light condition shortlists from vessel runs with Vivu

This skill turns a batch of forward camera videos from recent vessel runs (camera streams your team has already exported to mp4, picked from the voyage log by date and recorded weather) into one CSV per condition on the team's list: fog or low visibility, rough sea with whitecaps, a low sun glaring on the water, rain on the lens, and dusk or dark. Claude measures the batch, prices it against the user's Vivu plan, uploads it to a private Vivu project, runs one precise search per condition, cuts a contact sheet at half second steps for every window Vivu returns, labels each window looks right, looks wrong or can't tell, and writes every candidate into its condition's CSV with an empty reviewer_verdict column. The team decides which stretches go into the evaluation subset.

The value is in what only the picture holds. A voyage log records weather once a watch ("overcast, fresh breeze"); it does not say which stretch had the sun low ahead of the bow, when the fog closed in, whether drops sat on the lens, or whether the sea carried whitecaps. Turning the boat changes a backlit stretch into a glare stretch, and fog comes and goes within minutes. Exported camera streams have no audio or captions to search. Vivu's search is used only to shortlist: in our test run the first fog wording also returned an overcast squall and rain on a window, the first rough sea wording returned the ship's own wake, and the same wording run twice returned different candidate lists. So every candidate is labeled from its frames, rejected candidates stay in the CSV, and a short list never means the batch had no such stretch.

When to use

Use when someone asks to find the foggy stretches in this month's vessel runs, to pull rough sea or rain on lens clips for a robustness eval set, to sort a batch of bow camera videos by sea state and light, or to check which recorded conditions the eval set is missing.

Not for these:

1. A one off question about one file ("is there fog in this run?"): search Vivu directly and look at the frames.
2. Finding objects: a particular ship, buoy, kayak or person. This skill buckets whole stretches by condition and never searches for targets.
3. Navigation or safety decisions, collision analysis, or anything on the vessel in real time. The CSVs are review lists made after the run.
4. Frame accurate labels, masks, boxes or a sea state number (Beaufort or Douglas scale): use the team's annotation tool and the weather record. This skill makes review lists, not labels.
5. Hundreds of hours at once. Pick a subset from the voyage log first (dates, routes, recorded weather); one run is sized for about an hour of footage.
6. Thermal or infrared streams: untested. Night footage was tested on a single clip with ship lights in view; a dark stream with no lights is untested. The skill can run on both, but treat the rows as unmeasured and have the reviewer look at every one.

Working principles

1. Report measured numbers, not estimates. When a number is an estimate, say so.
2. Nothing is verified until it has been checked against the source video. Vivu's results are candidates, Claude's labels are a first pass, and the reviewer's verdict is the decision.
3. Stop and tell the user when a required capability or tool is missing. Do not guess around it.
4. Ask the user before anything that is expensive to redo (the batch and its index minutes, before upload) and before writing the full CSVs (one sample row first). The skill posts and sends nothing; if the user asks Claude to put the CSVs somewhere, ask again before each write.
5. Labels come from frames, never from Vivu's reason text, which describes the condition it was asked for even when the frame shows something else.
6. The CSVs keep every candidate and carry no recall or precision figures. A search misses some stretches and caps how many come back, so an empty or short list does not prove the batch has none.

What you need before starting

Check each item at the start of the run and tell the user plainly what is missing before doing anything else.

This skill needs Claude Code on the user's computer (the terminal or the Code tab of Claude Desktop), because it reads local files and runs ffmpeg. Claude on the web and cloud sessions cannot reach the files. It downloads nothing, so no residential IP is needed. Nothing recurs, so no scheduler is involved; run it once per batch.

Inputs to collect

Ask for anything missing, most important first.

1. BATCH_DIR: the folder that holds the exported mp4 files for this batch. Required. A good batch follows the voyage log: a few runs for each recorded weather (clear, fog, rain, strong wind, late evening), so every condition has a chance to appear, plus some clear calm runs for contrast.
2. The conditions and what counts for each. Default: the five conditions in Step 4 with the label rules in Step 5. If the team defines a condition differently (for example, haze where the far shore is still visible counts as low visibility, or only Beaufort 6 and above counts as rough), the team's definition wins; write it into config.json before labeling.
3. BATCH_NAME: a short name such as a vessel and a date range. It names the working folder and the Vivu project. Default: the name of BATCH_DIR.
4. Batch size. Default: up to 60 minutes of video, or what remains of the plan this month, whichever is smaller.
5. Who reviews the CSVs. Default: the user.

Files and state

Keep everything in one working folder next to BATCH_DIR:

conditions-BATCH_NAME/
  config.json      BATCH_DIR, conditions with query, maximum_results and label rules, Vivu project id
  files.csv        source_file, duration_s, listed_name, video_id, log_weather
  results/         search results without the result page link, one JSON per condition
  sheets/          one folder per condition with contact sheets
  CONDITION.csv    one per condition: every candidate, Claude's label, empty reviewer_verdict
  state.json       steps done, files uploaded, conditions searched with job ids, conditions cut off, candidate keys labeled

The commands in Steps 2 and 5 run from the folder that holds both BATCH_DIR and conditions-BATCH_NAME/, so the relative paths in them resolve.

A rerun reads state.json first and skips finished steps. A file already marked uploaded is never uploaded again, a condition with a saved result is not searched again, and a candidate key (video_id, start_ms and condition) that already has a label is not labeled again. The next batch gets its own folder and its own Vivu project, so its searches do not return last month's candidates.

Step 1: Check the Vivu connector and the setup

Goal: confirm every requirement before spending anything.

1. Call vivu_get_account. If the tool does not exist, tell the user to add the Vivu connector in Claude (https://mcp.vivu.ai/mcp) and stop. If the account does not show can_create_projects: true, or a later write call fails with "has not granted vivu.write", ask the user to reconnect Vivu and allow write access, then stop until they have.
2. Run ffmpeg -version and ffprobe -version.
3. List BATCH_DIR and run the ffprobe length command on one file.

Done when vivu_get_account shows can_create_projects: true, ffmpeg and ffprobe print their versions, and ffprobe prints a length for a file in BATCH_DIR.

Step 2: Measure the batch and price it

Goal: an indexed set the user's plan can pay for, approved before upload.

1. Run the ffprobe length command on every mp4 in BATCH_DIR and write files.csv with source_file and duration_s. Add listed_name: the file name with each space, bracket and plus sign replaced by an underscore, because that is how Vivu will list it. If the user has the voyage log, copy its weather note for each file into log_weather; it helps the reviewer and costs nothing.
2. Call vivu_get_usage and show one table:

Plan facts: Free is $0 a month with 20 index minutes a month and 50 search credits a month; Premium is $30 a month with 180 index minutes and 500 search credits. A precise search uses 5 credits in total. A typical batch of twelve runs of three and a half minutes each is about 42 index minutes (estimate), which is more than Free covers in a month and fits in Premium.

3. If the batch does not fit, offer levers in this order: pick a smaller subset with the voyage log (the weather the eval set lacks most), split the batch across two runs or two months, and only then a larger plan. Never drop files the user asked for without saying which ones.
4. Confirm the Compliance items with the user.

The account used in our test run returns no allowance figures, so the comparison with a real plan and the user's approval were not exercised in our test run.

Done when the user approves the file list and its index minutes, and config.json records both.

Step 3: Upload and index

Goal: every file in the batch indexed in a private Vivu project for this batch.

1. Call vivu_list_projects and reuse a project named "Sea conditions BATCH_NAME" if one exists. Otherwise call vivu_create_project with that name and visibility "private". The default visibility is organization, which shows the project to everyone in the user's Vivu organization, and vessel footage often shows crew, other boats and harbors.
2. Call vivu_open_upload_page with the project ID immediately before uploading. The link expires in 180 seconds and works once, so never post or store it. Give it to the user to open in their own browser and select the files, or open it in a browser tool that can attach local files. Claude in Chrome accepts at most 10 MB per upload call and run videos are usually larger, so they normally go through the user's own browser or the Vivu web app. Never split or recompress a file to fit.
3. Poll vivu_list_videos until every file shows ready. Match each Vivu file name to files.csv by listed_name and record its video_id there and in state.json.

In our test run the 12 videos (18.76 minutes) were all ready 461 seconds after the upload started. They went through the same upload page with an automated browser, so opening the link in the user's own browser was not exercised in our test run.

Done when every file in files.csv shows ready and has a video_id.

Step 4: Search each condition

Goal: a saved list of candidate windows for every condition.

The fog and rough sea wordings describe only what the picture shows (the shore and horizon gone, whitecaps out to the horizon). Wordings that named the weather and the boat's motion returned look alikes, as the worked example shows; keep the picture words if the team edits them. Matching a whole picture to a condition is a weak capability: in our test run the rain and dusk searches returned more false candidates than real ones and the glare search returned one false, so treat those lists as leads and expect the reviewer to throw rows out.

The searches run side by side, one per condition, because each condition is its own list for the reviewer. Within a condition the chain is: the precise search finds candidate windows, the contact sheet shows what the camera saw across each window, Claude labels it, and the reviewer decides. Every search is precise because only precise returns a time window; fast returns whole files with an empty reason, and the batch is already a chosen subset. Exported camera streams have no audio and no on screen text about the weather, so there is no second modality to switch to; the voyage log's weather note in files.csv is the only other signal, and it is per run, not per stretch.

maximum_results is also the ceiling on how many candidates come back. 40 leaves room for an hour of footage with several stretches per run; a search that returns exactly 40 was cut off, and the batch holds more candidates than one search can return. Then tell the user, mark the condition cut off in state.json, and offer to split the batch into two smaller batches with their own projects (quote the index minutes again and wait for approval as in Step 2). In our test run no search came close to 40, so a cut off search was not exercised in our test run.

Run each query with vivu_search_videos (project_id, query, mode, maximum_results). It returns a job ID. Call vivu_get_search_results until complete is true; each status call can wait up to 45 seconds, so a pending search is not a stalled one. Save each completed result as results/FIELD.json without its result_page_url field, and record the job_id in state.json. Show the Vivu result page link in the live reply only; it expires after four hours, so it never goes into a saved file or a CSV.

Run each condition once. Running the same query again on the same project can return a different list (see the worked example), so a second run is not a check on the first. If the team changes a wording, the new wording is untested until its first candidates have been through Step 5.

In our test run 12 videos (18.76 minutes) were indexed, and the searches, including rewordings and repeat runs, used an estimate of 55 credits.

Done when every condition has a completed result saved in results/, and every condition whose search returned exactly its maximum_results is marked cut off in state.json.

Step 5: Make contact sheets for every candidate and label it

Goal: every candidate labeled looks right, looks wrong or can't tell from what its frames show.

For each result in results/FIELD.json, cut contact sheets of its window from the local file, one frame every 2 seconds:

mkdir -p conditions-BATCH_NAME/sheets/FIELD
ffmpeg -v error -y -ss START -to END -i FILE -vf "fps=0.5,scale=320:-2,tile=6x5" conditions-BATCH_NAME/sheets/FIELD/rRANK_STEM_STARTMS_%02d.png

FIELD is the condition, FILE the local source file (BATCH_DIR/ followed by its source_file), STEM its name without .mp4, RANK the result_number written with two digits (r01, r09) so the files sort by rank, STARTMS the result's start_ms, and START and END the result's start_ms and end_ms divided by 1000. Each sheet holds 30 tiles covering 60 seconds of the window; the last sheet is padded with black tiles. Tile k on sheet n (both counted from 0, left to right and top to bottom) is at START + 60 * n + 2 * k seconds. A condition lasts for many seconds, so one frame every 2 seconds is enough to see where it starts and stops inside the window. Look at every sheet of a window: windows can be much longer than the stretch they point at.

Whitecaps, drops on the lens and a distant rain shower are only a few pixels on a tile. When the label depends on them, extract a full size frame at that tile's second and look at it:

ffmpeg -v error -y -ss SECONDS -i FILE -frames:v 1 -q:v 3 conditions-BATCH_NAME/sheets/FIELD/rRANK_STEM_SECMS_full.png

SECONDS is the time of the tile, SECMS the same time in milliseconds.

Label rules (defaults; the team's definitions from Inputs win):

1. fog_low_visibility looks right when the horizon and the far shore are hidden and nearby ships or structures fade into grey. It looks wrong for an overcast sky with a sharp shoreline, rain on a window, and haze where the far shore is still a clear line.
2. rough_sea looks right when whitecaps or breaking crests cover the open sea surface out toward the horizon. It looks wrong for the vessel's own wake or bow wash, spray on the lens, and small chop with no whitecaps. Moderate seas sit on the line; the full frame decides more often than the tile.
3. low_sun_glare looks right when a low sun is in or at the edge of the frame and a bright glint band lies on the water. It looks wrong for a warm evening sky reflected on calm water with no sun in view.
4. rain_on_lens looks right when drops or streaks sit on the lens or the window the camera looks through, or a rain curtain is clearly visible. It looks wrong for fog with no drops.
5. dusk_or_dark looks right when sky and water are dark (late twilight or night). It looks wrong for daytime fog that is dim and blue, and for dawn with deck lights on but a bright horizon.
6. can't tell when the full frame still does not settle it. The reviewer looks at the source.

Write a note of what the frames show for every candidate, such as "bridge tower a grey shadow in fog, no horizon" or "lens covered in drops from 00:52; before that a grey squall". Never copy the reason into the note. Record each candidate key and its label in state.json.

Worked example from our test run

The test corpus was public footage published under a Creative Commons license: 12 videos (18.76 minutes) cut at fixed offsets from bow, deck and cockpit cameras on sailboats, cruise ships, a catamaran and a kayak, with the audio removed and the files renamed so the names said nothing about the weather. Fog, rough sea, low sun glare, rain on the lens and dark stretches were marked by hand on contact sheets before any search, together with look alikes (a ship's wake in fog, hull wash at night, golden hour on calm water, an overcast squall with a clear shoreline) and stretches too borderline to count either way. Most clips were handheld or deck cameras that turn around, unlike a fixed forward camera.

1. fog_low_visibility, first wording ("fog or mist over the water ... visibility is low"): 7 candidates, 5 real and 2 false, 0 of the fog stretches marked by hand missed. The false ones were an overcast squall with the shore sharp in every tile and rain drops on a cabin window. A second run of the same wording returned 9 candidates, 6 real and 3 false.
2. fog_low_visibility, the wording in the table, after 1 rewording on the same corpus: 2 candidates, 2 real and 0 false, and 1 of the 3 fog stretches marked by hand was missed (dense fog around a cruise ship's deck and wake). Run again, the same wording returned 5 candidates, 5 real, 0 false and 0 missed.
3. rough_sea, first wording (large waves, spray over the bow, the camera pitching and rolling): 5 candidates, 2 real and 3 false. The false ones were the ship's own wake foam in fog, small chop in fog and a lens so wet the sea could not be seen. A second run returned 5 candidates, 3 real and 2 false, both the ship's wake again.
4. rough_sea, the wording in the table, after 1 rewording on the same corpus: 2 candidates, 2 real and 0 false, and the same files in a second run. The corpus had only 1 stretch marked as rough by hand, and it was found, so this says little about recall. One of the real candidates had been marked borderline from the small tiles; the full frame showed whitecaps out to the horizon.
5. low_sun_glare: 4 candidates, 3 real and 1 false, 0 missed. The false one was calm water at golden hour with the sun out of frame.
6. rain_on_lens: 6 candidates, 2 real, 3 false and 1 unjudged, 0 missed. Two of the false ones were fog described as "a curtain of rain"; the unjudged one was a dark cloud base that may have been a distant shower. One real window covered the whole clip while the lens was wet only in its last third.
7. dusk_or_dark: 3 candidates, 1 real and 2 false. The real one was night with ship lights; the false ones were daytime fog and dawn with deck lights on.

Done when every candidate in results/ has at least one contact sheet in sheets/ and a label in state.json.

Step 6: Write one CSV per condition and hand it over

Goal: a review list per condition that a data engineer can work through without opening Vivu.

Show the user one sample row and the field mapping, and wait for an OK before writing the rest:

source_file,rank,start_mmss,end_mmss,search_reason,contact_sheets,claude_label,claude_note,reviewer_verdict
RUN_07.mp4,3,00:00,01:10,"Water drops, splashes, and streaks accumulate directly on the camera lens during a rainstorm on a boat.",sheets/rain_on_lens/r03_RUN_07_0_01.png,looks right,"lens covered in drops and smears from about 00:52; before that a grey squall with only a stray drop, so the window is much wider than the wet stretch",

The row above is adapted from our test run (file name shortened); the user's rows carry their own file names.

claude_label and claude_note are Claude's reading of the frames, not a measurement. Borderline cases (moderate seas with some whitecaps, thin haze, dim dawn) depend on the team's definition, so the reviewer goes through every row, the looks right ones included, before anything enters the eval set. When the eval set needs tight edges, the reviewer sets them from the sheets; Vivu's windows often start early or run past the stretch.

Write FIELD.csv for every condition with one row per result, sorted by rank, including the rows labeled looks wrong: a reviewer may disagree with a label, and the rejected rows show what the search confuses. The CSV uses the user's own file names and times, never a Vivu result page link, so it stays usable after the link expires.

Then tell the user, per condition, how many candidates came back, how many Claude labeled looks right, looks wrong and can't tell, and whether the list was cut off at maximum_results. Say plainly that an empty or short list does not mean the batch has no such stretch, and that thin fog, moderate seas and a few drops on the lens are the easiest to miss or mislabel. Do not add recall or precision figures: the hand marks needed to measure them do not exist for the user's batch.

The CSVs stay in the working folder. Passing them on is the user's step. If the user asks Claude to upload or post them (a shared drive, a tracker, a chat), name the destination, show the rendered first rows and wait for a yes before each write.

In our test run the CSVs were written from the dry run's results and sheets, 17 rows across the five conditions; showing the sample row to a user and handing the CSVs over were not exercised in our test run.

Done when every condition has a CSV whose row count equals the number of results in results/FIELD.json, the user approved the sample row and the mapping, and state.json marks the batch done.

Compliance

1. Use footage the team owns or has the rights to analyze. For public vessel videos used as extra material, check the license and the platform's terms first and keep them for internal analysis only. Have the user confirm this before Step 3.
2. Crew, passengers and people on other boats or on shore did not agree to be filmed for a dataset. The skill sorts stretches by sea and light condition only; it does no face, hull number or identity recognition and never searches for people or particular vessels. Footage with children on board (ferries, passenger decks) needs the team's consent rules for minors, or leave those runs out. Follow the team's data use rules (some teams blur faces before any upload), and confirm before Step 3.
3. The skill makes no safety or navigation judgment, does not say why a run went wrong, and does nothing on the vessel or in real time. The CSVs are review lists for people, not annotation files.
4. The videos stay in the user's Vivu project until the user deletes them. Create the project as private; the default is visible to the whole Vivu organization. Delete videos or the project only when the user asks, and confirm first.
5. The skill writes only local files. Any upload or post of the CSVs goes through the approval in Step 6.

Known failure modes