Street condition work list from survey drives

---
name: street-condition-work-list
description: "Turn survey drive camera footage into a street condition work list CSV with Vivu: dumped bulky items, brush over the path, graffiti, downed signs. Use after a route drive, before crew checks."
---

Street condition work list from survey drives with Vivu

This skill turns the dashcam or street camera footage from a public works survey drive into a work list for the crew. The user gives Claude the folder of drive videos, the drive log (GPS track or route with start times) and, if they have one, a 311 complaint export. Claude cuts only the stretches of route that the complaints and the route point to, prices them against the user's Vivu plan, uploads them to a private Vivu project, runs one precise search per condition (dumped bulky items, brush or branches over the sidewalk or a sign, graffiti, a knocked over or leaning sign), cuts a contact sheet at half second steps for every window Vivu returns, labels each window from its frames, works out the approximate location from the drive log, and writes work_list.csv with an empty crew_verified column. The roads manager checks the list and enters the real items into the work order system.

The value is in what only the picture holds. A resident's service request gives a rough address and the caller's words; the drive log says which streets the car covered and when. Neither says whether the thing at the curb is a mattress or a pile of branches, how much brush hangs over the sidewalk, or which sign is leaning. Dashcam audio is engine and road noise, and the file names are timestamps, so a text tool has nothing to search. Vivu's search is only the first pass. In our test run the first wording for dumped items returned a tipped over trash cart that Vivu's reason text called a couch, and the sign search returned mostly upright signs with a reason that said each one was leaning. So every row is labeled from its frames, rejected candidates stay in the CSV, and a short list never means the street is clean.

When to use

Use when a roads or public works manager asks to "go through yesterday's survey drive and list the dumping and overgrowth", "check the route for downed signs", "find the graffiti on the 311 streets from this week's drive", or "make a work list from the patrol camera footage".

Not for these:

1. A one off question about one clip ("is the sign at this corner still down?"): search Vivu directly and look at the frames.
2. Potholes, cracks, faded paint and other small pavement defects. They are small and fast in dashcam footage and this skill does not search for them; use a pavement rating tool or a walking inspection.
3. Safety ratings or priorities. The skill lists what the camera saw; engineers decide severity and order.
4. People, vehicles or license plates. The skill never searches for or lists them.
5. A whole day of driving at once. Pick the stretches the complaints and the route point to; one run is sized for about 30 minutes of footage.
6. Night footage: untested. The skill can run, but the crew checks every row.

Working principles

1. Report measured numbers, not estimates. When a number is an estimate, say so.
2. Nothing is verified until it has been checked against the source video. Vivu's results are candidates, Claude's labels are a first pass, and the crew's check is the decision.
3. Stop and tell the user when a required capability or tool is missing. Do not guess around it.
4. Ask the user before anything that is expensive to redo (the stretches to index and their index minutes, before upload) and before writing the full CSV (one sample row first). The skill posts and sends nothing; if the user asks Claude to put the list somewhere, ask again before each write.
5. Labels and categories come from frames, never from Vivu's reason text, which describes the condition it was asked for even when the frame shows something else.
6. The list keeps every candidate and carries no recall figure. A search misses some items and caps how many come back, so a short list does not prove the street is clean.

What you need before starting

Check each item at the start of the run and tell the user plainly what is missing before doing anything else.

The 311 export is optional. Without it, the user picks the stretches by route and date.

This skill needs Claude Code on the user's computer (the terminal or the Code tab of Claude Desktop), because it reads local files and runs ffmpeg. Claude on the web and cloud sessions cannot reach the files. It downloads nothing, so no residential IP is needed. Nothing recurs, so no scheduler is involved; run it once per batch of drives.

Inputs to collect

Ask for anything missing, most important first.

1. DRIVE_DIR: the folder that holds the drive videos. Required.
2. The drive log file. Required for locations; without it the Location column says NOT VERIFIED.
3. The stretches to check: 311 complaint addresses and dates, or route segments. Default: every stretch the user names, cut 2 to 4 minutes each.
4. BATCH_NAME: a short name such as a district and a week. It names the working folder and the Vivu project. Default: the name of DRIVE_DIR.
5. The conditions and what counts for each. Default: the four in Step 4 with the label rules in Step 5. The department's own definitions win; write them into config.json before labeling.
6. Who checks the list. Default: the user.

Files and state

Keep everything in one working folder next to DRIVE_DIR:

streets-BATCH_NAME/
  config.json      DRIVE_DIR, drive log path, conditions with query and label rules, Vivu project id
  stretches.csv    clip_file, source_file, source_start_s, source_end_s, clock_start, duration_s, listed_name, video_id
  clips/           the cut stretches that get uploaded
  results/         search results without the result page link, one JSON per condition
  sheets/          one folder per condition with contact sheets and zoomed frames
  work_list.csv    every candidate, Claude's label, location, empty crew_verified
  state.json       steps done, clips uploaded, conditions searched with job ids, candidate keys labeled

The commands in Steps 2 to 5 run from the folder that holds both DRIVE_DIR and streets-BATCH_NAME/, so the relative paths in them resolve.

A rerun reads state.json first and skips finished steps. A clip already marked uploaded is never uploaded again, a condition with a saved result is not searched again, and a candidate key (video_id, start_ms and condition) that already has a label is not labeled again. The next batch gets its own folder and its own Vivu project, so its searches do not return last week's candidates.

Step 1: Check the Vivu connector and the setup

Goal: confirm every requirement before spending anything.

1. Call vivu_get_account. If the tool does not exist, tell the user to add the Vivu connector in Claude (https://mcp.vivu.ai/mcp) and stop. If the account does not show can_create_projects: true, or a later write call fails with "has not granted vivu.write", ask the user to reconnect Vivu and allow write access, then stop until they have.
2. Run ffmpeg -version, ffprobe -version and python3 --version.
3. List DRIVE_DIR, run the ffprobe length command on one file, and open the drive log to find the start time of that file.

Done when vivu_get_account shows can_create_projects: true, the three tools print their versions, and one file in DRIVE_DIR has both a length and a start time.

Step 2: Pick the stretches and cut them

Goal: short clips that cover only the streets that need a look, so the index minutes go where the complaints are.

1. Turn each 311 complaint or route segment into a time range in a source file: find when the drive log passes the address, and take about a minute before and after. Merge ranges that overlap. Show the list (source file, start, end, why) and let the user add or drop stretches.
2. Cut each stretch with a re-encode, so the cut lands on the requested second rather than the nearest keyframe:

mkdir -p streets-BATCH_NAME/clips
ffmpeg -v error -ss START -to END -i DRIVE_DIR/SOURCE_FILE -an -c:v libx264 -crf 22 streets-BATCH_NAME/clips/CLIP_NAME.mp4

START and END are seconds in the source file, SOURCE_FILE is the drive video, and CLIP_NAME is a short name with no spaces such as route3_0915_elm (route, clock time, street), so it survives Vivu's file name rules. -an drops the cabin audio, which carries nothing useful and may record conversations.

3. Write one row per clip to stretches.csv with source_start_s and the clock time the clip starts (source file start time from the drive log plus START). Step 7 needs that clock time.

Picking stretches from complaint addresses was not exercised in our test run: public footage comes with no complaints or drive log, so the test clips were cut at fixed offsets.

Done when every clip the user approved exists in clips/ and has a row with clock_start in stretches.csv.

Step 3: Measure the stretches and price them

Goal: an indexed set the user's plan can pay for, approved before upload.

1. Run ffprobe -v error -show_entries format=duration -of csv=p=0 streets-BATCH_NAME/clips/CLIP_NAME.mp4 for every clip and record duration_s. Add listed_name: the file name with each space, bracket, plus sign and percent sign replaced by an underscore, because that is how Vivu will list it.
2. Call vivu_get_usage and show one table:

Plan facts: Free is $0 a month with 20 index minutes a month and 50 search credits a month; Premium is $30 a month with 180 index minutes and 500 search credits. A precise search uses 5 credits in total. Ten stretches of three minutes each are 30 index minutes (estimate), more than Free covers in a month and well inside Premium; Free fits about six such stretches.

3. If the batch does not fit, offer levers in this order: shorter stretches around each complaint, fewer stretches (the oldest or repeat complaints first), split the batch across two runs or two months, and only then a larger plan. Never drop a stretch the user asked for without saying which one.
4. Confirm the Compliance items with the user.

The account used in our test run returns no allowance figures, so the comparison with a real plan and the user's approval were not exercised in our test run.

Done when the user approves the clip list and its index minutes, and config.json records both.

Step 4: Upload and index

Goal: every clip indexed in a private Vivu project for this batch.

1. Call vivu_list_projects and reuse a project named "Streets BATCH_NAME" if one exists. Otherwise call vivu_create_project with that name and visibility "private". The default visibility is organization, which shows the project to everyone in the user's Vivu organization, and street footage with residents, homes and plates is sensitive.
2. Call vivu_open_upload_page with the project ID immediately before uploading. The link expires in 180 seconds and works once, so never post or store it. Give it to the user to open in their own browser and select the files in clips/, or open it in a browser tool that can attach local files. Claude in Chrome accepts at most 10 MB per upload call, and a three minute dashcam clip is usually larger, so clips normally go through the user's own browser or the Vivu web app. Never split or recompress a clip to fit.
3. Poll vivu_list_videos until every clip shows ready. Match each Vivu file name to stretches.csv by listed_name and record its video_id there and in state.json.

In our test run the 10 clips (25 minutes) were all ready 538 seconds after the upload started. They went through the same upload page with an automated browser, so opening the link in the user's own browser was not exercised in our test run.

Done when every clip in stretches.csv shows ready and has a video_id.

Step 5: Search each condition

Goal: a saved list of candidate windows for every condition.

The searches run side by side, one per condition, because each condition is its own group in the work list. Within a condition the chain is: the precise search finds candidate windows, the contact sheet shows what the camera passed across each window, Claude labels it, and the crew decides. Every search is precise because only precise returns a time window; fast returns whole files with an empty reason, and the clips are already a chosen subset. Dashcam audio is road noise and there is no on screen text about the street, so there is no second modality to switch to.

The bulky_dumped and vegetation_blocking wordings are the second ones we tried on the same footage, after 1 rewording on the same corpus each (see the worked example). They describe size and what the path looks like because the first wordings, which named the object, returned a tipped over cart, a single bag and street trees. Keep them unless the department's definition differs.

sign_down is a low precision list. In a wide angle dashcam frame, poles near the left and right edges look tilted even when they stand straight, and the search does not tell the two apart. Keep the class only if the crew is willing to work through many looks wrong rows; otherwise drop it and say so.

maximum_results is also the ceiling on how many candidates come back. 40 leaves room for about 30 minutes of residential streets with several items per stretch; a search that returns exactly 40 was cut off, and the batch holds more candidates than one search can return. Then tell the user, mark the condition cut off in state.json, and offer to split the batch into two smaller batches with their own projects (quote the index minutes again and wait for approval as in Step 3).

In our test run no search came close to 40, so a cut off search was not exercised in our test run.

Run each query with vivu_search_videos (project_id, query, mode, maximum_results). It returns a job ID. Call vivu_get_search_results until complete is true; each status call can wait up to 45 seconds, so a pending search is not a stalled one. Save each completed result as results/FIELD.json without its result_page_url field, and record the job_id in state.json. Show the Vivu result page link in the live reply only; it expires after four hours, so it never goes into a saved file or the CSV.

Done when every condition has a completed result saved in results/, and every condition whose search returned exactly its maximum_results is marked cut off in state.json.

Step 6: Make contact sheets and label every candidate

Goal: every candidate labeled looks right, looks wrong or can't tell from what its frames show.

For each result in results/FIELD.json, cut contact sheets of its window from the local clip at two frames a second:

mkdir -p streets-BATCH_NAME/sheets/FIELD
ffmpeg -v error -y -ss START -to END -i streets-BATCH_NAME/clips/CLIP_NAME.mp4 -vf "fps=2,scale=320:-2,tile=6x4" streets-BATCH_NAME/sheets/FIELD/rRANK_CLIP_STARTMS_%d.png

FIELD is the condition, RANK the result_number written with two digits (r01, r09), STARTMS the result's start_ms, and START and END the result's start_ms and end_ms divided by 1000. Each sheet holds 24 tiles covering 12 seconds of the window; tile k on sheet n (both counted from 0, left to right and top to bottom; ffmpeg numbers the files from 1, so the file ending _1 is sheet 0) is at START + 12 * n + k / 2 seconds. Look at every sheet of a window. For windows longer than 30 seconds, use fps=1 (tile k on sheet n is at START + 24 * n + k seconds).

Tags, bags and sign posts near the frame edge are a few pixels on a tile. When a tile shows something small near the curb or on a wall, or the sheet looks empty but the reason names an object, extract a zoomed frame at that tile's second:

ffmpeg -v error -y -ss SECONDS -i streets-BATCH_NAME/clips/CLIP_NAME.mp4 -frames:v 1 -vf "crop=iw/2:ih/2:X:Y,scale=iw*3:-2" streets-BATCH_NAME/sheets/FIELD/rRANK_CLIP_SECMS_zoom.png

SECONDS is the time of the tile, SECMS the same time in milliseconds, X is 0 for the left half or iw/2 for the right half, and Y is 0 for the top half, ih/4 for the middle band or ih/2 for the bottom half.

Label rules (defaults; the department's definitions from Inputs win):

1. bulky_dumped looks right for a heap of bags, boards, debris or furniture, or a mattress or couch, on the curb strip, sidewalk or alley edge. It looks wrong for trash carts (standing or tipped), one bag, yard furniture in use and thin scattered litter.
2. vegetation_blocking looks right when brush, weeds or branches visibly cover part of the sidewalk, alley pavement or road edge, or hide part of a sign. It looks wrong for street trees beside a clear path and for a sign that is fully visible.
3. graffiti looks right for spray painted tags or lettering on a wall, fence, utility box or boarded window. It looks wrong for murals, painted store signs, truck livery and plain paint patches.
4. sign_down looks right only when the post is clearly off vertical near the centre of the frame, or the sign is on the ground. A post that looks tilted only near the left or right edge is lens distortion and looks wrong unless a centred frame shows the same lean.
5. can't tell when the zoomed frame still does not settle it. The crew looks on site.

Write a note of what the frames show for every candidate, such as "pile of boards and black bags at the right curb by a tree, 02:21 to 02:25" or "upright sign; tilt only at the frame edge". Never copy the reason into the note, and never record people or plates. Record each candidate key and its label in state.json.

Worked example from our test run

The test corpus was public Creative Commons city drive footage from a windshield wide angle camera, used in place of a department's survey drives: 10 clips of 150 seconds (25 minutes) cut at fixed offsets from five drives through residential streets, back alleys and one arterial road, with the audio removed and the files renamed so the names said nothing about content. Only a low resolution copy could be downloaded, which is softer than a real dashcam export. Items were marked by hand on contact sheets with a frame for every second before any search, together with look alikes (murals, a graffiti style truck trailer, trash carts, tidy blocks with street trees). Results are below as returned, real, false, and missed against the hand marks.

1. bulky_dumped, first wording (named the objects): 12 returned, 8 real and 4 false, and 1 missed. The false ones were a tipped over trash cart the reason called a couch, one bag on a lawn strip, yard chairs in use and thin litter along a vacant lot.
2. bulky_dumped, the wording in the table, after 1 rewording on the same corpus: 5 returned, 5 real and 0 false, and 2 missed. One miss was a bedding pile the first wording had found; the other was a heap both wordings missed.
3. vegetation_blocking, first wording: 6 returned, 3 real and 3 false, and 0 missed. The false ones were street trees beside a clear sidewalk and a stop sign the reason said was partly covered.
4. vegetation_blocking, the wording in the table, after 1 rewording on the same corpus: 3 returned, 3 real and 0 false, and 0 missed. The third window is borderline (weeds at a sidewalk edge) and the reason overstated it.
5. graffiti: 6 returned, 4 real and 2 false, and 0 missed. Murals and the truck livery were not returned; the false ones were a paint patch and a store wall.
6. sign_down: 15 returned, 4 real and 11 false, and 0 missed. Most false ones were upright posts near the frame edge.
7. A control search for a flooded street, which the footage did not contain, returned 0 results.
8. The work list for the final wordings held 29 rows.

Done when every candidate in results/ has at least one contact sheet in sheets/ and a label in state.json.

Step 7: Work out the location from the drive log

Goal: an approximate location for every candidate, so the crew can find it.

The clock time of a moment is the clip's clock_start plus start_ms / 1000 seconds. With a GPS track (a CSV with an ISO timestamp column and lat and lon columns), save this helper as locate.py and run it:

import csv, sys, datetime as dt
track_path, stretches_path, results_path, field = sys.argv[1:5]
pts = [(dt.datetime.fromisoformat(r["timestamp"]), r["lat"], r["lon"]) for r in csv.DictReader(open(track_path))]
starts = {r["listed_name"]: dt.datetime.fromisoformat(r["clock_start"]) for r in csv.DictReader(open(stretches_path))}
import json
for res in json.load(open(results_path))["results"]:
    t = starts[res["file_name"]] + dt.timedelta(milliseconds=res["start_ms"])
    best = min(pts, key=lambda p: abs((p[0] - t).total_seconds()))
    gap = abs((best[0] - t).total_seconds())
    print(field, res["result_number"], res["file_name"], t.isoformat(), best[1], best[2], f"{gap:.0f}s")

python3 locate.py TRACK.csv streets-BATCH_NAME/stretches.csv streets-BATCH_NAME/results/FIELD.json FIELD

TRACK.csv is the drive log; clock_start must use the same time zone as its timestamps. The last column is the gap between the moment and the nearest track point; above about 10 seconds the location is UNCERTAIN, because the car may have moved a block. Write the point as approx_location (and the nearest cross street if the user's map tool gives one). With only a paper route, write the street and the clock time instead and mark it UNCERTAIN. A Vivu window is several seconds wide, so the location is the start of the window, not the exact spot; the crew finds the item within that block.

Step 7 was not exercised in our test run: public footage has no drive log, so every row there says NOT VERIFIED.

Done when every candidate has an approx_location, UNCERTAIN, or NOT VERIFIED with the reason.

Step 8: Write the work list and hand it over

Goal: one CSV the roads manager can check and copy into the work order system without opening Vivu.

Show the user one sample row and the field mapping, and wait for an OK before writing the rest:

clip_file,rank,start_mmss,end_mmss,category,approx_location,search_reason,contact_sheets,claude_label,claude_note,crew_verified
SD_08.mp4,3,01:40,01:49,bulky_dumped,NOT VERIFIED (no drive log in the test),"A large pile of broken boards and bulky debris is visible on the ground beside the road next to the trash bins.",sheets/bulky_dumped/r03_SD_08_100000_1.png,looks right,"pile of broken boards and debris on the lawn strip in front of a house at 01:43 to 01:46",

The row above is from our test run, with contact_sheets written in the Step 6 naming (the test run's own sheet files used different names); the user's rows carry their own clip names and locations.

claude_label and claude_note are Claude's reading of a dashcam frame, not a site inspection. The crew checks every row, the looks right ones included, before a work order is opened. The skill never sets severity or priority.

Write work_list.csv with one row per result, grouped by condition and sorted by rank, including the rows labeled looks wrong: the manager may disagree with a label, and the rejected rows show what the search confuses. The CSV uses the user's own clip names, times and locations, never a Vivu result page link, so it stays usable after the link expires.

Then tell the user, per condition, how many candidates came back, how many Claude labeled looks right, looks wrong and can't tell, and whether the list was cut off at maximum_results. Say plainly that a short list does not mean the streets are clean, that small items at the frame edge are the easiest to miss, and that sign rows are mostly looks wrong. Do not add recall or precision figures: the hand marks needed to measure them do not exist for the user's batch.

The CSV stays in the working folder. Entering items into the work order system is the user's step. If the user asks Claude to upload or post the list (a shared drive, a tracker, a chat), name the destination, show the rendered first rows and wait for a yes before each write.

In our test run the CSV was written from the dry run's results and sheets; showing the sample row to a user and handing the list over were not exercised in our test run.

Done when work_list.csv has as many rows as the results in results/, the user approved the sample row and the mapping, and state.json marks the batch done.

Compliance

1. Use footage the department owns: its own survey car or fleet cameras. Public videos used for testing need a license that allows it, kept for internal analysis only. Have the user confirm this before Step 4.
2. Residents, passersby, homes and license plates appear in street footage. The skill lists street conditions only; it does no face, plate or identity recognition, never searches for people, and leaves people and plates out of every note. Follow the department's policy for people in frame (some blur faces before any upload), and confirm before Step 4. Do not use the skill to watch particular people or homes.
3. The skill reports what the camera passed in the public right of way. It makes no judgment about safety, severity, priority or whose responsibility an item is; engineers and the crew decide.
4. The clips stay in the user's Vivu project until the user deletes them. Create the project as private; the default is visible to the whole Vivu organization. Delete clips or the project only when the user asks, and confirm first.
5. The skill writes only local files. Any upload or post of the list goes through the approval in Step 8.

Known failure modes