Sentinel: following a vehicle across city cameras
My final year project. Many traffic camera feeds in one dashboard, plain English search over the footage, and a pipeline that links the same vehicle across cameras.
Context
Final year project, BS Artificial Intelligence, FAST-NUCES, 2026. I built the system. It runs on the public AI City CityFlowV2 dataset.
The problem
A city operator watching dozens of feeds cannot follow a car by eye, and cannot ask “where did the red pickup go?”. Two hard parts: handling many streams on one GPU, and deciding that a car on camera 2 is the same car that left camera 1.
What I built
- Ingestion and video wall. Up to 12 camera streams replayed over RTSP through MediaMTX, shown as a live grid in a React dashboard.
- Natural language search. Footage is cut into clips and embedded with Qwen3-VL-Embedding-2B, so a text query like “a red car” returns matching snapshots.
- Re-identification pipeline. YOLOv8m detects vehicles, ByteTrack follows them within a camera, a FastReID model fine tuned on CityFlow turns each track into a 2048 number fingerprint, and a matcher links tracks across cameras.
- Plumbing. Redis Streams carry tracking events durably. Video frames go through shared memory instead of Redis so the message bus stays light. Everything runs under Docker Compose.
Decisions
- Appearance alone is not enough, so the matcher uses more than looks. It also checks travel time between cameras and only allows one link per car per camera pair (the real matcher uses the Hungarian algorithm for that step).
- Precision over recall. A wrong match sends an operator after the wrong car; a missed match only loses a trail. Matches start as tentative and are confirmed later.
- Redis Streams over Kafka. Durable enough for one machine, far less to run.
- Interface built on labelled tracks. The multi camera dashboard view runs on the dataset’s ground truth tracks, which let me build and test the interface separately from the model.
Results
I measured the matching step on every vehicle in scene S01: 409 tracks across 5 cameras, which gives 716 true cross camera matches hidden among 66,087 wrong pairs. The embedding model is FastReID trained on a different dataset (VeRi-776), so it has never seen these cameras.
| Method | Threshold | Precision | Recall |
|---|---|---|---|
| Looks only | 0.70 | 17% | 37% |
| + travel time (60 s) | 0.70 | 29% | 37% |
| + one-to-one matching | 0.70 | 83% | 27% |
Same car pairs average 0.64 similarity, different car pairs 0.34. That gap looks big, but with 92 wrong pairs for every right one, the overlap is what decides precision. The interactive below uses these exact numbers.
What broke
- My first result did not survive a bigger test. Early on I measured 27 hand picked pairs and got 100% precision at a 0.71 threshold. On all 716 true pairs and 66,087 wrong ones, the same threshold gives 18% precision from looks alone. Small, clean samples hide the real difficulty, which is that wrong pairs vastly outnumber right ones.
- The model fine tuned on this scene is not a fair test. It was trained on the same cameras, so I report the model trained on a different dataset instead.
- Thresholds drifted between experiments (0.45, 0.55, 0.71) until I fixed one per evaluation.
- Search and live streaming compete for one GPU, so search pauses the streams.
Try it: when are two sightings the same car?
Every vehicle seen by one camera gets compared with every vehicle seen by the others. That is 716 true matches hidden among 66,087 wrong ones. Drag the threshold and switch on the tricks the matcher uses.
Real numbers: FastReID (R50-ibn, trained on VeRi-776, never shown this dataset) on all 409 vehicle tracks across the 5 cameras of CityFlowV2 scene S01. Travel time keeps pairs seen within 60 s of each other. One-to-one keeps only the mutual best match between each pair of cameras. Cars are drawn, not dataset footage.