Tags: #python #machinelearning #computervision #programming
I wanted to build something that sounds relatively simple: analyze border crossing cameras, count the vehicles waiting in line, and estimate how long drivers will have to wait.
The idea was to use YOLO11 to detect cars, trucks, and buses in publicly available camera images.
And it worked.
The model could recognize vehicles, draw bounding boxes, and count cars inside a defined traffic area.
But counting cars doesn't tell you how long someone will wait.
What I really wanted was something more interesting: identify a particular car in the queue, track its movement between camera snapshots, and estimate when it would reach the border checkpoint.
That's where things got complicated.
I explored this while building Na granici, a website that combines computer vision and historical observations to estimate waiting times at border crossings.
Here's what I learned.
The cameras weren't mine
This was the first important constraint.
I wasn't installing my own cameras or controlling a video stream.
The system uses camera images published on official websites covering border crossings.
That means I can't simply reposition a camera, change its angle, increase its resolution, or decide how frequently the source image updates.
Some cameras show long queues from useful angles.
Others show only a small section of the road.
Some images are affected by darkness, headlights, reflections, or poor weather.
And sometimes vehicles are so far away that they occupy only a handful of pixels.
In other words, I had to work with whatever images were available.
The challenge wasn't creating an ideal computer vision environment. It was extracting useful information from an environment I couldn't control.
Starting with YOLO11
For object detection, I used YOLO11 from Ultralytics.
YOLO (You Only Look Once) is a family of object detection models that can recognize and locate multiple objects within an image.
For each detected vehicle, it returns a class, confidence score, and bounding box.
I focused on four vehicle categories:
- Cars
- Motorcycles
- Buses
- Trucks
The regular analysis uses the smaller yolo11s.pt model. I also supported yolo11x.pt for more computationally demanding analyses and comparisons.
Here's a simplified example of the detection configuration:
from ultralytics import YOLO model = YOLO("yolo11s.pt") results = model.predict( source="border_camera.jpg", conf=0.10, iou=0.35, classes=[2, 3, 5, 7] ) for result in results: for box in result.boxes: vehicle_type = result.names[int(box.cls[0])] confidence = float(box.conf[0]) coordinates = box.xyxy[0].tolist() print(vehicle_type, confidence, coordinates) The class IDs correspond to vehicles in the pretrained COCO model.
The confidence threshold is intentionally relatively low.
Distant vehicles can be difficult to detect, and a higher threshold might exclude them.
But lowering the threshold introduces another problem: more false positives.
A road sign, reflection, or partially visible object might be incorrectly classified as a vehicle.
The solution wasn't simply changing the threshold.
I needed additional image processing and filtering.
Detecting vehicles isn't the same as detecting a queue
Imagine a border camera showing 40 vehicles.
Some are waiting to enter the country.
Others are travelling in the opposite direction.
A few might be parked near the checkpoint.
YOLO doesn't know which vehicles are relevant to the waiting-time calculation.
It simply detects objects.
To address this, I used a Region of Interest (ROI).
An ROI is a polygon defining the area of the image where vehicles should be counted.
Using OpenCV, we can determine whether a detected vehicle belongs to the configured region.
import cv2 import numpy as np roi = np.array([ [100, 200], [900, 200], [850, 700], [150, 700] ], dtype=np.int32) def inside_roi(x, y): return cv2.pointPolygonTest( roi, (float(x), float(y)), False ) >= 0 This is a simplified point-in-polygon example; a full implementation must also define how a vehicle's bounding box is associated with the ROI.
The concept is similar to Ultralytics Region Counting.
The difficulty is that every camera has a different perspective.
A polygon that works perfectly for one border crossing might be completely useless for another.
And since I don't control the cameras, I must adapt the analysis to their existing geometry.
The experiment I really wanted: tracking one car
Counting vehicles was useful, but I wanted to go further.
Consider a border crossing with a visible queue.
At 12:00, a particular car is near the back.
At 12:01, it has moved forward.
At 12:02, it moves again.
If I could reliably recognize the same car across multiple observations, I could estimate its progression through the queue.
Eventually, I might be able to estimate when that vehicle would reach the checkpoint.
That would provide a more meaningful signal than simply counting cars.
But how do you recognize the same vehicle across different images?
YOLO doesn't solve vehicle identity
YOLO can detect a car in an image.
But when the next image arrives, it doesn't automatically know whether a detected car is the same one.
There are tracking approaches such as ByteTrack and BoT-SORT, supported by Ultralytics.
These are useful for associating detections across consecutive video frames.
However, my situation was different.
I wasn't processing continuous video.
I was working with periodically retrieved camera images.
Between two snapshots, vehicles might move significantly, become obscured, or disappear from the visible region.
A tracker designed for sequential video frames can't simply be assumed to work equally well with sparse snapshots.
Experimenting with feature matching
I explored classical computer vision techniques to investigate movement between images.
These included:
- SIFT feature extraction.
- Descriptor matching.
- RANSAC geometric verification.
- Colour histogram comparisons.
The OpenCV feature matching documentation describes how distinctive features can be matched between images and how RANSAC can reject geometrically inconsistent matches.
These methods can provide evidence that particular visual features moved between observations.
But they don't guarantee persistent vehicle identity.
Consider five nearly identical white cars.
One moves forward.
Another disappears behind a truck.
A third becomes visible.
Which car is which?
Even with additional visual features, the answer can be uncertain.
That uncertainty becomes more significant as the time between snapshots increases.
Why I wanted to analyze every camera once per minute
For tracking individual cars, I wanted much more frequent observations.
Ideally, the system would retrieve a fresh image and analyze each camera approximately once per minute.
That would give the movement analysis more opportunities to associate vehicles between observations.
Of course, this would only help if the official camera source actually provided a new image that frequently.
Fetching the same unchanged image repeatedly wouldn't improve tracking.
But there was another problem.
I had approximately 60 cameras to analyze, and my server couldn't process all of them every minute.
Each analysis involves more than downloading an image.
The processing pipeline can include:
- Retrieving the image from the camera source.
- Decoding and preparing it.
- Applying image processing and configured crops.
- Running YOLO11 inference.
- Filtering detections by region.
- Comparing results with previous observations.
- Updating queue statistics and estimates.
- Storing the results.
Running this pipeline for 60 cameras every minute would require completing approximately one full camera analysis per second on average.
And that's before accounting for temporary failures, additional processing, or more expensive model comparisons.
The server simply didn't have enough computational capacity for the frequency I wanted.
Smaller models help, but don't solve everything
Using yolo11s.pt instead of yolo11x.pt reduces computational requirements.
But increasing detection frequency also increases the cost of image processing, movement analysis, and storage operations.
There are several possible optimizations:
- Prioritize cameras with visible queues.
- Skip expensive inference when images haven't changed.
- Analyze selected image regions rather than unnecessary background areas.
- Use lightweight models for routine detection.
- Perform more expensive analysis only when needed.
- Process cameras asynchronously with a controlled worker queue.
These approaches could reduce the computational burden.
But there is still a tradeoff.
Analyzing fewer snapshots saves resources while making reliable vehicle association more difficult.
And even a much faster server couldn't force an official camera to provide more frequent images.
From vehicle counts to waiting times
Because reliable individual-car tracking wasn't available across all cameras, the waiting-time calculation needed another approach.
The simplest relationship is:
Estimated waiting time = Vehicles ahead / Estimated processing rate
For example, if a queue contains 30 cars and the checkpoint processes two cars per minute, those 30 cars represent approximately 15 minutes of processing.
But the processing rate isn't known from a single image.
It has to be estimated.
The application can investigate evidence such as:
- Changes in detected vehicle counts.
- Supported movement between snapshots.
- Historical observations.
- Queue clearing events.
- Previous estimated conditions.
The problem is that a decrease in detected cars doesn't necessarily mean those cars crossed the border.
They might have become hidden.
The detector might have missed them.
Or they might have left the camera's field of view.
That's why the system must distinguish between direct observations and assumptions.
A waiting-time estimate based on weak evidence shouldn't be presented with the same confidence as one supported by consistent observations.
Historical patterns can help
I also explored statistical forecasting based on previous observations.
The idea was to estimate future conditions using historical records near the requested time.
For example, traffic patterns on a Friday afternoon might differ from those on a Tuesday morning.
The forecasting process separates weekday and weekend patterns, selects relevant historical observations, and calculates daily medians.
When enough qualifying observations exist, historical information can be combined with the current estimate.
This is a separate problem from object detection.
YOLO11 tells us which vehicles appear in an image.
The statistical model helps estimate what conditions might look like later.
Neither method independently guarantees the actual waiting time.
What I learned about real-world computer vision
The biggest lesson was that detecting objects is often the easiest part of building a useful computer vision application.
YOLO11 can identify vehicles in images remarkably well under suitable conditions.
But the real-world application has to deal with much more:
Limited camera access. I couldn't control the official cameras, their positions, or their image updates.
Unreliable visual conditions. Darkness, glare, compression, and distant vehicles reduced detection reliability.
Ambiguous movement. Two similar vehicles appearing in different snapshots don't necessarily represent the same vehicle.
Computational limits. Processing approximately 60 cameras every minute was beyond what my server could sustain.
Uncertain waiting times. Counting vehicles doesn't reveal how quickly a border checkpoint is processing traffic.
And perhaps most importantly, improving one component doesn't necessarily improve the final result.
A more accurate detector won't solve the absence of fresh images.
A faster server won't solve poor camera visibility.
And a sophisticated tracking algorithm won't guarantee vehicle identity when the observations are too far apart.
What would I try next?
If I continued working on individual-vehicle tracking, I would start by selecting a small number of cameras with the most suitable viewing angles.
I would first measure how frequently their source images actually change.
Then I would test vehicle association on those cameras rather than attempting to track cars across all 60.
A practical approach might combine object detection, visual appearance features, geometric constraints, and queue-position changes.
I would also compare the results against manually verified vehicle movements.
Only after establishing that the method works reliably on a smaller dataset would it make sense to expand the analysis.
And if exact vehicle tracking proves too unreliable, estimating aggregate queue movement might be a better use of the available images.
Because ultimately, the goal isn't to track cars for the sake of tracking them.
It's to provide useful waiting-time information.
Final thoughts
I started with a simple question:
Can YOLO count the vehicles waiting at a border crossing?
The answer was yes, with limitations.
But the more interesting question became:
Can I recognize a particular vehicle, follow its progress through the queue, and predict when it will reach the checkpoint?
That's a much harder problem.
It combines object detection, feature matching, temporal analysis, computational optimization, and statistical forecasting.
And when you're using cameras you don't control, every one of those problems becomes more complicated.
The neural network was only one part of the system. The real challenge was turning imperfect, occasional observations into information people could actually use.
Have you worked with object tracking from periodic snapshots rather than continuous video? I'd be interested in how you handled object identity, especially with similar-looking vehicles and limited processing resources.
