People Detection, Tracking, and Zone Analytics: Technical Description

TL;DR

The task is detecting people in a video stream, assigning unique IDs (tracking), and behavior analytics: line crossing, zone dwell time, counting.

YOLOv26 for detection + ByteTrack/BoT-SORT for tracking. ONNX/TensorRT for realtime GPU inference.

Line Crossing — detecting when a person crosses a virtual line with direction (entry/exit).

Zone Dwell Time — precise measurement of how long each person stays in a zone (useful for retail: checkout time, queue time, shelf time).

The system logs all events to CSV/DB and sends realtime alerts via webhook/Telegram on violations.

Problem statement

The system must process a video stream from IP cameras in realtime, detect people, assign unique IDs, and produce structured analytics: who entered, who exited, how long they spent in a zone, and whether perimeter violations occurred.

Key scenario: a retail store: counting visitors in/out, checkout time, queue analytics, density heatmaps — all automatically from the video stream.

Architecture Pipeline

Camera (RTSP)
    ↓
Frame Reader (OpenCV / ffmpeg)
    ↓
YOLOv26 Inference (ONNX / TensorRT)
    ↓
Tracking (ByteTrack / BoT-SORT)
    ↓
Zone Engine (line crossing + dwell time)
    ↓
Event Logger (CSV / DB / webhook / Telegram)
    ↓
Dashboard / API

Each pipeline stage is independent: swap the model, switch the tracker, configure zone analytics via config — without code changes.

YOLOv26: people detection

YOLOv26 (Ultralytics) is the foundation. The model takes a frame (RGB, 640×640) and returns bounding boxes with confidence scores and class labels (person = class 0 in COCO).

Model choice depends on hardware and FPS requirements:

Inference: PyTorch for development and debugging. ONNX or TensorRT for production — 2–5x GPU speedup. For edge devices (NVIDIA Jetson), TensorRT is mandatory.

Filtering: confidence threshold (default 0.5), NMS IoU threshold (0.45), class filter (person only), minimum bbox size (removes false positives in the background).

Tracking: ByteTrack / BoT-SORT

Tracking turns a set of disconnected bounding boxes into a coherent movement history for each person. Without tracking, counting entries/exits or dwell time is impossible.

ByteTrack — associates bboxes across frames by IoU (Intersection over Union) and visual similarity. Each person gets a track_id that persists while they remain in frame.

BoT-SORT — improved alternative: handles camera motion and works better with partial occlusions. Recommended for dense scenes.

Tracking parameters:

1) track_buffer — how many frames to keep a lost track (default: 30). At 15 FPS, the track "survives" 2 seconds after disappearing.

2) match_thresh — IoU threshold for association (default: 0.8). Higher = stricter matching.

3) A new ID is assigned when a new object appears. Old tracks are removed after track_buffer expires.

Line Crossing: detecting line intersections

Line Crossing detects when the center of a person's bounding box crosses a virtual line. This is the basis for counting entries and exits.

How it works:

1) A virtual line is defined on the image by two points: (x1, y1) and (x2, y2). The line can be any orientation — horizontal, vertical, diagonal.

2) For each track on each frame, the position of the bbox center relative to the line is checked using the cross product. If the sign changes between two consecutive frames — a crossing occurred.

3) Direction is determined by the sign of the cross product: positive = crossing left-to-right (entry), negative = right-to-left (exit). Directions are configurable.

4) Each crossing is logged: track_id, direction (in/out), timestamp, crossing coordinate, bbox confidence.

def check_line_crossing(prev_pos, curr_pos, line_start, line_end):
    d1 = cross_product(prev_pos, line_start, line_end)
    d2 = cross_product(curr_pos, line_start, line_end)
    
    if d1 * d2 < 0:  # sign changed → crossing
        direction = 'in' if d2 > 0 else 'out'
        return True, direction
    return False, None

Multiple independent lines are supported on a single frame — for different entry/exit points, gates, and passages.

Zone Dwell Time: time spent in a zone

Zone Dwell Time measures the total time each person spends inside a defined area. A key metric for retail (checkout time, shelf time, queue time) and industrial (time in a hazardous zone).

How it's calculated:

1) A zone is defined as a polygon (array of points) or rectangle on the frame. One camera can have multiple zones.

2) On each frame, the system checks whether the bbox center is inside the polygon (point-in-polygon test).

3) If the track is in the zone — delta_time (1/FPS seconds) is added to the dwell_time accumulator for this track_id in this zone.

4) When the track leaves the zone or is lost — a "dwell complete" event is logged with track_id, zone_id, total_dwell_time.

if point_in_polygon(center, zone_polygon):
    dwell_tracker[track_id][zone_id] += 1.0 / fps
else:
    if track_id in dwell_tracker:
        total = dwell_tracker[track_id].pop(zone_id)
        log_event('dwell_complete', track_id, zone_id, total)

Practical applications:

Retail: checkout time — average dwell time at registers = average service time. Rising dwell time = not enough cashiers.

Retail: queue analytics — dwell time in queue zone + people count in zone = predicted wait time.

Industrial: dangerous zones — dwell_time exceeding a threshold triggers an alert. Person has been in the danger zone too long.

Warehouses: loading time — dwell time in the gate zone = vehicle loading/unloading time. Efficiency analytics.

Alerting and logging

The system generates structured events of five types:

Delivery channels: CSV (default) — for analytics and reports. PostgreSQL / SQLite — for production. Webhook (HTTP POST) — for ERP/helpdesk integration. Telegram Bot API — realtime alerts to responsible staff. MQTT — for IoT/edge integrations.

Tech stack

Common implementation mistakes

1) Counting entries/exits without tracking — each new bbox is a "new person", causing double-counting on occlusions.

2) Not tuning confidence threshold for lighting — too many false positives at night or false negatives with backlight.

3) Using a single line for bidirectional passages without direction — entries and exits merge into one number.

4) Ignoring track loss during occlusions — person behind a column loses their ID and gets a new one, dwell time resets.

5) Computing dwell time without FPS reference — variable FPS causes inaccurate accumulator values.

6) Not filtering small bboxes in the background — detections on the far plane create noise in statistics.

FAQ

Key Takeaways

1) YOLOv26 + ByteTrack = reliable combo for realtime people detection and tracking.

2) Line Crossing via cross product — simple and accurate method for entry/exit counting.

3) Zone Dwell Time — FPS-based accumulator tied to track_id and zone_id. Key metric for retail.

4) TensorRT is mandatory for production: 2–5x inference speedup without accuracy loss.

5) Proper filtering (confidence, min bbox, track_buffer) matters more than model choice.