Tools

tamserve in Practice: 4x Faster Multi-Camera Tracking

tamserve combines SAM 3 detection with lightweight EdgeTAM tracking to run four times faster than Meta's SAM 3 tracker on a single RTX 4090 while retaining 92% of its accuracy. Its architecture shares model instances across cameras and allows targets to be added without restarting the stream.

Font size

Introduction

Meta's native SAM 3 tracker runs at roughly 8 fps and may exhaust GPU memory on long videos. The open-source project tamserve combines infrequent detection with lightweight tracking, delivering a fourfold speedup on a single RTX 4090. Here's how it handles multiple video streams and whether its reported performance holds up in production.

The Core Idea in the Repository Demo

A common mistake in multi-camera video tracking is to assign both object recognition and motion tracking to a large model. That approach increases memory consumption and steadily reduces real-time performance.

The open-source tamserve takes a different approach: SAM 3 discovers objects, while EdgeTAM tracks them across frames. Each model handles the task it is best suited to.

According to the project's benchmarks, specified-target processing runs four times faster than Meta's SAM 3 tracker on a single RTX 4090 while retaining 92% of the tracker's accuracy.

The demo uses multiple live video streams. The user first enters:

forklift

The system finds and tracks two forklifts, continuously marking the targets without interrupting the workflow.

The user then enters:

person

The system adds people to the target list and continues running. There is no need to stop the task or reprocess the entire video.

Two details stand out:

  • New targets can be added to an active tracking session through text prompts.
  • Multiple cameras share the same model instances instead of loading separate copies for each stream.

As a result, tamserve is more than an offline tool that draws boxes around objects in a recorded video. It behaves more like a real-time video Agent whose target set can be modified while tracking continues.

Why Not Use SAM 3 Directly?

Meta's SAM 3 includes a tracker, but the native implementation processes one recorded video at a time at approximately 8 fps. Longer videos can exhaust memory, placing particular pressure on GPU memory.

For live cameras, this approach creates two additional problems.

First, 8 fps does not equal reliable real-time performance. When targets move quickly, become occluded, or leave the frame, a low frame rate increases response latency.

Second, SAM 3's memory usage becomes increasingly difficult to control as videos get longer. With multiple cameras, repeating the model instance for each stream compounds the problem.

This reflects a common engineering trade-off in computer vision: not every frame requires another round of expensive semantic detection.

tamserve's Engineering Trade-Offs

tamserve uses a hybrid architecture. SAM 3 detects objects every few frames, while lightweight EdgeTAM tracks motion between those detections.

The pipeline has two stages:

Every N frames: SAM 3 detects objects
Remaining frames: EdgeTAM extends target positions

SAM 3 understands what a forklift or a person is. EdgeTAM handles movement between adjacent frames. Together, they eliminate the need to rerun semantic inference on every frame.

Resource management is another key design choice. tamserve shares one instance of each model across all cameras.

With eight video streams, loading a separate copy of SAM 3 and EdgeTAM for each stream would scale GPU memory usage with the number of cameras. Sharing the instances keeps the models resident on the GPU, while each stream mainly manages its own input and tracking state.

This design also enables the system to continue after a new target is entered. Adding person only requires updating the target set, not rebuilding the entire multi-camera pipeline.

Results on a Single RTX 4090

The project's reported benchmark results are straightforward:

  • Hardware: One RTX 4090
  • Comparison: Meta SAM 3 tracker
  • Specified-target processing speed: 4x faster
  • Accuracy: Retains 92% of SAM 3 tracker's accuracy

If the SAM 3 tracker runs at approximately 8 fps, a simple proportional calculation yields about 32 fps. Actual end-to-end latency also depends on camera frame rate, detection interval, video decoding, and the number of targets, so 32 fps should not be treated as a guarantee across all deployments.

The engineering direction is sound. Separating expensive, infrequent detection from inexpensive, high-frequency tracking improves not only throughput but also peak memory use and scalability across multiple streams.

Original project:

Editorial Assessment

tamserve's value lies not in introducing a new model, but in decoupling detection from tracking. SAM 3 handles semantics, EdgeTAM handles temporal continuity, and the two run at different frequencies. This differs from the end-to-end vision systems that have traditionally been common: rather than asking one model to handle recognition, localization, and memory, the system divides computation according to the cost of each task.

This hybrid architecture appears better suited to warehouses, industrial campuses, and retail environments than running one model across every frame. As the number of cameras increases, sharing model instances and controlling peak GPU memory often matter more than a few percentage points of single-frame accuracy.

The current figures, however, come from the author's own tests. The definition of the 92% accuracy metric, target counts, occlusion conditions, and throughput methodology all need closer examination against the repository code. Production environments may also introduce backlighting, dust, and inconsistent camera frame rates. The sensible approach is to begin with a proof of concept, then stress-test it using the actual number of streams rather than promise a fixed latency.

Source

Source: https://github.com/aryamandarda/tamserve