Ensuring security in high risk areas such as an airport is an important but complex problem. Effectively tracking
personnel, containers, and machines is a crucial task. Moreover, security and safety require understanding the interaction
of persons and objects. Computer vision (CV) has been a classic tool; however, variable lighting, imaging, and random
occlusions present difficulties for real-time surveillance, resulting in erroneous object detection and trajectories.
Determining object ID via CV at any instance of time in a crowded area is computationally prohibitive, yet the
trajectories of personnel and objects should be known in real time. Radio Frequency Identification (RFID) can be used to
reliably identify target objects and can even locate targets at coarse spatial resolution, while CV provides fuzzy features
for target ID at finer resolution. Our research demonstrates benefits obtained when most objects are “cooperative” by
being RFID tagged. Fusion provides a method to simplify the correspondence problem in 3D space. A surveillance
system can query for unique object ID as well as tag ID information, such as target height, texture, shape and color,
which can greatly enhance scene analysis. We extend geometry-based tracking so that intermittent information on ID
and location can be used in determining a set of trajectories of N targets over T time steps. We show that partial-targetinformation
obtained through RFID can reduce computation time (by 99.9% in some cases) and also increase the
likelihood of producing correct trajectories. We conclude that real-time decision-making should be possible if the
surveillance system can integrate information effectively between the sensor level and activity understanding level.
Keywords: Tracking, stereo, RFID, trajectories, frames, fusion