acytoo

2D Kalman Filter Multiple Object Tracking

Some times we have following cases

  1. Where the variable of interest can only be measured indirectly.
  2. Measurements are available from various sensors but might be subject to noise.

We can use Kalman Filter for those cases, but remember, general Kalman Filter works well for objects/measurements changing at constant velocity or constant acceleration.

I will not repeat how Kalman Filter works, it's just math, there are already tons of documents, and I don't think I can write a better explanation, at least for now. So I put some documents/videos I found useful, please check them for detailed information.

https://en.wikipedia.org/wiki/Kalman_filter

https://www.kalmanfilter.net/default.aspx

In this post we have two tasks, 1: tracking single ball through occlusion; 2: tracking two balls crossing each other

Talk is cheap, show me the code!

#Task 1:

Tracking single object through occlusion.

Source video: download

Tracking one object is easy, even with occlusion.

  1. Initialize the tracker
  2. Loop the tracker algorithm
#Architecture:
kalman = KalmanFilter()
kalman.update(initial_position) # Not necessary, but can provide better result if you update the initial position

while frame:
  detection = detect(frame)
  
  kalman.predict()

  if detected: # if detected object, correct Kalman Filter
    kalman.update()
#Result:

The green box is the detector result, and red dot is the tracking/prediction result from Kalman Filter

#Task 2:

Tracking multiple objects, distinguish them, recognize them when they crossing each other.

Source video: download

Tracking multiple objects is difficult than tracking single object, we need to assign tracker to each object after detection, and update tracker correspondingly. If one object doesn't show up in a few frames, delete its tracker. If new object shows up, add one tracker. Usually we maintain a list of trackers with number one more than the number of objects detected.

To main the tracker list, we need a new struct, tracker

from dataclasses import dataclass

@dataclass
class Tracker:
  id: int
  centroid: tuple
  kalman: KalmanFilter()
  consecutive_invi_num = 0

id is the identifier to that object(tracker), centroid is the predicted position of the object, kalman is the KalmanFilter, consecutive_invi_num is the number of frames which the object is invisible, if one object is invisible for a long time, we delete his tracker.

How do we assign tracker to object? Let's say we have 2 trackers, and 2 objects, we assign the tracker to the object, such that the overall distance between tracker's centroid and object's detected centroid is smallest.

How to calculate distance is your choice, for simplicity we chose Euclidean distance here.

Example 1:

Distances object1 object 2
tracker 1 10 1
tracker 2 2 12

In this case, tracker 1 is for object 2 and tracker 2 is for object 1.

Example 2:

Distances object1 object 2
tracker 1 20 1
tracker 2 12 0

In this case, tracker 1 is for object 2 and tracker 2 is for object 1. Even the distance between tracker 2 and object 2 is 0, we still assign tracker 2 to object 1, because the overall distance is smaller.

Luckily, scipy already wrote this function for us:

row_ind, col_ind = linear_sum_assignment(costs)

For detailed information about this function and the algorithm behind it, please refer to scipy's document.

#Architecture:
trackers = []

while frame:

  detection = detect(frame)
  
  for tracker in trackers:
    tracker.centroid = tracker.kalman.predict()

  ########### Tracker management ###############

  # calculate cost for assigning detections to tracks
  costs = []
  for tracker in trackers:
    current_costs = []
    for c in detections:
      current_costs.append(distance(tracker.centroid, cen)) # prediction and detection
    costs.append(current_costs)

  # assign trackers to object

  # update Kalman Filter
  for tid, oid in zip(tracker_id, obj_id):
    trackers[tid].kalman.update(detection[obj_id])
#Result:

Tracking two balls:

#Full code:

https://github.com/Acytoo/Kalman-Filter-Multiple-Object-Tracking

#References:

The Kalman filter is borrowed from here(python)

The multiple object tracking idea is learn from here(matlab)

相关文章：