What are applications of object/human tracking in autonomous cars?


Objects tracking is finding the trajectory of each object in consecutive frames. Human tracking is a subset of object tracking which just considers humans.

I've seen many papers that divide tracking methods into two parts:

  1. Online tracking: Tracker just uses current and previous frames.
  2. Offline tracking: Tracker uses all frames.

All of them mention that online tracking is suitable for autonomous driving and robotics, but I don't understand this part. What are the applications of object/human tracking in autonomous driving?

Do you know some related papers?

Farhad Bat

Posted 2018-09-07T17:14:48.587

Reputation: 105

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than facts, references, or specific expertise. If this question can be reworded to fit the rules in the help center – quintumnia – 2018-09-07T17:59:05.267

1It purely depends on which algorithm the self-driving car is using. But what exactly do you mean by human tracking? A lot of modern self-driving cars use the YOLO (you only look once) algorithm for object detection. So at each sampling time, it looks at the surroundings of the car and identify objects, such as humans. – Rui Nian – 2018-09-07T18:13:38.457

1@RuiNian Object detection determines different objects and their location in a frame, Object tracking/Human tracking determines trajectory(Path) of objects/humans in consecutive frames.

I have seen many papers that mention online tracking is useful for autonomous driving. But I don't understand how tracking is useful for autonomous driving. – Farhad Bat – 2018-09-07T18:21:04.880

2I personally don't have a lot of industry experience with self driving cars, but the reason for tracking in other control problems is for trajectory optimization. Traditionally, predictive controls are used to forecast the trajectories of dynamic states (person in this case) in the environment so the controller (the car in this case) can behave optimally in the most probabilistic scenario. For example, if a person is biking at 50 km/h across a street, there is a high chance he is going to cross the street, so the car must behave differently than if there was no biker. – Rui Nian – 2018-09-07T18:30:27.490

1Online trajectory optimization is very high computational cost, so I believe most trajectory optimizations for cars are done using offline methods, and the models are put online. I know the UC Berkeley self driving car team does this. – Rui Nian – 2018-09-07T18:30:29.897

1@quintumnia I edited the question. – Farhad Bat – 2018-09-07T18:39:10.840



There are two important parts:

  • When you track, you have a trajectory you can expect. So it becomes easier to "look into the future" is you understand the past
  • When you loose an object you had a trajectory of before, you might be able to come up with an explanation that can influence your behavior

Martin Thoma

Posted 2018-09-07T17:14:48.587

Reputation: 1 023


Offline tracking requires knowlege of the future of an object.

For real time applications such as the ones you cited, offline tracking might be impeditive given that the method needs to wait for future frames.

Pedro Henrique Monforte

Posted 2018-09-07T17:14:48.587

Reputation: 189