我們在自動駕駛汽車的背景下解決聯合感覺和運動預測問題。為了實作這一目标，我們提出了 PnPNet，這是一個端到端的模型，它将連續的傳感器資料作為輸入，并在每個時間步輸出物體軌迹和未來的軌迹。其關鍵元件是一個新穎的跟蹤子產品，它從檢測到的物體軌迹線上生成物體軌迹，并利用軌迹級特征進行運動預測。具體來說，通過解決資料關聯問題和軌迹估計問題，在每一個時間步驟中更新物體軌迹。重要的是，整個模型是端到端可訓練的，并從所有任務的聯合優化中受益。我們在兩個大規模的駕駛資料集上驗證了 PnPNet，并顯示出了比最先進的模型有了顯著的改進，具有更好的閉塞恢複能力和更準确的未來預測能力。
原文題目：PnPNet: End-to-End Perception and Prediction with Tracking in the Loop
原文：We tackle the problem of joint perception and motion forecasting in the context of self-driving vehicles. Towards this goal we propose PnPNet, an end-to-end model that takes as input sequential sensor data, and outputs at each time step object tracks and their future trajectories. The key component is a novel tracking module that generates object tracks online from detections and exploits trajectory level features for motion forecasting. Specifically, the object tracks get updated at each time step by solving both the data association problem and the trajectory estimation problem. Importantly, the whole model is end-to-end trainable and benefits from joint optimization of all tasks. We validate PnPNet on two large-scale driving datasets, and show significant improvements over the state-of-the-art with better occlusion recovery and more accurate future prediction.
原文作者：Ming Liang, Bin Yang, Wenyuan Zeng, Yun Chen, Rui Hu, Sergio Casas, Raquel Urtasun
原文位址：https://arxiv.org/abs/2005.14711
PnPNet：具有環路跟蹤的端到端感覺和預測（CS CV）.pdf