行為規劃的内容包括車輛如何生成安全的、可行駛的軌迹以到達目的地：
我們會使用計算機視覺和傳感器融合得到的資料，來了解我們周圍的環境，也将使用從定位子產品得到的資料來精确了解我們具體置身何處，路徑規劃使用所有這些資料來決定下一步執行何種動作，
然後，路徑規劃構造出一條軌迹，讓控制器去執行。
無人駕駛系統的規劃層包括三層結構：
算法在任務規劃中應用非常廣泛，作為離散空間搜尋算法，算法多用于離散空間最優路徑搜尋問題的解決方案；
行為規劃，是無人車系統決策的核心部分，通常使用有限狀态機來設計一個行為決策模型。
行為規劃器目前是個黑盒子，它把地圖資料，目的地路線，作為輸入，結合周圍環境中靜态或者動态障礙物的下一步可能動向，
生成無人車的下一個動作，對此動作，會有規劃器負責平滑，安全，無碰撞的執行。
行為子產品的職責，是提出一些建議動作，這些動作因該是可行的、最安全的、不會違規的，高效的，但不負責動作執行的細節。也不負責進行碰撞檢測。
我們使用“有限狀态機”來解決行為規劃問題。
有限狀态機
有限狀态機基于有限的離散狀态來做決策，
有限狀态機是一個非常簡單的抽象反應系統，因為它隻針對特定的外界輸入産生數量有限的相應。
其核心思想是，通過有限的狀态描述定義，組合産生大量的複雜的邏輯行為。
一個有限狀态機通常包括，輸入集合，輸出集合，轉換邏輯；每個狀态機根據是否有輸出可以分為兩類，接收器(Acceptor)和變換器(Transducer)。如上圖中，S4是接收器，其他是轉換器。
在道路行駛場景中，可以用5個狀态來組合表達無人車的所有可能駕駛動作。
設計狀态機如下圖：
轉換函數
行為規劃的狀态機需要輸入的資料是
路徑規劃實作僞代碼：
def transition_function(predictions, current_fsm_state, current_pose, cost_functions, weights):
# only consider states which can be reached from current FSM state.
possible_successor_states = successor_states(current_fsm_state)
# keep track of the total cost of each state.
costs = []
for state in possible_successor_states:
# generate a rough idea of what trajectory we would
# follow IF we chose this state.
trajectory_for_state = generate_trajectory(state, current_pose, predictions)
# calculate the "cost" associated with that trajectory.
cost_for_state = 0
for i in range(len(cost_functions)) :
# apply each cost function to the generated trajectory
cost_function = cost_functions[i]
cost_for_cost_function = cost_function(trajectory_for_state, predictions)
# multiply the cost by the associated weight
weight = weights[i]
cost_for_state += weight * cost_for_cost_function
costs.append({'state' : state, 'cost' : cost_for_state})
# Find the minimum cost state.
best_next_state = None
min_cost = 9999999
for i in range(len(possible_successor_states)):
state = possible_successor_states[i]
cost = costs[i]
if cost < min_cost:
min_cost = cost
best_next_state = state
return 成本函數
在狀态機轉換過程函數中，必須用到一個值，就是成本函數，即在可行的轉換狀态中，選擇最優的狀态。我們希望設計合理的成本函數，可以對事物作出正确評估。
如何為車速設計成本函數
一方面想盡快到達目的地，一方面又不能超速，需要的控制量就是汽車的期望車速，假設成本函數值是(0~1)之間，且成本函數是線性的，通過調整成本函數的權重，進而調節成本函數的相對重要性.
設計一個車速和成本函數的關系圖圖下：
一個執行個體，假設無人車位于左下角，目的地是右下角的G點，無人車前方有一個車輛，但是速度非常慢；
這時無人車的可選項有兩個，一是keep lane直到終點，二是先變道左側，超過前車，再變道回目的車道，快速行駛至目的地。
這兩個軌迹，哪個更優？我們用兩個參數來模組化：
：距離目的地橫向距離；
代碼實作：
double goal_distance_cost(int goal_lane, int intended_lane, int final_lane,
double distance_to_goal) {
// The cost increases with both the distance of intended lane from the goal
// and the distance of the final lane from the goal. The cost of being out
// of the goal lane also becomes larger as the vehicle approaches the goal.
int delta_d = 2.0 * goal_lane - intended_lane - final_lane;
double cost = 1 - exp(-(std::abs(delta_d) / distance_to_goal));
return cost;
} 大部分情況下，一個簡單的代價函數不能較好的獲得複雜的車輛行為。代價函數可以有多個實作，比如針對以下需求設計代價函數
Your task in the implementation will be to create a cost function that satisifes:
The cost decreases as both intended lane and final lane are higher speed lanes.
The cost function provides different costs for each possible behavior: KL, PLCR/PLCL, LCR/LCL.
The values produced by the cost function are in the range 0 to 1. double inefficiency_cost(int target_speed, int intended_lane, int final_lane,
const std::vector<int> &lane_speeds) {
// Cost becomes higher for trajectories with intended lane and final lane
// that have traffic slower than target_speed.
double speed_intended = lane_speeds[intended_lane];
double speed_final = lane_speeds[final_lane];
double cost = (2.0*target_speed - speed_intended - speed_final)/target_speed;
return cost;
} 設計成本函數是一個困難的工作，而将它們組合起來以生成可用的車輛行為則是非常困難的工作，
其中一些困難與成本函數設計有關，既要解決新問題，又不會影響老問題；
針對不同場景，需調節不同參數
時序排程問題
由于行為子產品相比其他子產品，更新頻率較低，而整個系統不允許存在某個特别慢的子產品，這會阻止其他子產品正确運作。
解決同步問題方法：比如行為子產品周期更新時，預測子產品循環尚未完成，
這時，如果行為子產品選擇等待，則會阻塞整個循環管線，影響下遊子產品，正确做法是使用現有資料，并接受這些資料并非最新資料的事實。
有限狀态機是行為規劃問題的一種解決方案，在狀态空間較小情況下，優先狀态機通常工作的很好。例如在高速公路上駕駛，
但對于更複雜的場景，比如城市裡駕駛，其他方法可能更适用，
Additional Resources on Path Planning
Indoors
Intention-Net: Integrating Planning and Deep Learning for Goal-Directed Autonomous Navigation by S. W. Gao, et. al.
Abstract: How can a delivery robot navigate reliably to a destination in a new office building, with minimal prior information? To tackle this challenge, this paper introduces a two-level hierarchical approach, which integrates model-free deep learning and model-based path planning. At the low level, a neural-network motion controller, called the intention-net, is trained end-to-end to provide robust local navigation. The intention-net maps images from a single monocular camera and “intentions” directly to robot controls. At the high level, a path planner uses a crude map, e.g., a 2-D floor plan, to compute a path from the robot’s current location to the goal. The planned path provides intentions to the intention-net. Preliminary experiments suggest that the learned motion controller is robust against perceptual uncertainty and by integrating with a path planner, it generalizes effectively to new environments and goals.
City Navigation
Learning to Navigate in Cities Without a Map by P. Mirowski, et. al.
Abstract: Navigating through unstructured environments is a basic capability of intelligent creatures, and thus is of fundamental interest in the study and development of artificial intelligence. Long-range navigation is a complex cognitive task that relies on developing an internal representation of space, grounded by recognizable landmarks and robust visual processing, that can simultaneously support continuous self-localization (“I am here”) and a representation of the goal (“I am going there”). Building upon recent research that applies deep reinforcement learning to maze navigation problems, we present an end-to-end deep reinforcement learning approach that can be applied on a city scale. […] We present an interactive navigation environment that uses Google StreetView for its photographic content and worldwide coverage, and demonstrate that our learning method allows agents to learn to navigate multiple cities and to traverse to target destinations that may be kilometers away. […]
Intersections
A Look at Motion Planning for Autonomous Vehicles at an Intersection by S. Krishnan, et. al.
Abstract: Autonomous Vehicles are currently being tested in a variety of scenarios. As we move towards Autonomous Vehicles, how should intersections look? To answer that question, we break down an intersection management into the different conundrums and scenarios involved in the trajectory planning and current approaches to solve them. Then, a brief analysis of current works in autonomous intersection is conducted. With a critical eye, we try to delve into the discrepancies of existing solutions while presenting some critical and important factors that have been addressed. Furthermore, open issues that have to be addressed are also emphasized. We also try to answer the question of how to benchmark intersection management algorithms by providing some factors that impact autonomous navigation at intersection.
Planning in Traffic with Deep Reinforcement Learning