1. Flink cluster architecture
- Job Manager: 管理節點，每個叢集至少一個，管理整個叢集計算資源，Job管理與排程執行，以及Checkpoint協調
- Task Manager: 每個叢集有很多Task Manager， 負責計算資源提供
- Client: 本地執行應用main()方法解析Job Graph對象，病最終将Job Gragh送出到Job Manager運作，同時監控job執行的狀态
2. Job Manager
- Checkpoint Coordinator
- JobGraph => Execution Graph
- Task deploy and dispatch
- RPC communication(Actor System)
- Job acceptance(Job Dispatch)
- 叢集資源管理(ResourceManager)
- TaskManager注冊與管理
3. Task Manger
- Task Execution
- Network Manager
- Shuffle Environment Management
- RPC Communication(Actor system)
- Heartbeat with Job Manager and Resource Manager
- Data Exchange
- Memory Management
- Register to Resource Management
- Offer slots to Job Manager
4. Client
- Application’s main() Method 執行
- Job Graph Generate
- Execution Environment 管理
- Job 送出與運作
- Dependency jar ship
- RPC with Job Manager
- 叢集部署(Cluster Deploy)
5. Job Graph
- 通過DAG方式表達使用者程式
- 不通接口程式的抽象表達
- 用戶端和叢集之前的job描述載體
- 節點(Vertices), Result參數
- Flink1.11之前隻能在Client中生成