KubeCon: Flink on K8s目录集群概况玩转Flink on Kubernetes

目录

• 集群概况
• 玩转Flink on Kubernetes
  • WindowJoin
  • SQL

BigData Manager简介

BigData on Kubernetes

部署向导

运维管理

开发者IDE

简介

Example illustrating a windowed stream join between two data streams.

The example works on two input streams with pairs (name, grade) and (name, salary) respectively. It joins the steams based on "name" within a configurable window. The example uses a built-in sample data generator that generates the steams of pairs at a configurable rate.

• Source1：name, grade
• Source2：name, salary
• Result：name, grade, salary

  

过程

创建Deployment

1. 浏览器打开
   • 0) http://47.92.41.71:8080/app/
   • 1) http://39.98.106.146:8080/app/
   • 2) http://39.98.8.131:8080/app/
   • 3) http://39.98.7.153:8080/app/
   • 4) http://39.98.106.174:8080/app/
   • 5) http://39.98.8.106:8080/app/
   • 6) http://39.98.106.162:8080/app/
   • 7) http://47.92.44.149:8080/app/
   • 8) http://39.98.7.241:8080/app/
   • 9) http://39.98.7.20:8080/app/
3. Organization中输入用户名
4. Configuration：
   • Intepreter选择Blink/JAR
   • Blink Version: 3.2.1 / blink-3.2-SNAPSHOT
   • Jar URI: hdfs:///example/flink-examples-WindowJoin.jar
   • entryClass: org.apache.flink.streaming.examples.join.WindowJoin
5. 点击Create Deployment

启动job

如上图操作

查看结果

1. 点击“Blink UI”，跳转到Apache Flink的dashboard
2. Jobs -> Running Jobs -> Windowed Join Example
3. 结果日志查看：Task Managers -> Path, ID -> Log

停止Job

1. 回到Deployment页面，点击Cancel（集群资源有限，为了后续体验，请一定停掉此job）

实时热门商品， 每隔5分钟输出最近一小时内点击量最多的前 N 个商品(例子详情，请移步

http://wuchong.me/blog/2018/11/07/use-flink-calculate-hot-items/

，天池大赛的数据）

列名称	说明
用户ID	整数类型，加密后的用户ID
商品ID	整数类型，加密后的商品ID
商品类目ID	整数类型，加密后的商品所属类目ID
行为类型	字符串，枚举类型，包括(‘pv’, ‘buy’, ‘cart’, ‘fav’)
时间戳	行为发生的时间戳，单位秒

2. • Intepreter选择Blink/SQL
   • Execution Mode：STREAM
   • Artifact：HotItem
   • Runtime Configuration：
     • state.backend.type = rocksdb
     • state.backend.rocksdb.ttl.ms = 129600000

Job信息