使用了很久的saiku,決定跟蹤一下代碼,看看它的執行核心過程:
一、入口controller代碼
1.1、頁面打開之後,會發送一個ajax請求
Request URL:
http://l-tdata2.tkt.cn6.qunar.com:8080/saiku/rest/saiku/api/query/execute
Request Method:
POST
1.2、controller,java檔案org.saiku.web.rest.resources.Query2Resource
如果有緩存,直接輸出資料
沒有緩存,計算在輸出資料
二、service代碼
2.1、service,執行核心代碼 org.saiku.service.olap.ThinQueryService的private CellDataSet execute(ThinQuery tq, ICellSetFormatter formatter)方法
執行mdx語句
Long start = (new Date()).getTime();
log.debug("Query Start");
CellSet cellSet = executeInternalQuery(tq); //這是執行mdx語句的地方,需要較長時間
log.debug("Query End");
String runId = "RUN#:" + ID_GENERATOR.get();
Long exec = (new Date()).getTime();
三、核心代碼
執行mdx語句 org.saiku.service.olap.ThinQueryService的CellSet executeInternalQuery(ThinQuery query) throws Exception 方法
CellSet executeInternalQuery(ThinQuery query) throws Exception {
String runId = "RUN#:" + ID_GENERATOR.getAndIncrement();
QueryContext queryContext = context.get(query.getName());
if (queryContext == null) {
queryContext = new QueryContext(Type.OLAP, query);
this.context.put(query.getName(), queryContext);
}
// 根據資料立方體建立olap的jdbc連結
OlapConnection con = olapDiscoverService.getNativeConnection(query.getCube().getConnection());
if (StringUtils.isNotBlank(query.getCube().getCatalog())) {
con.setCatalog(query.getCube().getCatalog());
}
if (queryContext.contains(ObjectKey.STATEMENT)) {
Statement s = queryContext.getStatement();
s.cancel();
s.close();
s = null;
queryContext.remove(ObjectKey.STATEMENT);
}
OlapStatement stmt = con.createStatement(); // 執行個體化Statement對象
queryContext.store(ObjectKey.STATEMENT, stmt);
query = updateQuery(query);
try {
String mdx = query.getParameterResolvedMdx();
log.info(runId + "\tType:" + query.getType() + ":\n" + mdx);
CellSet cs = stmt.executeOlapQuery(mdx); //這裡是執行mdx語句的過程,耗時最久
queryContext.store(ObjectKey.RESULT, cs);
//追蹤代碼cs使用
log.info("cs:" + cs.toString());
if (query != null) {
queryContext.store(ObjectKey.QUERY, query);
}
//追蹤代碼query使用
log.info("query:" + query.toString());
return cs;
} finally {
stmt.close();
queryContext.remove(ObjectKey.STATEMENT);
}
}
四、執行日志:
上面的注釋,是通過日志來作證的,日志如下:
2016-06-12 14:46:21,571 DEBUG [org.saiku.web.rest.resources.Query2Resource] TRACK /query/F7CE71C7-3E29-0A6A-9BC9-FDDA1A129BB7 POST tq:false file:/homes/saiku_search.saiku
2016-06-12 14:46:21,686 DEBUG [org.saiku.service.olap.ThinQueryService] Query Start
2016-06-12 14:46:21,814 INFO [org.saiku.service.olap.ThinQueryService] RUN#:1 Type:QUERYMODEL:
WITH
SET [~COLUMNS] AS
{[category_name_id].[category_name_id].[category_name].Members}
SET [~ROWS_rpt_date_rpt_date] AS
{[rpt_date].[rpt_date].[2016-06-07]}
SET [~ROWS_partner_partner] AS
Hierarchize({{[partner].[partner].[All partners]}, {[partner].[partner].[name].Members}})
SET [~ROWS_from_area_id_from_area_id] AS
Hierarchize({{[from_area_id].[from_area_id].[All from_area_ids]}, {[from_area_id].[from_area_id].[name].Members}})
SET [~ROWS_utmr_page_id_utmr_page_id] AS
{[utmr_page_id].[utmr_page_id].[All utmr_page_ids]}
SET [~ROWS_in_track_in_track] AS
{[in_track].[in_track].[All in_tracks]}
SET [~ROWS_dist_city_dist_city] AS
{[dist_city].[dist_city].[All dist_citys]}
SET [~ROWS_current_city_current_city] AS
{[current_city].[current_city].[All current_citys]}
SET [~ROWS_from_value_id_from_value_id] AS
{[from_value_id].[from_value_id].[All from_value_ids]}
SET [~ROWS_page_id_page_id] AS
{[page_id].[page_id].[All page_ids]}
SELECT
NON EMPTY CrossJoin([~COLUMNS], {[Measures].[num], [Measures].[gid]}) ON COLUMNS,
NON EMPTY NonEmptyCrossJoin([~ROWS_rpt_date_rpt_date], NonEmptyCrossJoin([~ROWS_partner_partner], NonEmptyCrossJoin([~ROWS_from_area_id_from_area_id], NonEmptyCrossJoin([~ROWS_utmr_page_id_utmr_page_id], NonEmptyCrossJoin([~ROWS_in_track_in_track], NonEmptyCrossJoin([~ROWS_dist_city_dist_city], NonEmptyCrossJoin([~ROWS_current_city_current_city], NonEmptyCrossJoin([~ROWS_from_value_id_from_value_id], [~ROWS_page_id_page_id])))))))) ON ROWS
FROM [saiku_search_detail_cube]
2016-06-12 14:50:58,344 INFO [org.saiku.service.olap.ThinQueryService] cs:mondrian.olap4j.FactoryJdbc41Impl$MondrianOlap4jCellSetJdbc41@2c72fc4f
2016-06-12 14:50:58,344 INFO [org.saiku.service.olap.ThinQueryService] query:org.saiku.olap.query2.ThinQuery@3112bd55
2016-06-12 14:50:58,344 DEBUG [org.saiku.service.olap.ThinQueryService] Query End
2016-06-12 14:50:58,442 DEBUG [org.saiku.service.olap.ThinQueryService] cellSet2Matrix End
2016-06-12 14:50:58,443 DEBUG [org.saiku.service.olap.ThinQueryService] calculateTotals End
2016-06-12 14:50:58,443 INFO [org.saiku.service.olap.ThinQueryService] RUN#:2 Size: 23/76 Execute: 276658ms Format: 98ms Totals: 1ms Total: 276757ms
上面是執行了一個超級資料的日志,紅色部分标志出了執行時間最久的部分,日志是我重新編譯代碼得出的,可見執行的核心代碼就是第三部分标出的紅色部分代碼
五、olap4j引擎
第四部分的代碼,核心是建立olap的jdbc連結。下面是原文
olap4j is an open Java API for OLAP.
Think of it like JDBC, but for accessing multi-dimensional data.
olap4j is a common API for any OLAP server, so you can write an analytic application on one server and easily switch it to another. Built on that API, there is a growing collection of tools and components.
總結:
- saiku前端發送mdx查詢ajax請求
- saiku後端接收mdx語句
- 包裝一下查詢内容
- 調用olap4j引擎查詢資料庫結果
- 修飾資料并傳回
- saiku前端展示出來