Presto自定義函數@SqlNullable血淚史

看到标題我們會想到是由于@SqlNullable注解引發的問題，我們先看一段代碼，正是這段有意思的代碼，讓我糾結了2個多小時，引發了Presto的問題。

@Description("user_id")
@ScalarFunction("user_id")
@SqlType(StandardTypes.VARCHAR)
public static Slice userId(@SqlType(StandardTypes.VARCHAR) Slice value) {
    String _value = value.toStringUtf8();
    if (StringUtils.containsWhitespace(_value)) {
        _value = StringUtils.replace(_value, " ", "+");
    } 
    return Slices.utf8Slice(makeErrorMsgBase64(_value));
}
複制代碼           

這段代碼很簡單，就是我們将傳遞進來的base64的字元串解碼成實際的字元串，單單從代碼上看是不會有什麼問題的。當我們實際運作這個函數的時候問題就出現了，以下是一個使用示例：

select user_id(str) from temp.users limit 100;
複制代碼           

看執行的SQL似乎也沒有什麼異常，但就是這麼簡簡單單的一個函數引發了Presto的問題，那就是java.lang.NullPointerException: undefined錯誤，這個錯誤的具體内容如下

java.lang.NullPointerException: undefined
	at io.prestosql.type.VarcharOperators.equal(VarcharOperators.java:53)
	at io.prestosql.$gen.CursorProcessor_20200927_063218_2398.filter(Unknown Source)
	at io.prestosql.$gen.CursorProcessor_20200927_063218_2398.process(Unknown Source)
	at io.prestosql.operator.ScanFilterAndProjectOperator$RecordCursorToPages.process(ScanFilterAndProjectOperator.java:323)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
	at io.prestosql.operator.WorkProcessorUtils$YieldingProcess.process(WorkProcessorUtils.java:181)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
	at io.prestosql.operator.WorkProcessorUtils.lambda$processStateMonitor$2(WorkProcessorUtils.java:200)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils.lambda$flatten$6(WorkProcessorUtils.java:277)
	at io.prestosql.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:319)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils$3.process(WorkProcessorUtils.java:306)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
	at io.prestosql.operator.WorkProcessorUtils.lambda$processStateMonitor$2(WorkProcessorUtils.java:200)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorUtils.getNextState(WorkProcessorUtils.java:221)
	at io.prestosql.operator.WorkProcessorUtils.lambda$finishWhen$3(WorkProcessorUtils.java:215)
	at io.prestosql.operator.WorkProcessorUtils$ProcessWorkProcessor.process(WorkProcessorUtils.java:372)
	at io.prestosql.operator.WorkProcessorSourceOperatorAdapter.getOutput(WorkProcessorSourceOperatorAdapter.java:149)
	at io.prestosql.operator.Driver.processInternal(Driver.java:379)
	at io.prestosql.operator.Driver.lambda$processFor$8(Driver.java:283)
	at io.prestosql.operator.Driver.tryWithLock(Driver.java:675)
	at io.prestosql.operator.Driver.processFor(Driver.java:276)
	at io.prestosql.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:1076)
	at io.prestosql.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:[email protected])
	at io.prestosql.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:484)
	at io.prestosql.$gen.Presto_341____20200925_110330_2.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
複制代碼           

通過上面錯誤我們推算出應該是由于資料導緻空指針異常，那麼問題出在哪裡呢？問題就出在查詢str這個字段中，這個字段我們經過實際的查詢發現是有''的資料，在做轉換的時候出現了空指針問題，于是我們修改UDF的源碼為

@Description("user_id")
@ScalarFunction("user_id")
@SqlType(StandardTypes.VARCHAR)
@SqlNullable
public static Slice userId(@SqlNullable @SqlType(StandardTypes.VARCHAR) Slice value) {
    String _value = value.toStringUtf8();
    if (StringUtils.containsWhitespace(_value)) {
        _value = StringUtils.replace(_value, " ", "+");
    } 
    return Slices.utf8Slice(makeErrorMsgBase64(_value));
}
複制代碼           

我們在方法和參數上添加了@SqlNullable注解用于标記此函數可以接收空的資料，這似乎看着也沒有問題，我們将該函數重新釋出，再次執行SQL發現還存在相同的問題，于是又将代碼修改為以下内容

@Description("user_id")
@ScalarFunction("user_id")
@SqlType(StandardTypes.VARCHAR)
public static Slice userId(@SqlNullable @SqlType(StandardTypes.VARCHAR) Slice value) {
    String _value = value.toStringUtf8();
    if (StringUtils.containsWhitespace(_value)) {
        _value = StringUtils.replace(_value, " ", "+");
    } 
    return Slices.utf8Slice(makeErrorMsgBase64(_value));
}
複制代碼           

删除了方法上的@SqlNullable注解，再次運作發現不會再出現這個錯誤，但是Presto服務中不斷的報出空指針錯誤，隻是不在回報給查詢用戶端，原本以為此問題已經解決，然而更有意思的事情發生了，我們使用了343版本測試成功後，線上版本是341，更新線上後發現此問題再次複現，如果再次在方法上加上@SqlNullable注解在341版本上又會修複這個問題，目前已将這個問題回報給官方，推薦大家使用343版本！

原文連結：https://juejin.cn/post/7171982796108333063