天天看點

spark需要maven管理嗎_使用spark2.4和maven3.6.0組合開發踩過的坑

ERROR:thread "main" java.lang.ArrayIndexOutOfBoundsException: 10582

這是paranamer版本的問題。在spark-core / spark-sql之前添加以下依賴項。

com.thoughtworks.paranamer

paranamer

2.8

Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Stopwatch.elapsedMillis()J

com.google.guava

guava

21.0

Exception in thread "main" java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.()V from class org.apache.hadoop.mapred.FileInputFormat

org.apache.hadoop

hadoop-common

2.7.7

org.apache.hadoop

hadoop-mapreduce-client-core

2.7.7

猜測是因為maven的版本太高