星火卡夫卡流给不兼容的杰克逊异常

问题描述：

java.lang.ExceptionInInitializerError 
    at org.apache.spark.streaming.dstream.InputDStream.<init>(InputDStream.scala:78) 
    at org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.<init>(DirectKafkaInputDStream.scala:62) 
    at org.apache.spark.streaming.kafka010.KafkaUtils$.createDirectStream(KafkaUtils.scala:150) 
    at org.apache.spark.streaming.kafka010.KafkaUtils$.createDirectStream(KafkaUtils.scala:127) 
    at chirpanywhere.stream.classification.service.streaming.StreamingClassificationPipeline.createStreamAndTransform(StreamingClassificationPipeline.scala:69) 
    at chirpanywhere.stream.classification.service.streaming.StreamingClassificationPipeline.streamAndUpdateMLModelCache(StreamingClassificationPipeline.scala:58) 
    at chirpanywhere.stream.classification.service.streaming.StreamingClassificationPipeline.run(StreamingClassificationPipeline.scala:54) 
    at chirpanywhere.stream.classification.service.Boot$.main(Boot.scala:10) 
    at chirpanywhere.stream.classification.service.Boot.main(Boot.scala) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:498) 
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Incompatible Jackson version: 2.7.8 
    at com.fasterxml.jackson.module.scala.JacksonModule$class.setupModule(JacksonModule.scala:64) 
    at com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:19) 
    at com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:730) 
    at org.apache.spark.rdd.RDDOperationScope$.<init>(RDDOperationScope.scala:82) 
    at org.apache.spark.rdd.RDDOperationScope$.<clinit>(RDDOperationScope.scala) 
    at org.apache.spark.streaming.dstream.InputDStream.<init>(InputDStream.scala:78) 
    at org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.<init>(DirectKafkaInputDStream.scala:62) 
    at org.apache.spark.streaming.kafka010.KafkaUtils$.createDirectStream(KafkaUtils.scala:150) 
    at org.apache.spark.streaming.kafka010.KafkaUtils$.createDirectStream(KafkaUtils.scala:127) 
    at chirpanywhere.stream.classification.service.streaming.StreamingClassificationPipeline.createStreamAndTransform(StreamingClassificationPipeline.scala:69) 
    at chirpanywhere.stream.classification.service.streaming.StreamingClassificationPipeline.streamAndUpdateMLModelCache(StreamingClassificationPipeline.scala:58) 
    at chirpanywhere.stream.classification.service.streaming.StreamingClassificationPipeline.run(StreamingClassificationPipeline.scala:54) 
    at chirpanywhere.stream.classification.service.Boot$.main(Boot.scala:10) 
    at chirpanywhere.stream.classification.service.Boot.main(Boot.scala) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:498)

有人能告诉我出了什么问题。

我build.sbt看起来是这样的：

val akkaHttpV = "10.0.0" 
    val sparkV = "2.1.0" 

"log4j" % "log4j" % "1.2.17", 
    "com.typesafe.akka" %% "akka-http" % akkaHttpV, 
    "com.typesafe.akka" %% "akka-http-spray-json" % akkaHttpV, 
    "org.slf4j" % "slf4j-api" % "1.7.21", 
    "org.apache.spark" %% "spark-core" % sparkV, 
    "org.apache.spark" %% "spark-sql" % sparkV, 
    "org.apache.spark" %% "spark-streaming" % sparkV, 
    "org.apache.spark" %% "spark-mllib" % sparkV, 
    "com.github.blemale" %% "scaffeine" % "2.0.0" % "compile", 
    "org.apache.spark" %% "spark-streaming-kafka-0-10" % sparkV , 
    "com.datastax.spark" %% "spark-cassandra-connector" % "2.0.0-M3", 
    "org.json4s" %% "json4s-native" % "3.5.0" , 
    "org.scalatest" %% "scalatest" % "3.0.0" % "test",

错误非常简单。你正在编译哪个杰克逊依赖项，以及Spark使用哪一个？ –

我还没有添加任何杰克逊依赖！因此问这个问题！ –

尝试使用[SBT依赖关系图]（https://github.com/jrudolph/sbt-dependency-graph）查看依赖关系，以查看哪个依赖关系正在使用它。 –

答

异常是由杰克逊斯卡拉模块检查的一个部署它jackson-databind版本时抛出。 Scala模块需要相同的次要版本;在这种情况下，异常消息是不完整的（它应该包括两个版本） - 我猜你有更新的jackson-databind（2.7.8是一个很好的新版本）和旧的Scala模块，也许是一个Spark包（2.5.3？）。因为经常“足够接近”的小版本可以工作：但解决方案是确保数据绑定和Scala模块具有相同的次版本（并且可能也是相同的修补版本）。

星火卡夫卡流给不兼容的杰克逊异常

相关推荐