您的位置: 首页  >  技术问答  >  GCP数据流 - 出现SSLHandshakeException

GCP数据流 - 出现SSLHandshakeException

分类: 技术问答 • 2022-09-22 09:56:01
问题描述：

我遇到了运行时大（> 1000级的CPU），中型（100-1000 CPU）的数据流的工作如下：GCP数据流 - 出现SSLHandshakeException

例外：“javax.net.ssl.SSLHandshakeException：关闭远程主机握手期间 连接

错误不是致命的，但是，一旦它开始它再次出现，大约每30秒。乔布斯显示此错误写不完（我已经等了4倍的预计运行时间）并产生非常有限的结果（不到预期产出的4％）在工作早期就收到了，之后不再生产。

我使用BigQueryIO和JdbcIO Apache光源和接收器。

重要的是要指出，我的工作在6月初正常工作，但自7月初以来已开始显示此错误。

我在Google的企业支持中有一个公开的案例，但我们只是说结果不是即将发布的。从Google获得的唯一兴趣点是可能发生错误，“以防员工扩展并大量访问云存储”。

但是，没有解决方案附加到该声明。 这是记录在日志中填写错误的例子：

exception: "javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake 
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1002) 
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385) 
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413) 
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397) 
    at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559) 
    at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) 
    at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:153) 
    at com.google.api.client.http.javanet.NetHttpRequest.execute(NetHttpRequest.java:93) 
    at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:981) 
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419) 
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352) 
    at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469) 
    at com.google.cloud.hadoop.util.ResilientOperation$AbstractGoogleClientRequestExecutor.call(ResilientOperation.java:166) 
    at com.google.cloud.hadoop.util.ResilientOperation.retry(ResilientOperation.java:66) 
    at com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel.getMetadata(GoogleCloudStorageReadChannel.java:573) 
    at com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel.openStreamAndSetMetadata(GoogleCloudStorageReadChannel.java:645) 
    at com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel.performLazySeek(GoogleCloudStorageReadChannel.java:560) 
    at com.google.cloud.hadoop.gcsio.GoogleCloudStorageReadChannel.read(GoogleCloudStorageReadChannel.java:289) 
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65) 
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109) 
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103) 
    at java.io.InputStream.read(InputStream.java:101) 
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:81) 
    at org.apache.beam.sdk.util.VarInt.decodeLong(VarInt.java:79) 
    at org.apache.beam.sdk.util.VarInt.decodeInt(VarInt.java:63) 
    at org.apache.beam.runners.dataflow.internal.IsmFormat$KeyPrefixCoder.decode(IsmFormat.java:709) 
    at com.google.cloud.dataflow.worker.runners.worker.IsmReader.readKey(IsmReader.java:1001) 
    at com.google.cloud.dataflow.worker.runners.worker.IsmReader.access$2000(IsmReader.java:79) 
    at com.google.cloud.dataflow.worker.runners.worker.IsmReader$WithinShardIsmReaderIterator.advance(IsmReader.java:953) 
    at com.google.cloud.dataflow.worker.runners.worker.IsmReader$WithinShardIsmReaderIterator.start(IsmReader.java:943) 
    at com.google.cloud.dataflow.worker.runners.worker.IsmReader$IsmCacheLoader.call(IsmReader.java:581) 
    at com.google.cloud.dataflow.worker.runners.worker.IsmReader$IsmCacheLoader.call(IsmReader.java:570) 
    at com.google.cloud.dataflow.worker.runners.worker.IsmReader$IsmCacheLoader.call(IsmReader.java:555) 
    at com.google.cloud.dataflow.worker.repackaged.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4904) 
    at com.google.cloud.dataflow.worker.repackaged.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3628) 
    at com.google.cloud.dataflow.worker.repackaged.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2336) 
    at com.google.cloud.dataflow.worker.repackaged.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2295) 
    at com.google.cloud.dataflow.worker.repackaged.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2208) 
    at com.google.cloud.dataflow.worker.repackaged.com.google.common.cache.LocalCache.get(LocalCache.java:4053) 
    at com.google.cloud.dataflow.worker.repackaged.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4899) 
    at com.google.cloud.dataflow.worker.runners.worker.IsmReader.fetch(IsmReader.java:606) 
    at com.google.cloud.dataflow.worker.runners.worker.IsmReader.getBlock(IsmReader.java:771) 
    at com.google.cloud.dataflow.worker.runners.worker.IsmReader.access$1000(IsmReader.java:79) 
    at com.google.cloud.dataflow.worker.runners.worker.IsmReader$IsmPrefixReaderIterator.get(IsmReader.java:642) 
    at com.google.cloud.dataflow.worker.runners.worker.IsmSideInputReader$ListOverReaderIterators.getUsingLong(IsmSideInputReader.java:679) 
    at com.google.cloud.dataflow.worker.runners.worker.IsmSideInputReader$ListOverReaderIterators.access$1300(IsmSideInputReader.java:625) 
    at com.google.cloud.dataflow.worker.runners.worker.IsmSideInputReader$ListOverReaderIterators$ListIteratorOverReaderIterators.next(IsmSideInputReader.java:720) 
    at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1042) 
    at com.application.strategy.simulator.MainStrategySimulator$1.processElement(MainStrategySimulator.java:224) 
    at com.application.strategy.simulator.MainStrategySimulator$1$auxiliary$4N23tth9.invokeProcessElement(Unknown Source) 
    at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:199) 
    at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:157) 
    at com.google.cloud.dataflow.worker.runners.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:244) 
    at com.google.cloud.dataflow.worker.runners.worker.ForwardingParDoFn.processElement(ForwardingParDoFn.java:42) 
    at com.google.cloud.dataflow.worker.runners.worker.DataflowWorkerLoggingParDoFn.processElement(DataflowWorkerLoggingParDoFn.java:47) 
    at com.google.cloud.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:48) 
    at com.google.cloud.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:52) 
    at com.google.cloud.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:198) 
    at com.google.cloud.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:159) 
    at com.google.cloud.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:72) 
    at com.google.cloud.dataflow.worker.runners.worker.DataflowWorker.executeWork(DataflowWorker.java:336) 
    at com.google.cloud.dataflow.worker.runners.worker.DataflowWorker.doWork(DataflowWorker.java:295) 
    at com.google.cloud.dataflow.worker.runners.worker.DataflowWorker.getAndPerformWork(DataflowWorker.java:242) 
    at com.google.cloud.dataflow.worker.runners.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:123) 
    at com.google.cloud.dataflow.worker.runners.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:103) 
    at com.google.cloud.dataflow.worker.runners.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:90) 
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
    at java.lang.Thread.run(Thread.java:745) 
Caused by: java.io.EOFException: SSL peer shut down incorrectly 
    at sun.security.ssl.InputRecord.read(InputRecord.java:505) 
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983) 
    ... 69 more
+0

对此深感抱歉！看看.. – Pablo

答

这个问题的其他实例看网上后，错误似乎只在使用Java 7.这是你的Java版本的情况发生？

如果是，我会建议尝试Java 8，看看是否可以解决这个问题。让我知道如果我们能够进一步帮助！

相关github上的问题：

https://github.com/spotify/scio/issues/604

https://github.com/ljader/redmine-mylyn-plugin/issues/67

相关推荐