有时长时间运行ssh命令会停止打印到标准输出

Question

我一直在使用Perl :: Net :: SSH来自动运行远程盒子上的一些脚本。但是，这些脚本中的一些需要很长时间才能完成（一两个小时），有时我会停止从中获取数据，而不会实际上丢失连接。

下面是我使用的代码：

sub run_regression_tests { 
    for(my $i = 0; $i < @servers; $i++){ 
     my $inner = $users[$i]; 
     foreach(@$inner){ 
      my $user = $_; 
      my $server = $servers[$i]; 

      my $outFile; 
      open($outFile, ">" . $outputDir . $user . "@" . $server . ".log.txt"); 
      print $outFile "Opening connection to $user at $server on " . localtime() . "\n\n"; 
      close($outFile); 

      my $pid = $pm->start and next; 

       print "Connecting to [email protected]" . "$server...\n"; 

       my $hasWentToDownloadYet = 0; 
       my $ssh = Net::SSH::Perl->new($server, %sshParams); 
       $ssh->login($user, $password);    

       $ssh->register_handler("stdout", sub { 
        my($channel, $buffer) = @_;    
        my $outFile; 
        open($outFile, ">>", $outputDir . $user . "@" . $server . ".log.txt");     
        print $outFile $buffer->bytes;    
        close($outFile);     

        my @lines = split("\n", $buffer->bytes); 
        foreach(@lines){ 
         if($_ =~ m/REGRESSION TEST IS COMPLETE/){ 
          $ssh->_disconnect(); 

          if(!$hasWentToDownloadYet){ 
           $hasWentToDownloadYet = 1; 
           print "Caught exit signal.\n"; 
           print("Regression tests for ${user}\@${server} finised.\n"); 
           download_regression_results($user, $server); 
           $pm->finish; 
          } 
         } 
        } 

       }); 
       $ssh->register_handler("stderr", sub { 
        my($channel, $buffer) = @_;    
        my $outFile; 
        open($outFile, ">>", $outputDir . $user . "@" . $server . ".log.txt"); 

        print $outFile $buffer->bytes;    

        close($outFile);     
       }); 
       if($debug){ 
        $ssh->cmd('tail -fn 40 /GDS/gds/gdstest/t-gds-master/bin/comp.reg'); 
       }else{ 
        my ($stdout, $stderr, $exit) = $ssh->cmd('. ./.profile && cleanall && my.comp.reg'); 
        if(!$exit){ 
         print "SSH connection failed for ${user}\@${server} finised.\n"; 
        } 
       } 
       #$ssh->cmd('. ./.profile'); 

       if(!$hasWentToDownloadYet){ 
        $hasWentToDownloadYet = 1; 
        print("Regression tests for ${user}\@${server} finised.\n"); 
        download_regression_results($user, $server); 
       } 

      $pm->finish;   
     } 
    } 
    sleep(1); 
    print "\n\n\nAll tests started. Tests typically take 1 hour to complete.\n"; 
    print "If they take significantly less time, there could be an error.\n"; 
    print "\n\nNo output will be printed until all commands have executed and finished.\n"; 
    print "If you wish to watch the progress tail -f one of the logs this script produces.\n Example:\n\t" . 'tail -f ./[email protected]' . "\n"; 
    $pm->wait_all_children; 
    print "\n\nAll Tests are Finished. \n"; 
} 

这里是我的％sshParams：

my %sshParams = (
    protocol => '2', 
    port => '22', 
    options => [ 
     "TCPKeepAlive yes", 
     "ConenctTimeout 10", 
     "BatchMode yes" 
    ] 
); 

有时长时间运行命令的随机一个刚刚停止印刷/烧制的标准输出或标准错误事件并不会退出。 SSH连接不会死（据我所知），因为$ssh->cmd仍然阻塞。

任何想法如何纠正这种行为？

Answer 1

在您％sshParams哈希，您可能需要添加“TCPKEEPALIVE是”你的选择：

$sshParams{'options'} = ["BatchMode yes", "TCPKeepAlive yes"]; 

这些选项可能会或可能不适合你，但TCPKEEPALIVE是我会建议设置用于任何长时间运行的SSH连接。如果您的路径中有任何一种有状态的防火墙，它可能会丢失状态，如果它长时间没有通过连接传递流量。

Answer 2

它可能由于您查看REGRESSION TEST IS COMPLETE标记的输出的方式而失败。它可能分成两个不同的SSH数据包，所以你的回调将永远不会发现它。

更好，使用时，它是这一个班轮做过结束远程命令：

perl -pe 'BEGIN {$p = open STDIN, "my.comp.reg |" or die $!}; kill TERM => -$p if /REGRESSION TEST IS COMPLETE/}' 

否则，将关闭远程连接，但不停止远程进程，将留活着。

除此之外，你应该尝试使用Net::OpenSSH或Net::OpenSSH::Parallel，而不是净:: SSH :: Perl中的：

use Net::OpenSSH::Parallel; 

my $pssh = Net::OpenSSH::Parallel->new; 

for my $i (0..$#server) { 
    my $server = $server[$i]; 
    for my $user (@{$users[$ix]}) { 
     $pssh->add_host("$user\@$server", password => $password); 
    } 
} 

if ($debug) { 
    $pssh->all(cmd => { stdout_file => "$outputDir%USER%\@%HOST%.log.txt", 
         stderr_to_stdout => 1 }, 
       'fail -fn 40 /GDS/gds/gdstest/t-gds-master/bin/comp.reg'); 
} 
else { 
    $pssh->all(cmd => { stdout_file => "$outputDir%USER%\@%HOST%.log.txt", 
         stderr_to_stdout => 1 }, 
       '. ./.profile && cleanall && my.comp.reg'); 
} 

$pssh->all(scp_get => $remote_regression_results_path, "regression_results/%USER%\@%HOST%/"); 

$pssh->run;