ssh隧道到运行在集群计算节点上的服务器

Question

我有一个脚本来启动运行在集群特定节点上的Jupyter Notebook服务器，并且需要能够从本地Windows机器（浏览器界面）访问运行实例。 ssh命令是通过paramiko处理的，我想要创建隧道的类似pythonic方式，而不仅仅是系统调用。我尝试了sshtunnel，不幸的是，这是我目前的绊脚石。

我需要创建两个隧道，如下所述。为了清楚起见，HOME是Windows机器，HOST是群集头节点，NODE是服务器运行的特定节点。

服务器被绑定到本地主机，默认情况下在端口8888母国和东道国下方的端口是任意的，我只是用它们来说明（在现实中，最便捷的母港也是8888）

[HOME] <------> [HOST] <------> [NODE] 
1111   2222   8888 

我知道下面的代码是不完整的，只是显而易见的错误，因为我无法弄清楚如何正确组合本地/远程绑定。现在我不知道我是否在咆哮错误的树 - 文档表明我可以一次创建两个隧道，但是我需要两个单独的实例吗？

from sshtunnel import SSHTunnelForwarder 

t1 = SSHTunnelForwarder(
    ('headnode.server', 22), 
    ssh_username='meagain', 
    ssh_pkey='~/.ssh/id_rsa', 
    local_bind_address = ('localhost',1111), 
    remote_bind_addresses =[ ('localhost',2222), ('compute.node',8888)]) 
t1.start() 

编辑：尝试选项2，如下所述。使用SSH客户端，配置设置并在命令行上正常工作。作为参考，这是阿纳康达下用蟒3.6，Windows 7的机器（因此附加文件路径的下方争论）

print(sshtunnel.__version__) 
'0.1.0' 
print(paramiko.__version__) 
'2.1.2' 



t1 = sshtunnel.SSHTunnelForwarder(
    NODE, 
    ssh_username=USER, 
    ssh_pkey=paramiko.RSAKey.from_private_key_file(os.path.expanduser(os.path.normpath('~/.ssh/id_rsa'))), 
    ssh_proxy_enabled=True, # DEFAULT 
    ssh_config_file=os.path.expanduser(os.path.normpath('~/.ssh/config')), # if rather than default 
    local_bind_address=('localhost', 1111), 
    remote_bind_address=('localhost', 8889) 
) 
t1.start() 

日志上：

2017-04-06 12:28:25,523| INF | MainThrea/[email protected] | 0 keys loaded from agent 
    t1.start() 
    File "D:\Anaconda3\lib\site-packages\sshtunnel.py", line 1224, in start 
    reason='Could not establish session to SSH gateway') 
    File "D:\Anaconda3\lib\site-packages\sshtunnel.py", line 1036, in _raise 
    raise exception(reason) 
sshtunnel.BaseSSHTunnelForwarderError: Could not establish session to SSH gateway 
>>> 2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] | paramiko.ssh_exception.ProxyCommandFailure: ('ssh HOST -W NODE:22', 'An operation was attempted on something that is not a socket') 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] | 
2017-04-06 12:28:25,538| ERR | MainThrea/[email protected] | Could not connect to gateway NODE:22 : ssh HOST -W NODE:22 
Traceback (most recent call last): 
    File "D:\Bioinformatics\Scripts\bsub_jupyter\tunneller.py", line 234, in <module> 
2017-04-06 12:28:25,523| INF | MainThrea/[email protected] | Connecting to gateway: NODE:22 as user 'USER' 
2017-04-06 12:28:25,523| DEB | MainThrea/[email protected] | Concurrent connections allowed: True 
2017-04-06 12:28:25,523| DEB | MainThrea/[email protected] | Trying to log in with key: b'1e44a013d97d417e4900025c6c11d073' 
2017-04-06 12:28:25,523| DEB | MainThrea/[email protected] | Connecting via proxy: 'HOST' 
2017-04-06 12:28:25,523| ERR | Thread-2/[email protected] | Exception: ('ssh HOST -W NODE:22', 'An operation was attempted on something that is not a socket') 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] | Traceback (most recent call last): 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] | File "D:\Anaconda3\lib\site-packages\paramiko\proxy.py", line 96, in recv 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] |  [self.process.stdout], [], [], select_timeout) 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] | OSError: [WinError 10038] An operation was attempted on something that is not a socket 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] | 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] | During handling of the above exception, another exception occurred: 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] | 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] | Traceback (most recent call last): 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] | File "D:\Anaconda3\lib\site-packages\paramiko\transport.py", line 1749, in run 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] |  self._check_banner() 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] | File "D:\Anaconda3\lib\site-packages\paramiko\transport.py", line 1893, in _check_banner 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] |  buf = self.packetizer.readline(timeout) 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] | File "D:\Anaconda3\lib\site-packages\paramiko\packet.py", line 331, in readline 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] |  buf += self._read_timeout(timeout) 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] | File "D:\Anaconda3\lib\site-packages\paramiko\packet.py", line 485, in _read_timeout 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] |  x = self.__socket.recv(128) 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] | File "D:\Anaconda3\lib\site-packages\paramiko\proxy.py", line 107, in recv 
2017-04-06 12:28:25,538| ERR | Thread-2/[email protected] |  raise ProxyCommandFailure(' '.join(self.cmd), e.strerror) 

Answer 1

由于端口8888被绑定到localhost作为你提到，它不会从HOST到达。如果是这样的话，你有两个选择：

选项A

使用sshtunnel的ssh_proxy参数：

proxy = paramiko.ProxyCommand('ssh HOST -l USER -i IDENTITY_FILE -W headnode.server:22') 
t1 = SSHTunnelForwarder(
    (NODE, 22), 
    ssh_username='meagain', 
    ssh_pkey='~/.ssh/id_rsa', 
    ssh_proxy=proxy, 
    local_bind_address=('localhost', 1111), 
    remote_bind_addresses=('localhost', 8888) 
) 

选项B

使用ssh配置文件。

你需要有一个ProxyCommand条目的有效配置文件NODE如：

Host HOST 
    User HOSTUSER 
    IdentityFile HOSTUSER_PKEY 
Host NODE 
    ProxyCommand ssh HOST -W %h:%p 

然后，打开通道，让它读ProxyCommand项：

t1 = SSHTunnelForwarder(
    (NODE, 22), 
    ssh_username='meagain', 
    ssh_pkey='~/.ssh/id_rsa', 
    ssh_proxy_enabled=True, # DEFAULT 
    ssh_config_file='/path/to/your-config-file', # if rather than default 
    local_bind_address=('localhost', 1111), 
    remote_bind_addresses=('localhost', 8888) 
)