download list of files from a remote ssh/sftp server
I’ve approached this problem before. I’m on a server and I want to copy some directories from another server using scp. You have to write down all the paths of the directory you want of the first server, then go to the second, then do a command like:
scp -r server1:"directory1 directory2 directory3 ..." .
Which is easy with simple filenames but as soon as you need directories in different paths or long named directories it will be pain. And you don’t have shell completion. You could say: “Set up key authentication and you will have shell completion on the remote server”. Well, I don’t wanna set up keys for a server and I want a more practical way to solve this.
I wanted to connect through ssh from nautilus (just type ssh://server on the path bar), surf my remote directories, then download what i need to a second server. Well I did a python program to do that.
It’s pretty straightforward. You connect with nautilus to a server, select all the directories you want then copy them. Open the shell of another server, then do: python3 download.py -s server1 CTRL+V
The CTRL+V will copy all the filename absolute paths straight to the shell. Just hit enter, it will ask for password, just give it and you are done. It will start downloading all the list of directories of files you want from server1
This is the python program that makes this possible.
<pre>
#!/usr/bin/python
# -*- coding: UTF-8 -*-
'''
This programs allows easy copies of remote directories to the local path using scp
usage:
python3 download.py [options] list-of-files
options:
-s, --server server:set the server instead of deducing it
@author: Vincenzo Ampolo <vincenzo.ampolo@gmail.com>
'''
import sys
import os
import getopt
import urllib.parse
def shellquote(s):
"""escapes the given string s such that it can be used safely in a shell
"""
return "'" + s.replace("'", "'\\''") + "'"
def main(argv=None):
if argv is None:
argv = sys.argv
opts, extraparams = getopt.gnu_getopt(argv[1:], "hvs:", ["help", "server"])
verbose = False
server = None
for o, a in opts:
if o == "-v":
verbose = True
elif o in ("-h", "--help"):
print(__doc__)
sys.exit(0)
elif o in ("-s", '--server'):
server = a
else:
assert False, "UnhandledOption"
l = []
for path in extraparams:
url = urllib.parse.urlparse(path)
if not server:
server = url.host
quoted_path = shellquote(urllib.parse.unquote(url.path))
l.append(quoted_path)
files_to_download = ' '.join(l)
cmd = 'scp -r %s:"%s" .'%(server, files_to_download)
os.system(cmd)
return 0
if __name__ == '__main__':
sys.exit(main())
</pre>
