With the purpose of secure file and data transfer, SFTP has been a top choice for many users. While it is an efficient and user friendly protocol, engaging with an SFTP server from a programming language such as Python may take a tad more deciphering. With the guidance of the following post, we hope you will come out the other side feeling familiar and ready to utilize and connect to SFTP from Python.
Requirements
Prior to jumping in, some setup is required. We’ll use the ‘pysftp’ package to connect and pass commands to the SFTP server. When you are ready to install it, manually run:
pip install pysftpOr create a requirements.txt file and declare your dependencies within. Then, save the following as requirements.txt:
pysftp>=0.2.9Then run:
pip install -r requirements.txtConnecting to SFTP
In this post, we’ll be using an environment variable named SFTPTOGO_URL that contains all the information required to connect to an SFTP server using a URI format: sftp://user:password@host. The variable is parsed to extract the URI parts using `urlparse`, and the remote server’s host key is verified by default using ~/.ssh/known_hosts.
Once the connection is established, the SFTP client object will be assigned to the variable: connection.
import pysftp
from urllib.parse import urlparse
import os
class Sftp:
def __init__(self, hostname, username, password, port=22):
"""Constructor Method"""
# Set connection object to None (initial value)
self.connection = None
self.hostname = hostname
self.username = username
self.password = password
self.port = port
def connect(self):
"""Connects to the sftp server and returns the sftp connection object"""
try:
# Get the sftp connection object
self.connection = pysftp.Connection(
host=self.hostname,
username=self.username,
password=self.password,
port=self.port,
)
except Exception as err:
raise Exception(err)
finally:
print(f"Connected to {self.hostname} as {self.username}.")
def disconnect(self):
"""Closes the sftp connection"""
self.connection.close()
print(f"Disconnected from host {self.hostname}")Listing Files
Now that the connection is all set up, we can use it to list files on the remote SFTP server. This is achieved by calling the connection object’s listdir function or the listdir_attr function. Either one of these functions can take a remote path argument or, if omitted, will list the files and directories in the current remote directory. The listdir function returns a list of filenames (as string), while the listdir_attr function returns a list of SFTPAttributes objects containing the file size, creation, modification timestamps and permissions, in addition to the file name. In our example, our wrapper function returns a generator, allowing the function caller to iterate over the returned files.
def listdir(self, remote_path):
"""lists all the files and directories in the specified path and returns them"""
for obj in self.connection.listdir(remote_path):
yield obj
def listdir_attr(self, remote_path):
"""lists all the files and directories (with their attributes) in the specified path and returns them"""
for attr in self.connection.listdir_attr(remote_path):
yield attrUpload File
The next step is to upload a file. Use the connection object’s put function and pass the path to the local file and the remote path, which is also where the file should end up by the end of the upload. The function call would look like: connection.put("./local.txt", "./remote.txt")
def upload(self, source_local_path, remote_path):
"""
Uploads the source files from local to the sftp server.
"""
try:
print(
f"uploading to {self.hostname} as {self.username} [(remote path: {remote_path});(source local path: {source_local_path})]"
)
# Download file from SFTP
self.connection.put(source_local_path, remote_path)
print("upload completed")
except Exception as err:
raise Exception(err)Download File
The last job left for us to complete is downloading our files. Use the connection object’s get function, and pass the path to the remote file and the local path, where you would store the downloaded file. You would call the function like so: connection.get("./remote.txt", "./download.txt")
def download(self, remote_path, target_local_path):
"""
Downloads the file from remote sftp server to local.
Also, by default extracts the file to the specified target_local_path
"""
try:
print(
f"downloading from {self.hostname} as {self.username} [(remote path : {remote_path});(local path: {target_local_path})]"
)
# Create the target directory if it does not exist
path, _ = os.path.split(target_local_path)
if not os.path.isdir(path):
try:
os.makedirs(path)
except Exception as err:
raise Exception(err)
# Download from remote sftp server to local
self.connection.get(remote_path, target_local_path)
print("download completed")
except Exception as err:
raise Exception(err)The Whole Thing
So we’ve made it to the finish line! If you would like to run the entire program from start to finish, copy the following code and save it as main.py:
import pysftp
from urllib.parse import urlparse
import os
class Sftp:
def __init__(self, hostname, username, password, port=22):
"""Constructor Method"""
# Set connection object to None (initial value)
self.connection = None
self.hostname = hostname
self.username = username
self.password = password
self.port = port
def connect(self):
"""Connects to the sftp server and returns the sftp connection object"""
try:
# Get the sftp connection object
self.connection = pysftp.Connection(
host=self.hostname,
username=self.username,
password=self.password,
port=self.port,
)
except Exception as err:
raise Exception(err)
finally:
print(f"Connected to {self.hostname} as {self.username}.")
def disconnect(self):
"""Closes the sftp connection"""
self.connection.close()
print(f"Disconnected from host {self.hostname}")
def listdir(self, remote_path):
"""lists all the files and directories in the specified path and returns them"""
for obj in self.connection.listdir(remote_path):
yield obj
def listdir_attr(self, remote_path):
"""lists all the files and directories (with their attributes) in the specified path and returns them"""
for attr in self.connection.listdir_attr(remote_path):
yield attr
def download(self, remote_path, target_local_path):
"""
Downloads the file from remote sftp server to local.
Also, by default extracts the file to the specified target_local_path
"""
try:
print(
f"downloading from {self.hostname} as {self.username} [(remote path : {remote_path});(local path: {target_local_path})]"
)
# Create the target directory if it does not exist
path, _ = os.path.split(target_local_path)
if not os.path.isdir(path):
try:
os.makedirs(path)
except Exception as err:
raise Exception(err)
# Download from remote sftp server to local
self.connection.get(remote_path, target_local_path)
print("download completed")
except Exception as err:
raise Exception(err)
def upload(self, source_local_path, remote_path):
"""
Uploads the source files from local to the sftp server.
"""
try:
print(
f"uploading to {self.hostname} as {self.username} [(remote path: {remote_path});(source local path: {source_local_path})]"
)
# Download file from SFTP
self.connection.put(source_local_path, remote_path)
print("upload completed")
except Exception as err:
raise Exception(err)
if __name__ == "__main__":
sftp_url = os.environ.get("SFTPTOGO_URL")
if not sftp_url:
print("First, please set environment variable SFTPTOGO_URL and try again.")
exit(0)
parsed_url = urlparse(sftp_url)
sftp = Sftp(
hostname=parsed_url.hostname,
username=parsed_url.username,
password=parsed_url.password,
)
# Connect to SFTP
sftp.connect()
# Lists files with attributes of SFTP
path = "/"
print(f"List of files with attributes at location {path}:")
for file in sftp.listdir_attr(path):
print(file.filename, file.st_mode, file.st_size, file.st_atime, file.st_mtime)
# Upload files to SFTP location from local
local_path = "/Users/saggi/Downloads/tls2.png"
remote_path = "/tls2.png"
sftp.upload(local_path, remote_path)
# Lists files of SFTP location after upload
print(f"List of files at location {path}:")
print([f for f in sftp.listdir(path)])
# Download files from SFTP
sftp.download(
remote_path, os.path.join(remote_path, local_path + '.backup')
)
# Disconnect from SFTP
sftp.disconnect()Finally, run it using the command:
python main.py
Congratulations on connecting to SFTP using Python!
Check out more code samples on Github.