Six Methods to Upload Files in Python

In this section, you'll use a few different methods to upload a single file in Python: the requests library, the ftplib module, the Filestack API, and a Django web app.

The simplest and most popular method of uploading files to any API from Python scripts uses the requests library. Run the following command to install it:

pip install requests

Next, save the following code snippet in a file titled requests-test.py in a new directory named single and run the command python3 requests-test.py within that directory to try it out:

Next, save the following code snippet in a file titled requests-test.py in a new directory named single and run the command python3 requests-test.py within that directory to try it out:

import requests

# define the relative path of the sample file
file_path = "../samples/sample.pdf"

# store the target API URL
target_url = "https://httpbin.org/post"

# create a reference to the file
target_file = open(file_path, "rb")

# send the request
response = requests.post(target_url, files = {"form_field_name": target_file})

# check the result
if response.ok:
    print("Upload complete")
    print(response.text)
else:
    print("Something went wrong")

Upload complete
{
  "args": {}, 
  "data": "", 
  "files": {
    "form_field_name": "data:application/octet-stream;base64,JVBERi0xLjc.......<file data truncated>.........DQ4IDAgUgogICAvU2l6ZSA4NDkKPj4Kc3RhcnR4cmVmCjE1ODUwOTgKJSVFT0Y="
    }, 
    "form": {}, 
    "headers": {
      "Accept": "*/*", 
      "Accept-Encoding": "gzip, deflate", 
      "Content-Length": "1601484", 
      "Content-Type": "multipart/form-data; boundary=1c4fe623a51f48696fb51cbc00fb9741", 
      "Host": "httpbin.org", 
      "User-Agent": "python-requests/2.28.1", 
      "X-Amzn-Trace-Id": "Root=1-64003a15-3d327a725810a53578d242db"
    }, 
    "json": null, 
    "origin": "106.202.211.62", 
    "url": "https://httpbin.org/post"
}

If you're looking to upload files to an FTP server in Python, you can use the ftplib library to connect to and authenticate with any FTP server and interact with it.

# create a directory for the FTP server
mkdir ftp-server

# change your working directory
cd ftp-server

# install the library
pip3 install pyftpdlib

# start the server
python3 -m pyftpdlib -w --user=username --password=password

Running the commands listed above will start a new FTP server at ftp://127.0.0.1:2121, and you can use the username username and password password to authenticate with it.

You can test your FTP server via the terminal by running the following command in a new terminal window:

curl --user username:password ftp://127.0.0.1:2121/

This should list the files stored in the working directory of the FTP server. If it works, you can now proceed to install the FTP client library by running the following command in the new terminal window:

pip3 install ftplib

Finally, you can upload files to the FTP server you created earlier. Save the following code snippet in a file named ftp-test.py in the single directory and run the command python3 ftp-test.py in a new terminal window inside the single directory:

import ftplib

# initialize the FTP library
ftp = ftplib.FTP()

# define the target FTP server's host and port
ftp_server_host = "localhost"
ftp_server_port = 2121

# connect to the ftp server
ftp.connect(ftp_server_host, ftp_server_port)

# define the username and password for authenticating with the FTP server
ftp_server_username = "username"
ftp_server_password = "password"

# authenticate with the username and password
ftp.login(ftp_server_username, ftp_server_password)

# define the relative path of the sample file
file_path = "../samples/sample.pdf"

# create a reference to the file to be uploaded
target_file = open(file_path, "rb")

# define the STOR command to run on the FTP server
ftp_stor_command = "STOR ftp-server/sample.pdf"

# run the STOR command to upload the file to the server
ftp.storbinary(ftp_stor_command, target_file)

# close the file and ftp instance once done
target_file.close()
ftp.quit()

Check the ftp-server directory you created earlier to find the uploaded file:

In real-world scenarios, the FTP server will be created and run on a remote host and will be accessed over the network by the ftplib library.

pip3 install filestack

Finally, you can start uploading files to Filestack from Python. Save the following code in a file named filestack-test.py in the single directory and run the command python3 filestack-test.py inside the single directory in a new terminal window:

from filestack import Client

# define the Filestack API key
FILESTACK_API_KEY = "<Your API Key Here>"

# create the Filestack client with the API key
client = Client(FILESTACK_API_KEY)

# define the relative path of the sample file
file_path = "../samples/sample.pdf"

# upload the file
new_filelink = client.upload(filepath=file_path)

# print the URL of the file once uploaded
print(new_filelink.url)

While this method of uploading files in Python is simple, it's specific to the Filestack service and will require you to purchase a subscription after a certain amount of usage. This method is great if you are looking to use Filestack to handle file uploads in your app. However, if you're looking to create a custom file-handling service, this method is not for you.

To begin, you first need to set up Django on your local system. You can install it via pip:

pip install Django

Once done, you can create a new Django project using the following command:

python -m django startproject fileserver

This will create a new Django project named fileserver in your current working directory.

Next, you need to change your working directory to the project's root and create a new app:

cd fileserver
python -m django startapp fileuploader

Once the app is created, you need to make a few changes to the server's settings.py and urls.py files.

Add the following code snippet to the bottom of the fileserver/settings.py file, making sure to add import os to the top of the file with the rest of the imports:

MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
MEDIA_URL = '/media/'

These lines define the location of the media folder for your app (ie, where you'll store the uploaded files) and a URL path that you will use later in order to access your uploaded files. You also need to add fileuploader as an installed app in the project in the same file:

INSTALLED_APPS = [
    'fileuploader',
    # other apps
]

Next, add the newly created app to the urls.py file of the server so you can access it via the home route of your app:

# add these imports to the top
from django.conf import settings
from django.conf.urls.static import static
from fileuploader import views as uploader_views

urlpatterns = [
    path('', uploader_views.FileUploadView.as_view(), name='fileupload'),
    # other URL patterns
] + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT)

Once you're done with these, create a model in your app that stores the metadata related to each uploaded file. You'll create this model in the fileuploader/models.py file by adding the following code:

class FileUpload(models.Model):
    file = models.FileField()    
    upload_date = models.DateTimeField(auto_now_add =True)

You will also need to create a view to render and handle the upload form that the user will see. You can do that by adding the following code to the fileuploader/views.py file:

# add the imports to the top
from django.views.generic.edit import CreateView
from django.urls import reverse_lazy
from .models import FileUpload

class FileUploadView(CreateView):
    model = FileUpload
    fields = ['file', ]
    success_url = reverse_lazy('fileupload')
    
    def get_context_data(self, **kwargs):
        context = super().get_context_data(**kwargs)
        context['documents'] = FileUpload.objects.all()
        return context

Finally, you'll create an HTML file that will be served by the view to the user. You'll need to create an HTML file at fileuploader/templates/fileuploader/fileupload_form.html and add the following code to it:

<div style="padding:80px;margin:80px;border:1px solid #bbb">
    <h1>Django File Upload</h1>
    <form method="post" enctype="multipart/form-data">
      {% csrf_token %}
      {{ form.as_p }}
      <button type="submit">Submit</button>
    </form><hr>
    <ul>
    {% for document in documents %}
        <li>
            <a href="{{ document.file.url }}">{{ document.file.name }}</a>
            <small>({{ document.file.size|filesizeformat }}) - {{document.upload_date}}</small>
        </li>
    {% endfor %}
    </ul>
</div>

This completes the setup of the project. You can now update the database of the project (to add the new FileUpload model to it) and run the app with the following commands:

python manage.py makemigrations
python manage.py migrate
python manage.py runserver

You'll be able to view your app at http://localhost:8000:

You can try uploading a file via the app. Once you do, you'll notice that it appears in a list below the form and in the media folder in your project directory as well:

You can use the requests library to upload multiple files to any API endpoint in the same request. Save the following code in a file named requests-test.py in a new directory named multiple and run the command python3 requests-test.py in a new terminal window inside the multiple folder:

import requests

target_file_1_path = "../samples/sample.pdf"
target_file_2_path = "../samples/sample-2.pdf"
target_file_3_path = "../samples/sample-3.pdf"
target_file_4_path = "../samples/sample-4.pdf"

# create references to the files to be uploaded
target_file_1 = open(target_file_1_path, "rb")
target_file_2 = open(target_file_2_path, "rb")
target_file_3 = open(target_file_3_path, "rb")
target_file_4 = open(target_file_4_path, "rb")

# create a dict object with the file objects
target_files = {
    "file-1": target_file_1,
    "file-2": target_file_2,
    "file-3": target_file_3,
    "file-4": target_file_4,
}

# define the target API URL
target_url = "https://httpbin.org/post"

# send the request
response = requests.post(target_url, files=target_files)

# check the result
if response.ok:
    print("Upload complete")
    print(response.text)
else:
    print("Something went wrong")

The output of this method will look similar to this:

...MDAwMCBuCjAwMDAwNTA2OTYgMDAwMDAgbgowMDAwMDUwNzE5IDAwMDAwIG4KMDAwMDA1MDgyOSAwMDAwMCBuCjAwMDAwNTA4NTIgMDAwMDAgbgowMDAwMDUwOTYyIDAwMDAwIG4KMDAwMDA1MDk4NSAwMDAwMCBuCjAwMDAwNTEwOTYgMDAwMDAgbgowMDAwMDUxMTE5IDAwMDAwIG4KMDAwMDA1MTIzMCAwMDAwMCBuCjAwMDAwNTEyNTMgMDAwMDAgbgowMDAwMDUxMzY0IDAwMDAwIG4KMDAwMDA1MTM4NyAwMDAwMCBuCjAwMDAwNTE0OTggMDAwMDAgbgowMDAwMDUxNTIxIDAwMDAwIG4KMDAwMDA1MzgwNCAwMDAwMCBuCjAwMDAwNjMyMDggMDAwMDAgbgowMDAwMDYzMjMzIDAwMDAwIG4KMDAwMDA3MzIwNSAwMDAwMCBuCjAwMDAyNTE3NzQgMDAwMDAgbgowMDAwMjUxODAxIDAwMDAwIG4KMDAwMDYzNjI0MCAwMDAwMCBuCjAwMDA2MzYyNjcgMDAwMDAgbgowMDAwNjM4NjgwIDAwMDAwIG4KMDAwMDYzODcwNSAwMDAwMCBuCjAwMDExODUyMjkgMDAwMDAgbgowMDAxMTg1MjU2IDAwMDAwIG4KMDAwMTE4NTYyMiAwMDAwMCBuCjAwMDExODU2NDQgMDAwMDAgbgowMDAxNDI5MzEzIDAwMDAwIG4KMDAwMTQyOTM0MCAwMDAwMCBuCjAwMDE0Mjk4MzMgMDAwMDAgbgowMDAxNDI5ODU3IDAwMDAwIG4KMDAwMTY5MTQzOSAwMDAwMCBuCjAwMDE2OTE0NjYgMDAwMDAgbgowMDAxNjkxOTU3IDAwMDAwIG4KMDAwMTY5MTk4MSAwMDAwMCBuCjAwMDE2OTIyODUgMDAwMDAgbgowMDAxNjkyMzkxIDAwMDAwIG4KMDAwMTY5MjQxNCAwMDAwMCBuCjAwMDE2OTI1OTcgMDAwMDAgbgowMDAxNjkyNjc1IDAwMDAwIG4KdHJhaWxlcgo8PCAvSUQgWyAoc29tZSkgKGlkKSBdCiAgIC9Sb290IDY2NiAwIFIKICAgL1NpemUgNjY3Cj4+CnN0YXJ0eHJlZgoxNjkyNzM4CiUlRU9G"
  }, 
  "form": {}, 
  "headers": {
    "Accept": "*/*", 
    "Accept-Encoding": "gzip, deflate", 
    "Content-Length": "6526416", 
    "Content-Type": "multipart/form-data; boundary=02a0d6e8307afe718ae618fabe707af9", 
    "Host": "httpbin.org", 
    "User-Agent": "python-requests/2.28.1", 
    "X-Amzn-Trace-Id": "Root=1-64001aac-5d68991324ab8df43df800fe"
  }, 
  "json": null, 
  "origin": "106.202.211.62", 
  "url": "https://httpbin.org/post"
}

Uploading files in Python to the Amazon S3 storage service is also a common use case. If your application uses Amazon S3 to store and access files, you can use the code snippet below to upload files to your bucket. Save the code snippet in a file named s3-test.py in a new folder named s3 and run the command python3 s3-test.py in a new terminal window inside the s3 folder:

import boto3

# create a new Amazon S3 client
s3 = boto3.client('s3')

# store the name of your S3 bucket (replace with your S3 bucket name)
bucket_name = 'my-test-bucket-kh'

# create reference to the file location
target_file = '../samples/sample.pdf'

# store the name of the target file in S3
s3_target_file_name = 'sample.pdf'

# upload the file
s3.upload_file(target_file, bucket_name, s3_target_file_name)

# verify that the file was uploaded
response = s3.list_objects(Bucket=bucket_name)
for content in response.get('Contents', []):
    print(content.get('Key'))

Before running this code snippet, you need to ensure that you have the AWS CLI installed and configured. You can follow this article to set it up on your system. You'll also need to install the boto3 Python client for AWS by running the following command:

pip3 install boto3

Uploading files in Python is a common task that Python developers come across in their daily work. While the requests library is a versatile solution for your file uploading needs, it's good to be aware of some alternatives. In this article, you learned about some of the popular ways of uploading files in Python. You also saw how you could use the Amazon S3 client boto3 to upload files to your S3 bucket directly from your Python script.