Using python SDK to upload and download the local file into S3 bucket

This blog post will look at using the Python boto3 library to upload the local file into S3 bucket and download the file in S3 bucket to local file.

Boto3 is the Python Applications Development Kit (SDK) for Amazon Web Services (AWS), which enables users to create software that utilizes services like Amazon S3 and Amazon EC2.

Requirements:

Requirements to upload and download the file

• You must have an Amazon Web Services account.
• The code was written for:
  • Python 3
  • AWS SDK for Python (Boto3)
• Install the AWS SDK for Python (Boto3).

Configure AWS access keys:

Through SSH client, connect to remote desktop. In the Linux terminal, run the following commands.

cd .aws

cat credentials

This file should contain lines in the following format:

[default]
aws_access_key_id = <YOUR_ACCESS_KEY_ID>
aws_secret_access_key = <YOUR_SECRET_ACCESS_KEY>

Replace the values of <YOUR_ACCESS_KEY_ID> and <YOUR_SECRET_ACCESS_KEY> by your AWS credentials.

Create a folder to store all the files:

You can create a folder through the terminal and open the VS code

mkdir blog2

cd blog2

code .

Python program to upload a local file to S3 bucket:

The python code below stores the arguments that we enter through the terminal. If the file is present in the local system, then it will upload the local file into the bucket.

import sys
import os
import boto3
import botocore

def main():
    args = sys.argv[1:]
    if len(args) < 3:
        print('Not enough parameters.\n'\
              'Proper Usage is: python s3_upload.py '\
              '<BUCKET_NAME> <OBJECT_NAME> <LOCAL_FILE_NAME>')
        sys.exit(1)

    bucket_name = args[0]
    key_name = args[1]
    local_file_name = args[2]
    print('Bucket:     ' + bucket_name)
    print('Object/Key: ' + key_name)
    print('Local file: ' + local_file_name)

    # Create an S3 Client
    s3_client = boto3.client('s3')

    if not os.path.isfile(local_file_name):
        print("Error: File Not Found!!")
        sys.exit(1)

    # Upload object
    try:
        print('Uploading object ...')
        s3_client.upload_file(local_file_name, bucket_name, key_name)
        print('Uploaded')

    except botocore.exceptions.ClientError as e:
        if e.response['Error']['Code'] == "NoSuchBucket":
            print("Error: Bucket does not exist!!")
        elif e.response['Error']['Code'] == "InvalidBucketName":
            print("Error: Invalid Bucket name!!")
        elif e.response['Error']['Code'] == "AllAccessDisabled":
            print("Error: You do not have access to the Bucket!!")
        else:
            raise

    return


# This is the standard boilerplate that calls the main() function.
if __name__ == '__main__':
    main()

When we run the command below then it displays the output whether the file is uploaded or not.

python S3_upload.py <BUCKET_NAME> <OBJECT_NAME> <LOCAL_FILE_NAME>

As you can see screenshot.PNG file which we uploaded through python SDK can be seen in S3 bucket.

Python program to Download an object from S3 bucket into local file:

The python code below stores the arguments that we enter through the terminal.

import sys
import boto3
import botocore
def main():
  args = sys.argv[1:]
  if len(args) < 3:
      print('Not enough parameters.\n'\
            'Proper Usage is: python S3_download.py '\
            '<BUCKET_NAME> <OBJECT_NAME> <LOCAL_FILE_NAME>')
      sys.exit(1)

  bucket_name = args[0]
  key_name = args[1]
  local_file_name = args[2]

  print('Bucket:     ' + bucket_name)
  print('Object/Key: ' + key_name)
  print('Local file: ' + local_file_name)
  
  # Create an S3 Client
  s3_client = boto3.client('s3')

  # Download object
  try:
      print('Downloading object ...')
      s3_client.download_file(bucket_name, key_name, local_file_name)
      print('Downloaded')

  except botocore.exceptions.ClientError as e:
      if e.response['Error']['Code'] == "404":
          print("Error: Not Found, problem with the parameters!!")
      elif e.response['Error']['Code'] == "400":
          print("Error: Bad request, problem with the bucket!!")
      else:
          raise

  return


# This is the standard boilerplate that calls the main() function.
if __name__ == '__main__':
    main()

Using the python program, you can download the object that we mention in the terminal using the following command.

python S3_download.py <BUCKET_NAME> <OBJECT_NAME> <LOCAL_FILE_NAME>

The downloaded file is store in the same path as the .py files.