AWS SQS using Boto3 – UW-Milwaukee Cloud Computing

What is AWS SQS?

Aws SQS stands for Simple Queue Service. This Service provides the customer with a fully managed queue for storing messages as they travel between different applications or microservices. And enables you to decouple and scale distributed systems, and serverless applications.

This service is very popular among top brands like BMW, oyster, and many others.

AWS SQS Architecture

Let try to understand the SQS architecture with help of diagrams. So in the first diagram there are three distributed system components that are used to perform different tasks. The other diagram represents the multiple SQS servers which are filled with queues that contains messages.

Suppose that component 1 after completing the task assigned to it sends a message(consider the name of the message to ‘a’) to the queue.

This message is not only stored in one server but stored across various SQS servers, The component 2 is notified that there is a message in the queue and component 2 can pull the message a from the queue and now can start working on the task assigned to it and this same point of time the visibility timeout period start(Deeper Understanding). The point to remember is that as component 1 is sending a message to the queue it becomes a producer of the message and component 2 becomes the consumer of the message after component 2 pulls the message from

the queue it can process the message and delete it from the queue the same thing repeats again,

The component 2 finishes its task and sends a message to the queue which is then pulled by the component3.

Visibility time out period:

Whenever a consumer receives and processes a message from the queue the message is not deleted and it remains in the queue, this is because there is no guarantee the consumer has received the message. For say there could be any connectivity issue or an issue in the consumer’s application.

So that’s why only the consumer must delete the message from the queue after receiving and processing it. So , it is a time period during which amazon SQS prevents other consumers from receiving and processing the same message let’s say if one consumer is accessing a particular message no other consumer can access the same message during the visibility timeout period. By default visibility timeout period for a message is 30 seconds and the maximum is 12 hours,but the minimum is 0. Consider a scenario in which the consumer sends the received message request and the message is not returned at this time the consumer can send the receive message request again and if the message is not returned after multiple failed attempt then the visibility timeout stops and other consumers can now access the message.

Benefits of SQS :

SQS is a cloud-based service that can be made available to any software, application, or service. While here are few Benefits we get for using AWS SQS service.

Ensure Safety of Messages(Durability)
Reliably Deliver Messages
Keep Sensitive Data Secure
Granular Scalability

Types of SQS

Basically, there are two types of queue that are offered to the users by the AWS Cloud provider. They are as follow.

Standard Queue: > Standard Queues are the default option for Amazon SQS, allowing you to work with an unlimited number of API calls per second per API action. >It offers best-effort ordering, which guarantees that messages are generally delivered in the same order in which they are sent, but it does not guarantee delivery.
FIFO Queue: >This queue the order of messages is preserved. > FIFO Queues have a transaction rate of 300 transactions every second but all of the capabilities of standard queues.

There are many operations that can be done on AWS SQS. We will be using AWS SDK for Python(Boto3).Boto3 makes it simple to hook up your Python application, library, or script to AWS services such as Amazon S3, Amazon EC2, and Amazon DynamoDB.

Whenever we are working with boto3 make sure that it is installed, Here is the command to install boto3.

Pip install boto3

To run operations on SQS using Python via boto3 SDK. We need to connect to AWS using Acccess key and Secret ID, which you can get when creating a IAM user.

In order to create go to IAM page, then click on users which is on left hand of the IAM page. This is how the page looks.

Connecting to AWS account

Now, with the help of resource method in boto3 package .We’ll be able to connect to client AWS account. Here is the synatx.

Import boto3
c = boto3.resource('sqs',aws_access_key_id='Your Access key',
    aws_secret_access_key='<Your Secret key>',region_name='<Region of your user>').

Creating a queue

This function creates a new standard or FIFO queue. In the request, you can include one or more attributes. Create Queue returns the queue URL for an existing queue if you provide the name of an existing queue as well as the exact names and values of all the queue’s attributes.

res = c.create_queue(
 
      QueueName='MyQueue'
 
)

Python program to perform some operations on SQS queue.

Here, In this program we are going to send,read or delete the messages from sqs queue of aws.

import boto3


#AWS connection via boto3 'CLIENT'
c = boto3.resource('sqs',aws_access_key_id='AKIA5DHL62VV72EWSKGL',
    aws_secret_access_key='llUj9oFP04x7CTZ1wYpoTncND05E+w10qxJH2YOX',region_name='us-west-1')

accthttp='your sqs queue url'

###INPUT THE IAM CREDENTIALS WITH PERMISSIONS TO THE AWS ACCOUNT AND RESOURCE YOU'RE TRYING TO ACCESS.


#
#Creating sqs queue in client connected account
# res = c.create_queue(
#      QueueName='MyQueue'
# )
# # print(response)

print("##########################################")
print("AVAILABLE QUEUES : URLS")

#The response is NOT a resource, but gives you a message ID and MD5
for queue in c.queues.all():
    print(queue.url)
#
print("##########################################")
#SQS opeations
while True:



    print("Would You Like to : 1) Send a Message 2) Receive a Message 3:) Exit Program ?")
    store = input()
    #sending messages to queue created
    if int(store) == 1:
        print("Please Input the NAME of the QUEUE to send a message to:")
        store2 = input()
        url = accthttp+str(store2)
        print("Prease Input the Message")
        store3 = input()
        client.Queue(url=url).send_message(
            MessageBody=store3)
    # Operations to read and delete message from queues
    if int(store) == 2:
        print("Please Input the NAME of the QUEUE read a message from:")
        store2 = input()
        url = accthttp + str(store2)
        # receipt = c.Queue(url=url).receive_messages()
        # receipt1 = c.Queue(url=url).receive_messages()
        while True:
            print("Would You Like to Try the Queue? (1 or 2)")
            if int(input())== 1:

                receipt = c.Queue(url=url).receive_messages()

        # print(receipt)
                for message in receipt:
                    print(message.body)
                    print(message)
                    message.delete(QueueUrl=url, ReceiptHandle=message.receipt_handle)
                    print("this message has been deleted.")

            else:
                print("Cheers")
                break