Using localstack to test SQS and SNS locally

Summary:

Sometimes it’s hard to test locally a microservice architecture with messages flying around. I’ve found some alternatives to that, GoAWS and Localstack, and in this post I’ll cover how I manage to run localstack to mock my infrastructure.

Setup:

Before going into action, make sure you have installed python (I’m currently using Python 3.6.1), pip and aws-cli.

To set up localstack it’s pretty easy:

pip install localstack

Before we start make sure you have set fake aws credentials to not mess with production enviroment.

~/.aws/credentials

AWS_ACCESS_KEY_ID=foo
AWS_SECRET_ACCESS_KEY=bar

Starting Localstack:

Since localstack mocks all kind of services from aws, it’s possible to choose which one we would like to run.

$ export SERVICES=sqs,sns

To start localstack just :

$ localstack start

It’s possible to use with docker too:

$ localstack start --docker

Using aws-cli:

Create Topic:

$ aws --endpoint-url=http://localhost:4575 sns create-topic --name my_topic

List Topic:

$ aws --endpoint-url=http://localhost:4575 sns list-topics

Create Queue:

$ aws --endpoint-url=http://localhost:4576 sqs create-queue --queue-name my_queue

List Queue:

$ aws --endpoint-url=http://localhost:4576 sqs list-queues

$ aws --endpoint-url=http://localhost:4575 sns subscribe --topic-arn arn:aws:sns:us-east-1:123456789012:my_topic --protocol sqs --notification-endpoint arn:aws:sns:us-east-1:123456789012:my_queue

List subscriptions:

$ aws --endpoint-url=http://localhost:4575 sns list-subscriptions

After creating our infra we need to start sending messages so that our service can receive and use them. For that we are going to publish the message to the topic as followed:

$ aws --endpoint-url=http://localhost:4575 sns publish --topic-arn arn:aws:sns:us-east-1:123456789012:my_topic --message "Hi"

If you want it’s possible to send the content of a file too:

$ aws --endpoint-url=http://localhost:4575 sns publish --topic-arn arn:aws:sns:us-east-1:123456789012:my_topic --message file://file.json

Creating Topics and Queues automatically:

Since it’s a lot of hardwork to create all of those queues and topics by hand, and Localstack does not have an option to start with a config file(GoAws let you use a .yaml to set things up), I’ve managed to create a Python script to create them from a .yaml file. You can see the example from the config file here.

And then just:

$ ./create_sqs_sns.py my-queues-and-topics.yaml