Mocking AWS services with LocalStack

4 minutes read

When building cloud applications with AWS, we go through many vendor specific services such as S3, SQS, KMS and DynamoDB as well as many others, and for those to work properly we need credentials, IAM permissions and such, so it basically entails using your production environment or even building another one from the ground up, which sometimes is neither cheap or easy, one might lack the skills to do such thing or simply doesn’t have resources to do this on their own, not to mention that competing the same services and objects with others might be a nuisance (think of shared SNS topics or SQS queues, for example).

What is LocalStack about?

LocalStack is an open-source AWS mocking tool written in Python, it provides an API akin to the real AWS cloud environment through its own HTTP endpoint. You can install LocalStack through python’s pip package manager, but for sake of simplicity we will use Docker to make things much easier. If you’d like to know more, take a look at the project GitHub page

A LocalStack Docker Compose setup might look like this:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

version: '3.3'

services:

  localstack:
    image: localstack/localstack:0.12.11
    ports:
      - 4566:4566
    environment:
      - SERVICES=s3,sqs,sns             #list of services to be mocked
      - DEFAULT_REGION=sa-east-1        #AWS region
      - AWS_ACCESS_KEY_ID=hugoltsp      #access key for authentication
      - AWS_SECRET_ACCESS_KEY=hugoltsp  #secret key for authentication
      - DATA_DIR=/tmp/localstack/data   #container directory. enables persistence
    volumes:
      - ./setup/init.sh:/docker-entrypoint-initaws.d/setup.sh #initialization code. more on that later :)
      - ./data:/tmp/localstack/data #simple volume so you don't lose your data
    networks:
      - local
      
networks:
    local:

LocalStack AWS CLI

For this example we’ll use awscli-local which is a simple wrapper that has the same commands present in the official aws-cli, the difference is that it points to a local endpoint on every command out of the box so no need to override a command’s default URL every time with the --endpoint-url. As with LocalStack, you can install it on your machine via pip:

pip install awscli-local

Alternatively you can use a CLI profile in your installed aws-cli to issue commands against LocalStack, or even create a shell script with a dockerized version of the official AWS CLI image pointing to your LocalStack host by default, for instance:

#!/bin/bash

if [ $# -eq 0 ]
  then
    cmd='help'
  else
    cmd=$@
fi

docker run -e AWS_ACCESS_KEY_ID='hugoltsp' \
 -e AWS_SECRET_ACCESS_KEY='hugoltsp' \
 -e AWS_DEFAULT_OUTPUT='json' \
 -e AWS_DEFAULT_REGION='sa-east-1' \
 --network localstackdemo_local \
 --rm -it amazon/aws-cli:2.2.5 \
 --endpoint-url=http://localstack:4566 $cmd

Initialization Code

As mentioned earlier, it is possible to add initialization commands such as S3 buckets creation and others to your instance through the /docker-entrypoint-initaws.d/setup.sh entry, that’s possible because LocalStack’s default Docker image comes bundled with awscli-local. As soon as our chosen services are ready, it will execute any commands present on this file sequentially.

For example, the following code creates a few SQS queues, a single S3 bucket, a couple of SNS topics and their subscriptions:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54

# SQS

awslocal sqs create-queue \
--queue-name invoice-notification-mobile-dlq \
--region sa-east-1

awslocal sqs create-queue \
--queue-name invoice-notification-mobile \
--region sa-east-1 \
--attributes '{
  "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:sa-east-1:000000000000:invoice-notification-mobile-dlq\",\"maxReceiveCount\":\"2\"}",
  "MessageRetentionPeriod": "259200",
  "VisibilityTimeout": "90"
}'

awslocal sqs create-queue \
--queue-name invoice-notification-email-dlq \
--region sa-east-1

awslocal sqs create-queue \
--queue-name invoice-notification-email \
--region sa-east-1 \
--attributes '{
  "RedrivePolicy": "{\"deadLetterTargetArn\":\"arn:aws:sqs:sa-east-1:000000000000:invoice-notification-email-dlq\",\"maxReceiveCount\":\"2\"}",
  "MessageRetentionPeriod": "259200",
  "VisibilityTimeout": "90"
}'

# SNS

awslocal sns create-topic \
--name invoice-notification

awslocal sns subscribe \
--topic-arn arn:aws:sns:sa-east-1:000000000000:invoice-notification \
--protocol sqs \
--notification-endpoint http://127.0.0.1:4566/000000000000/invoice-notification-mobile \
--attributes '{
"FilterPolicy": "{\"origin\": [\"MOBILE\"]}"
}'

awslocal sns subscribe \
--topic-arn arn:aws:sns:sa-east-1:000000000000:invoice-notification \
--protocol sqs \
--notification-endpoint http://127.0.0.1:4566/000000000000/invoice-notification-email \
--attributes '{
"FilterPolicy": "{\"origin\": [\"EMAIL\"]}"
}'


# S3

awslocal s3 mb s3://invoices \
--region sa-east-1

More