Snapshot Testing in AWS CDK Python

Mon, Jun 19, 2023
5-minute read

Warning: This post is over 365 days old. The information may be out of date.

NOTE: Since writing this post, I’ve switched from pytest-snapshot to syrupy. Check out my new post for more information.

At Defiance Digital, we use the AWS CDK for almost everything. Generally we use TypeScript because it’s the original language for the CDK, everything using JSII transpiles back to TypeScript, and it has the most compability with the CDK. However, we have a few projects that use Python, and on those I’ve really been missing Jest snapshot testing.

For most CDK projects snapshot tests are the perfect way to make sure that your stacks don’t have unintended changes. We include fine-grained tests as the stacks are deployed and mature. However, every stack always starts with a snapshot test. So how do we do that in Python?

The code

First, let’s have a look at the dev group of our pyproject.toml file and the overall test suite for this stack. Note that we are using Poetry to manage our environment and dependencies via projen.

pyproject.toml

  [tool.poetry.dev-dependencies]
  bandit = "*"
  black = "*"
  flake8 = "*"
  isort = "*"
  mypy = "*"
  projen = "*"
  pycodestyle = "*"
  pydocstyle = "*"
  pytest-snapshot = "*" # This is the key dependency
  pytest = "*"

The pytest-snapshot dependency is what allows us to take snapshot tests. We also need to have pytest available as the testing framework.

test_customer_stack.py

from json import dumps
import pytest
from aws_cdk import App
from aws_cdk.assertions import Template

from customer_python_service.stacks.ecr_repo_stack import EcrRepoStack


@pytest.fixture(scope="module")
def template():
    app = App()
    stack = EcrRepoStack(app, "my-stack-test", repository_name="defiance-customer-python-service")
    template = Template.from_stack(stack)
    yield template


def test_ecr_repo_found(template):
    template.resource_count_is("AWS::ECR::Repository", 1)


def test_snapshot(template, snapshot):
    snapshot.assert_match(dumps(template.to_json()), "EcrRepoStack.json")

What’s happening here, exactly?

First, we import all our dependencies, which are just Python standard library, pytest, aws-cdk, and our CDK stack. Next, we define a Pytest fixture that can be used in our test definitions. This fixture is a Template object from the aws-cdk library, which is a representation of the CloudFormation template that will be generated by our stack. We can use this same fixture for both fine-grained tests and snapshot tests.

The test_ecr_repo_found function is a fine-grained test. It uses the Template object to assert that there is exactly one AWS::ECR::Repository resource in the template. This is a simple test that ensures that our stack is generating the correct resources. It’s here as an example of a very simple fine-grained test. Generally as the stack mature we will add more fine-grained tests to ensure that the stack is generating the correct resources and that they are configured correctly.

Finally, the test_snapshot function takes our template fixture and a snapshot argument from pytest-snapshot. It uses the snapshot fixture to assert that the template matches the snapshot. If the snapshot does not exist, it will be created. If it does exist, it will be compared to the current template. If the template has changed, the test will fail. If the template has not changed, the test will pass.

Running the tests

As I mentioned earlier, we’re using projen to manage our project. Projen has a built-in test command that will run all the tests in the project. At Defiance, we use that to run our snapshot tests, both manually and in CI/CD pipelines.

Here’s what our .projenrc.py snippet looks like to patch our test task to use Poetry:

test_task = project.tasks.try_find("test")
if test_task:
    test_task.reset("poetry run pytest tests/", receive_args=True)

Now we can run npx projen test to run our tests.

If you aren’t using projen, you can simply run poetry run pytest tests/ to run the tests, assuming they are in a ./tests folder. If you aren’t using Poetry, you can run pytest tests/ to run the tests.

This only works as long as the snapshot hasn’t updated. At some point in the lifecycle of this stack, the snapshot will update as features are added, so we need a way to allow that to happen:

project.add_task(name="test:update", exec="poetry run pytest --snapshot-update tests/")

Now we can run npx projen test:update to update our test snapshots.

If you aren’t using projen, you can simply run poetry run pytest --snapshot-update tests/ to update the snapshots. If you aren’t using Poetry, you can run pytest --snapshot-update tests/ to update the snapshots.

Note that the snapshot test will fail the first time you run this command. After that, your tests should pass. If you’re running your tests in a CI/CD pipeline be sure to run the update command first locally, then the test command.

Conclusion

Snapshot testing is a great way to ensure that your CDK stacks don’t have unintended changes. It’s a great way to get started with testing your CDK stacks and ensure that your stacks don’t have unintended changes as they mature. I missed this feature in our Python projects but with this pattern, we can have the same functionality in Python as we do in TypeScript.

At Defiance Digital, our mission is to empower startups and SMBs to achieve their full potential by delivering reliable, secure, and scalable end-to-end managed services tailored to their unique needs. We do this by providing personalized attention and exceptional results through direct access to elite cloud engineers who embrace our “customers as co-workers” ethos.

Please feel free to reach out with questions, comments, or suggested improvements at either michael.gray@defiance.ai or mike@graywind.org.

aws cdk python testing aws cdk python testing