Use moto to test AWS service in Python

When I develop kubernetes-service-dns-exporter, I'm using boto3 to communicate with AWS for route53 operation. I find out Moto when trying to research on how to mock aws services.

Moto is a fantastic library to help me to mock aws servcie API and test my code logic without cost and it's very easy to use, it has implement most of the common AWS api endpoint. moto could be simply installed by pip install moto.

mock aws client

use python decorator @mock_xxx , xxx is the service name you want to mock, e.g @mock_s3 will mock all s3 related operation in boto to use moto, see below example from moto

@mock_s3
def test_my_model_save():
    # Create Bucket so that test can run
    conn = boto3.resource('s3', region_name='us-east-1')
    conn.create_bucket(Bucket='mybucket')
    model_instance = MyModel('steve', 'is awesome')
    model_instance.save()
    body = conn.Object('mybucket', 'steve').get()['Body'].read().decode()

    assert body == 'is awesome'

work with pytest

we could leverage pytest's fixture feature to make sure the boto3 client does not connecting to real aws service by

@pytest.fixture(scope='function')
def aws_credentials():
    """Mocked AWS Credentials for moto."""
    os.environ['domain_name'] = 'test.com'
    os.environ['aws_access_key_id'] = 'testing'
    os.environ['aws_secret_access_key'] = 'testing'

we could also use pytest's fixture feature to generate the route53 client for the test method

@pytest.fixture(scope='function')
def route53(aws_credentials):
    with mock_route53():
        yield boto3.client('route53')

so in the test method, I do not need to use the mock_route53 decorator, I could write the test case like below

def test_route53(route53):
    assert os.getenv('domain_name') == 'test.com'
    resp = route53.create_hosted_zone(Name=os.getenv('domain_name'), 
                                      CallerReference=str(hash('xxx')))
    zone_id = resp['HostedZone']['Id']
    os.environ['hosted_zone_id'] = zone_id
    dns_name, record = route53_dns('CREATE', 'test_service', '127.0.0.2')
    assert dns_name == 'kube-test_service.test.com'
    assert record['ResponseMetadata']['HTTPStatusCode'] == 200