A Complete Guide To Deployments With Zappa and GitHub Actions

Photo by Victoire Joncheray on Unsplash

Trying to keep up with the latest AI research and looking for a more streamlined approach? Textlayer AI is the first purpose-built research platform for developers the gives you free access to personalized recommendations, easy-to-read summaries, and full chat with implementation support.

Deploying serverless applications can be challenging, especially when it comes to integrating continuous deployment workflows. This comprehensive guide will walk you through the process of deploying Zappa projects with GitHub Actions, ensuring a smooth and efficient deployment pipeline. Whether you’re using conventional CLI deployments or Docker-based ones, this guide covers it all.

We will explore how to set up GitHub Actions for both types of deployments and show you how to test your Docker-based deployments locally to replicate the AWS Lambda environment before pushing them to production. This guide assumes that you already have a Zappa project set up and a GitHub repository. If you don’t, you can follow the guides provided below to get started:

• Setting up a Zappa project
• Creating a GitHub repository

By the end of this guide, you’ll have a solid understanding of how to automate your Zappa project deployments using GitHub Actions, and have a seamless CI/CD workflow.

Tools

We’ll be using the following tools and resources in this tutorial

• Zappa: Deploy Python apps on AWS Lambda and API Gateway.
• GitHub Actions: Automate your CI/CD pipelines.
• Docker: Develop and run applications in containers.
• AWS CLI: Manage AWS services from the command line.
• AWS ECR: Store and manage Docker container images.

References and Resources

For additional help and information, you can refer to these resources:

• Serverless Python deployments with GitHub Actions and Zappa: A detailed guide on using GitHub Actions with Zappa.
• Zappa Serverless Docker: Information on deploying Zappa projects using Docker.
• Sample Repository: A sample GitHub repository demonstrating Zappa deployments.
• Zappa Documentation: Official Zappa documentation for more detailed information.

These resources will provide additional context and examples to help you through the deployment process.

Steps

This guide will take you through the following steps to set up your Zappa deployment with GitHub Actions:

1. Setting up GitHub Actions for Zappa deployments
2. Configuring your Dockerfile
3. Testing your Docker deployment locally
4. Deploying your Docker image to AWS ECR
5. Deploying your Docker image to AWS Lambda with Zappa

Each step will be detailed to ensure you can follow along easily and set up your CI/CD pipeline efficiently.

1. Setting up GitHub Actions

To set up GitHub Actions for your Zappa deployments, follow these steps:

1. Navigate to your GitHub repository.
2. Click on the “Actions” tab.
3. Click on the “New workflow” button.
4. Select “Python application” as the template.
5. Replace the contents of the workflow file with the following:

name: CI/CD

on:
  push:
    branches:
      - main  # Change this to your main branch name
    paths:
      - app/**
      - requirements.txt
      - zappa_settings.json
      - application.py

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v4

    - name: Set up Python 3.9
      uses: actions/setup-python@v5
      with:
        python-version: 3.9
        cache: "pip"

    - name: Set up the virtual environment and install dependencies
      run: |
        python -m venv env
        source env/bin/activate
        python -m pip install --upgrade pip
        pip install -r requirements.txt

    - name: Deploy to AWS with Zappa
      env:
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
        AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      run: |
        # Install the CLI
        pip install awscli
        
        # Configure the CLI
        aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID
        aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY
        
        # Activate the virtual environment
        source env/bin/activate
        
        # Change "production" to your Zappa environment
        zappa update production

This workflow will deploy your Zappa project whenever changes are pushed to the main branch. Make sure to set up AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY as secrets in your GitHub repository. If you haven't deployed your Zappa project before, run zappa deploy production to initialize it.

You can also add a .github/workflows directory to your repository and place the workflow file there.

2. Configuring your Dockerfile

To create a Docker image compatible with AWS Lambda, you need to set up a Dockerfile in the root of your project. Here’s how you can do it:

1. Create a Dockerfile in the root directory of your project with the following content:

FROM public.ecr.aws/lambda/python:3.9

ARG FUNCTION_DIR="/var/task/"

COPY . ${FUNCTION_DIR}
WORKDIR ${FUNCTION_DIR}

# Install dependencies
RUN pip install -r requirements.txt

# Grab the Zappa handler.py and put it in the working directory
RUN ZAPPA_HANDLER_PATH=$( \
    python -c "from zappa import handler; print(handler.__file__)" \
    ) \
    && cp $ZAPPA_HANDLER_PATH ${FUNCTION_DIR}/handler.py

CMD ["handler.lambda_handler"]

1. Ensure you have a requirements.txt file in the root of your project, listing all the dependencies needed for your project.

This Dockerfile sets up a Lambda-compatible image that includes your application code and dependencies. It also ensures that the handler.py from Zappa is correctly placed.

Important: The Docker build process needs to be done in two steps:

1. Build the image with zappa save-python-settings-file your_zappa_environment_name.
2. Build the Docker image with docker build -t your_image_name:latest ..

These steps will prepare your application to run in a Docker container that mimics the AWS Lambda environment.

Testing your Docker deployment locally

Once you have built your Docker image, you can test it locally to ensure it functions as expected in a Lambda-like environment. Follow these steps:

Run the Docker container locally:

docker run -p 9000:8080 your_image_name:latest 

This command will start a local instance of your Docker container, exposing port 8080 inside the container to port 9000 on your local machine.

Test your Lambda function:

Send a POST request to your local Docker container to invoke the Lambda function. You can use tools like curl or Postman for this.

Using curl:

curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'

Make sure you have a valid payload to send to your function. This payload should match the expected input format of your Lambda function.

Verify the response:

Check the response from your Lambda function to ensure it is working as expected. If there are any issues, you can debug and fix them locally before deploying the image to AWS.

Testing your Docker deployment locally allows you to replicate the AWS Lambda environment and ensure that your function behaves correctly, saving time and preventing potential issues in production.

Deploying your Docker image to AWS ECR

To deploy your Docker image to AWS Elastic Container Registry (ECR), follow these steps:

Install the AWS CLI:

Ensure you have the AWS CLI installed on your machine. You can download and install it from here.

Log in to your ECR repository:

Use the following command to log in to your ECR repository. Replace your_region and your_account_id with your AWS region and account ID.

aws ecr get-login-password --region your_region | docker login --username AWS --password-stdin your_account_id.dkr.ecr.your_region.amazonaws.com

Tag your Docker image:

Tag your Docker image with the ECR repository URL. Replace your_image_name, your_region, and your_account_id with your image name, AWS region, and account ID.

docker tag your_image_name:latest your_account_id.dkr.ecr.your_region.amazonaws.com/your_image_name:latest

Push your Docker image to ECR:

Push the tagged Docker image to your ECR repository.

docker push your_account_id.dkr.ecr.your_region.amazonaws.com/your_image_name:latest

These steps will upload your Docker image to AWS ECR, making it available for deployment to AWS Lambda.

Deploying your Docker image to AWS Lambda with Zappa

To deploy your Docker image to AWS Lambda using Zappa, follow these steps:

Navigate to your Zappa project directory:

Ensure you are in the root directory of your Zappa project.

Add the following action to your GitHub Actions workflow file:

This action will build your Docker image, push it to ECR, and update your Zappa environment with the new image.

name: CI/CD-ECR

on:
  push:
    branches:
      - main  # Change this to your main branch name
    paths:
      - app/**
      - requirements.txt
      - zappa_settings.json
      - application.py

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v4

    - name: Set up Python 3.9
      uses: actions/setup-python@v5
      with:
        python-version: 3.9
        cache: "pip"

    - name: Set up the virtual environment and install dependencies
      run: |
        python -m venv env
        source env/bin/activate
        python -m pip install --upgrade pip
        pip install -r requirements.txt

    - name: Login to Amazon ECR
      env:
        AWS_REGION: ${{ secrets.AWS_REGION }}
        AWS_ACCOUNT_ID: ${{ secrets.AWS_ACCOUNT_ID }}
      run: |
        aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com

    - name: Build the Docker image
      env:
        AWS_REGION: ${{ secrets.AWS_REGION }}
        AWS_ACCOUNT_ID: ${{ secrets.AWS_ACCOUNT_ID }}
        AWS_REPOSITORY_NAME: ${{ secrets.AWS_REPOSITORY_NAME }}
        GITHUB_SHA: ${{ github.sha }}
      run: |
        docker build -t $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$AWS_REPOSITORY_NAME:$GITHUB_SHA .

    - name: Push the Docker image to Amazon ECR
      env:
        AWS_REGION: ${{ secrets.AWS_REGION }}
        AWS_ACCOUNT_ID: ${{ secrets.AWS_ACCOUNT_ID }}
        AWS_REPOSITORY_NAME: ${{ secrets.AWS_REPOSITORY_NAME }}
        GITHUB_SHA: ${{ github.sha }}
      run: |
        docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$AWS_REPOSITORY_NAME:$GITHUB_SHA

    - name: Deploy to AWS with Zappa
      env:
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
        AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        AWS_REGION: ${{ secrets.AWS_REGION }}
      run: |
        # Activate the virtual environment
        source env/bin/activate
        
        # Install Zappa and AWS CLI in the virtual environment
        pip install awscli zappa
        
        # Configure the CLI
        aws configure set aws_access_key_id $AWS_ACCESS_KEY_ID
        aws configure set aws_secret_access_key $AWS_SECRET_ACCESS_KEY
        aws configure set region $AWS_REGION
        
        # Deploy with Zappa
        zappa update production

Set up required secrets:

Make sure you have the following secrets set up in your GitHub repository:

• AWS_REGION
• AWS_ACCOUNT_ID
• AWS_REPOSITORY_NAME
• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY

Add the workflow file to your repository:

Place the workflow file in the .github/workflows directory of your repository.

With this setup, every push to the main branch will trigger the workflow, build your Docker image, push it to ECR, and update your Zappa environment with the new image.

Conclusion

In this guide, we have covered the entire process of deploying Zappa projects using GitHub Actions. We explored setting up GitHub Actions for both conventional CLI deployments and Docker-based deployments. We also looked at how to test your Docker-based deployments locally before pushing them to production, replicating the AWS Lambda environment.

By following these steps, you can automate your Zappa project deployments, ensuring a seamless and efficient CI/CD pipeline.

Hopefully this guide was helpful for you, and if it was please let me know by sharing and adding some applause.

Happy coding!