Monkey See, Django Do - Mailchimp Automation in Django

Mailchimp is a marketing tool that lets your marketing department have autonomy to control their own marketing emails. As developers, we can also use the flexible API for custom integrations. When we combine the two, we get a powerful and flexible tool that allows for complex marketing campaigns that are specific to product needs.

A key Mailchimp feature is lists - these are a collection of customers that receive your marketing campaigns. Each customer in the list can have a number of properties, called merge fields that use to assign additional data to each customer. These properties can then be rendered in emails, or used to segment the customers to send campaigns to a specific subset of our list.

The common use case for Mailchimp is having one list for all of your customers, using segments to send specific campaigns. We can specify a segment that has a date of birth in august, or everybody called John to target them specifically. However, when you would like to segment your customers in a more complex manor - such as customers who behave in a particular way on your application the previous day, mailchimp leaves you stuck.

We could have merge field that indicates whether a customer is eligible for inclusion in a segment, however we are unable to segment based on relative dates such as yesterday. To solve this, we generate our own list and the integrated API to populate with the customers we want. We all we need to do in Mailchimp is set up an automated campaign mail to send out daily to the automatically populated list.

Setting up

You’ll first need a Mailchimp account, and API key (which can be generated in the user settings for your application). You’ll then need to create the list on mailchimp, use the GUI on the site as its much easier than the API. Leave the merge fields empty for now, we will come back to them later.

We connect to the api using the python package mailchimp3. For this example, we’ll assume we have a customer and activity models - where customers can have many activities. Firstly we need to configure mailchimp3, to generate the api client we write the following:

from mailchimp3 import MailChimp
from django.conf import settings

client = MailChimp(MAILCHIMP_USERNAME, MAILCHIMP_API_KEY)

We can then check out what lists we have by running:

client.lists.all(get_all=True, fields="lists.name,lists.id")

This will return the names and IDs for all of the lists our account has. Now take note of the list ID you want to dynamically update - we’ll refer to this list ID as MAILCHIMP_LIST_ID.

Creating the extract

The first step is to create an extract which contains all the customers and relevant merge fields for the Mailchimp list. This is very business specific and can be as complex as you like. We use a simple example that returns all the customers who had an activity longer than 5 minutes yesterday:

from app.models import Activity
from django.utils.timezone import timedelta, now

def generate_extract(duration):

    kwargs = {
        'activity_date': now() - timedelta(days=1)
        'activity_duration': 300
    }
    return Activity.objects.filter(**kwargs) \
        .select_related('customer') \
        .distinct('email')

Now have a function that gets us our list of members, we need to transform them into a format that Mailchimp is expecting. The Mailchimp API lets us chose between making a request for each member we want to add, or to batch the requests into a single request. We choose the later implementation as we will be adding many customers to the list at a single time.

Sending to Mailchimp

The batch endpoint expects a list of operations that are in the mailchimp API format. For adding users to a list that is a POST request to /lists/MAILCHIMP_LIST_ID/members, with the body of each request being JSON in the following format:

{
    'status': 'subscribed',
    'email_address': 'xxx@gmail.com',
    'merge_fields': {
        field_1: xxx
        ...
    }
}

Where we can pass as many or as few merge fields as we like.

Given this, we need to create a function that takes a single customer model and returns an dict in this format with the merge_fields key containing an object with all the merge fields we want to include for our list! We can sends different types of data here, strings, numbers and dates for example.

def customer_to_mailchimp_member(customer):

    return {
        'status': 'subscribed',
        'email_address': 'xxx@gmail.com',
        'merge_fields': {
            FIRSTNAME: customer.first_name,
            LASTNAME: customer.last_name,
            DOB: customer.date_of_birth
        }
    }

Now we have this transformation function, we can apply it to every customer to create a list of members ready to upload.

Configuring the list

The next step is configure the mailchimp list to match all of the merge fields we want to include with our users. Mailchimp provides an easy to use interface to set this up, in the settings of your list under List fields and |* merge *| tags. Here you’ll need to specify the type of each merge field, and ensure the merge tag exactly matches the key in the merge_fields object from the customer transformer.

After the settings are configured, all that remains is to upload the users.

Uploading

Next we need to create a list of operations that make the POST request to our Mailchimp list endpoint. We write the following general for adding members already in a format ready to upload, given a list:

def batch_add_members_to_list(members, list_id):

    operations = [{
        'method': 'POST',
        'path': '/lists/' + list_id + '/members',
        'body': json.dumps(member)
    } for member in members]

    client.batches.create(data={'operations': operations})

Here we see that we construct a list of operations in a standard format to send to the endpoint, then we use the API client to make a single request which contains all of our operations. When Mailchimp receives this, it creates a new batch object attached to our account and returns the batch ID amongst other information about the newly created batch.

Managing batches

Once you send a batch to Mailchimp it may take some time to execute the operations contained in the request. You can track the progress of a batch using:

client.batches.get(BATCH_ID)

This will return an object that tells you the status of the batch, which is pending when it is queued, started once Mailchimp has begun executing the operations and finished once it is complete. Once the batch has finished the response object will tell us how many operations completed, and how many failed. Further, we can get the results of all the operations contained within from the response_body_url object - this is a link to a JSON array of responses for each operations request.

Once a batch is complete, you can view the members added to the list! Alternatively, if it hasn’t worked for you, you can debug using the batch management mentioned above.

---

To automatically add customers to our list, we use a webbook in our application that is called from an AWS lambda function at a specific time. The webhook then generates the customers and adds them to our Mailchip list using the method outlined above.