Why Campaigns are Emailing You

Campaigns have fundamentally changed over the last twenty years and are more people-powered than ever! While candidates still need compelling ideas, the ways supporters get involved with a campaign have totally changed. The barrier to entry has been lowered, and there is space for more people to be involved in the political process. Technology has greatly contributed to this shift, bringing candidates and issues right to a person in a way that works for them — mainly through email! This means that campaigns are spending more time talking to you.

Here at ActBlue, we are all about strengthening the horizontal relationship between campaigns and their small-dollar donors. There’s no denying that during an election cycle, our inboxes can be full of emails from different candidates and organizations encouraging us to get involved. However, behind every email there is a purpose, and it’s not always to get a donation! Many email programs are centered around cultivating a real relationship with their subscribers. Email provides a direct connection between candidates and supporters, creating an accessibility that simply wasn’t there before email programs and digital fundraising became popular. Here are some emails you might recognize!

Messaging Emails:

Running a grassroots campaign means candidates have to authentically and regularly keep supporters informed and engaged with their campaign. Messaging emails are centered around updating their supporters on what is happening on the campaign trail, a new policy rollout, big moments in the news cycle, and other important topics that subscribers might want to know more about. These emails give supporters an opportunity to learn more about a candidate and why they deserve support. And sometimes, messaging emails are about hearing from you! Many campaigns will ask supporters to share their stories, give their input via a survey, or sign a petition regarding a specific policy.

Grassroots Fundraising Emails:

Small-dollar donors are essential to the infrastructure of grassroots campaigns. Fundraising emails are used to bring more supporters into the fold, giving them a stake and a voice in important races. Campaigns can be expensive, and candidates need money to pay their staff, travel all over to meet the people they are hoping to represent, and cover many other expenses.

More and more candidates are looking to supporters to invest in their campaigns so they can build an inclusive movement that has the community support it takes to win. Here’s what Laticia, a small-dollar donor from Florida, has to say about fundraising emails:

I never gave to campaigns, but now I’m contributing regularly, albeit in small amounts. I feel connected with the candidate when I contribute. I receive their emails of gratitude and notes of campaign progress. I appreciate the honing of their ideas into policies and can witness the strength of their campaigns. By giving small amounts through ActBlue, I remain connected.

These relationships with grassroots donors also free candidates from big money and having to spend significant amounts of time courting one or two mega-donors. This means candidates can spend more time connecting with and serving constituents! Small-dollar donors have made our political landscape more representative and diverse than ever before, and this movement is only going to continue to lower the barrier of entry to politics! But don’t take it from us — check out what other grassroots donors across the country have to say about why they chip in.

Organizing Emails:

Grassroots campaigns rely on more than just small-dollar donations — they need people-powered support in every aspect of their campaign. Organizing emails let supporters know about opportunities to get involved in the campaign through events, canvassing, and phone banking. There are endless ways to support a candidate, and organizing is a great way to create a community within a campaign and meet other people who are passionate about the same issues as you!


Emails are an important way to stay informed about a candidate and the causes they care about. Every campaign email you get has a specific purpose, and that is to provide you with information and opportunities. That being said, if your inbox is overwhelming or you are no longer involved in a cause anymore, you can always unsubscribe. Campaigns want their supporters to feel excited when they get an email, and passionate supporters are the key to a healthy email list! It’s also important to note that ActBlue does not send fundraising emails (or texts) on behalf of any other candidates or groups.

Accessibility to a candidate is integral in ensuring they are fighting for you, and email is breaking down barriers between supporters and campaigns. We know how important it is to include all people in the political process and make sure everyone can be engaged and informed. So the next time you get an email from a campaign, take a moment to remember how far campaigns have come and the progress that has been made to ensure transparency and inclusion in the political process!

Email personalization: The key to engaging donors

The experts all agree: If you’re a campaign or organization, personalization is one of the most effective ways to engage one-on-one with the supporters on your email list. Personalization helps your emails stand out in a busy inbox and increases open rates, making it a popular and impactful strategy. But personalization does more than improve clicks — this strategy is a real way to make your digital communication with supporters as unique as your in-person conversation would be at an event!

Including information specific to individual recipients, like name, location, and donation history, reinforces that each and every one of your grassroots donors is a valued part of your team. Plus, small-dollar donors who are invested in your cause are more likely to volunteer, vote, and talk to friends and family! So developing deeper relationships with your supporters pays off in more ways than one — it is an important tool both for your group and the larger small-dollar donor movement.

Personalization might seem difficult if you’ve never done it before, but a couple simple tools will allow you to easily include personalized content in your emails. These strategies won’t just help you reach your fundraising goals — they will enable you to build authentic and individualized relationships with the grassroots donors who have chosen to support and advocate for your cause.

FIRST, A QUICK NOTE

Personalization combined with a smart email strategy, including fundraising emails and customized remarketing emails and receipts, is the best way to engage with your small-dollar donors. Although there are several great ways you can communicate with your supporters in our increasingly digital world, email is still an essential part of every digital fundraising program because it is accessible, affordable, and effective.

As long as you have an email address, you can talk directly to your small-dollar donors and ask them to invest in your cause. And unlike social media, where your messages are displayed for a broad audience, your emails go straight into individual inboxes. Personalizing small parts of your emails will put you on the path to developing unique and deep-rooted relationships with your supporters. Choosing to contribute to your cause is a personal decision — your emails should be personal too!

There are two main tools that are essential to email personalization: personalization tags and list segmentation, both of which are likely available to you through your email provider. Both are based on subscriber field data, or the information that supporters provide when they opt-in to your email list.

PERSONALIZATION TAGS

Personalization tags are small bits of code which automatically display personalized information. These will look slightly different depending on your email provider, but their basic structure includes the bit of personalized content you would like to include, plus a backup option if the recipient didn’t include that information when they signed up for your email list. Here’s an example from Campaign Monitor:

personalization tags

Your email provider should have a list of personalization tags for customizing different types of information, like the one above. The amount of personalization tags available to you will depend on the information you asked people to provide when they signed up for your email list. If you have already begun fundraising, you will also have opportunities for personalization based on whether a supporter has donated before or how much they have contributed in the past. If you’re new to personalization, we recommend customizing donor first names as a simple and effective first step!

Unlike more specific demographic information like age and gender, you will almost certainly have the name for everyone receiving your emails. And including a donor’s name, particularly in the subject line, will motivate them to open your email — according to Campaign Monitor, emails with personalized subject lines are 26% more likely to be opened.

To do this, simply copy and paste the personalization tag the email provider has supplied you into the subject line, and make sure to customize the backup text in case you don’t have someone’s information. The end result will look something like this:

personalized subject line

In the case of this example, the backup text could be “Only five states left, friend!”

Including names in the subject lines of your fundraising emails lets you directly address your supporters right in their inboxes. This kind of one-on-one communication motivates small-dollar donors not only to contribute, but to advocate for your cause in other ways because they are treated as an equal member of your team. Your grassroots donors are often the people most affected by your work, so building horizontal relationships with them is essential.

LIST SEGMENTATION

List segmentation refers to the practice of splitting up your email list based on subscriber field data, including location and donation history. Like personalization tags, segments of your email list can be accessed through your email provider. You can also download segmented lists of your donors and their emails in your ActBlue Dashboard!

By sending emails to specific groups of subscribers, you can tailor content to a particular set of your donors and speak directly to them in a way that is motivating and engaging. Welcome emails are a great example — campaigns and organizations send them out only to supporters who recently opted in to their email list.

welcome email

Welcome emails like the one above speak specifically to people who are new to your cause, recognizing their particular place on your team and introducing them to your story and work.

Segmenting your email list and sending your supporters information that is relevant to them is an important part of treating your donors like individuals, not ATMs. Have a group of volunteers that advocate for your campaign or organization? Send these dedicated supporters an email thanking them for their work. Notice that a particular community has stepped up for your cause? Recommend events they could attend, or send them a Tandem form that will expose them to other groups in their area. Plus, people are less likely to unsubscribe from your email list when they receive personalized emails rather than information that is over-generalized or irrelevant to them.

READY FOR THE NEXT STEP?

Although it might seem complicated at first, personalization is actually a powerful and deceivingly simple tool that ensures your emails are always unique and relevant to the small-dollar donors receiving them. Better yet, did you know ActBlue has a tool for personalizing your contribution forms? Read more about the snippet feature on our support site to take your personalization to the next level!

Spring cleaning: Spiff up your digital fundraising

From meeting monthly goals to meeting a news moment with rapid response, digital fundraising can get hectic. But with spring in full bloom, embrace the spring cleaning mood and check up on your grassroots fundraising program. Now’s the time to sharpen the digital tools that may have collected dust so your fundraising doesn’t stall in big moments. Here’s a rundown of what to tidy up right now to make your program stronger for the long haul.

Clean up the basics

First things first, double-check and update your default contribution form! Every campaign and organization on ActBlue is set up with a default form, which most supporters find when they search for you in our directory. Take this moment to refresh your form with your most up-to-date branding and a clear donation ask that’s relevant to your current messaging. Don’t forget that you can also A/B test your forms to figure out which elements on your form drive your donors to give.

Then make sure you’re nailing the details of your donors’ giving experience, from beginning to end. Take some extra time to freshen up the brandings on the form on your website’s homepage and other forms with high traffic. Beyond forms, do your *email receipts* have up-to-date brandings, too? It’s easy to forget about what happens to donors after they give, but you can finish the donation process with care by customizing the thank you text on your email receipt. And it doesn’t end there: Many of your invested supporters will want to share on social media that they gave to you. If you haven’t already, put in some time to make your social share pop, so excited donors fill their personal networks with your compelling images and messaging! You can even reuse social share settings on future forms and save time in big moments. And if you don’t think you have the digital chops to pull it off, we made this simple template for you to use.

Tidy up your email list!

Declutter your email list by keeping the engaged supporters and letting go of the inactive ones. If you’re an admin who sends regular emails to a growing list, keeping your list healthy and updated with only active subscribers gives you stronger returns on fundraising. It’s hard to let go of subscribers, but a more active list (that might be smaller) leads to more opens and donations. Metrics like open rates also affect whether your emails end up in the inbox purgatory of Gmail’s Promotions tab — or worse, spam.

How do you figure out who to unsubscribe? If you’re sending fundraising emails on a weekly basis or more, it’s generally good practice to unsubscribe people on your list who have not opened your emails for more than 90 days, which is the standard measure of inactivity. Once you’ve grouped them together, you can send them a courtesy email to let them know you’ve unsubscribed them and give them an option to resubscribe!

Give your fundraising a quick lift with these easy tools

Spring cleaning is also the perfect moment for you to implement tools that you’ve been too busy to try out but that can help grow your fundraising! Here are two simple tools you can quickly put to work to level up your program. First, turn on our Smart Recurring feature to encourage donors to support you for the long run. Smart Recurring automatically asks donors who make a one-time contribution to also start a smaller recurring donation to your campaign or organization. It helps you build a recurring donor program that will sustain your campaign for months — and our algorithms do all the math for you!

Second, add a *personal* touch to your remarketing email to encourage supporters to finish their donation! Our remarketing feature automatically sends a reminder email to supporters who didn’t completely finish their donation, but you can (and should) customize this email text. Personalizing allows you to explain how they’ll make a difference by completing their donation to your specific campaign or organization.

Lastly, and most importantly, keep your account’s security squeaky clean

Your security is our priority, and we’ve made it easier than ever for you to bolster your ActBlue account with two-factor authentication (2FA). 2FA adds an extra level of protection to your login by requiring you to enter two independent pieces of verification that confirm your identity when logging in. If you have already set up 2FA but haven’t downloaded backup codes, now’s the time to do so and store them in a secure place (like a password manager)! In a world of increasing cyber attacks, being proactive in securing your account is the most important task in cleaning up your fundraising.

With this quick scrub, you’ve refreshed the basics of your program and set up your grassroots fundraising for long-term success. If you realize there’s most dust than you thought, you can always reach us at support@actblue.com to help get the spring cleaning going!

Email Testing: What works (and what doesn’t) when deciding on fundraising ask amounts?

We’ve always used the ActBlue blog as a place to report back on our work and let everyone who’s part of the larger ActBlue community know about the huge impact their organizing work and their donations have on the progressive movement. Primarily, that means sharing stats and insights into the donations that run through our platform. But as a nonprofit, we also put a lot of resources into running our own email fundraising program, just like so many of the groups using ActBlue to fundraise. And we use our tools to do it!

During this election season, which is already proving to be busier than ever, we want to provide everyone using ActBlue with as much info and data to help guide your strategies as possible. That’s why we’re sharing two tests on the way we generate ask amounts for our ActBlue fundraising emails (and why we plan to keep sharing relevant test results throughout this year). If you have questions or thoughts on these tests, feel free to comment on this post. We hope this gets the gears turning in terms of some new tests for the other fundraising programs out there!


For our email fundraising, we have been re-evaluating how we use a donor’s previous contribution history to tailor the ask amounts we use in emails. We use ActBlue Express Lane for all of our fundraising emails, which allows ActBlue Express users to give simply by clicking a button in an email. Over the years, we’ve optimized the number of ask amounts we include in an email, as well as considered the range and distribution of the amounts we include.

 

express_box

 

We’ve also tested different sets of ask amounts for donors who tend to give different amounts. For example, the buttons for a donor who tends to give $30 once a year will be tailored to that amount and different than those a donor who regularly gives us $3 contributions will see.

To determine those asks, we typically look at our donors’ previous contribution history and use their highest previous contribution amount (HPC) to determine which set of ask amounts to use. If a donor’s HPC was below $25, they would be grouped into a segment of donors that would see the first set of buttons (top) in the image below, whereas if their HPC was between $25 and $50, they would be in the group of donors that saw the middle set of ask amounts.

ask_amounts

 

We fine-tuned these default ask amounts for each group of HPC amounts over many email tests until we saw the gains in our conversion rates plateau. This made us wonder if the conversion rate we had reached based on testing HPC amounts was the highest rate we could achieve, or whether we could reach a higher peak by starting with a different premise. With about six months left in the election cycle, we’ve been focused on maximizing recurring sign-ups rather than one-time contributions, so we started to think about alternatives to HPC that were better tailored towards action rates and recurring signups.

First we tested average previous contributions (APC) for donors who have given more than one contribution. Our hypothesis was that a donor would be more likely to take action when given ask amounts tailored towards the average of their previous contributions rather than their highest contribution amount. We tested this hypothesis by splitting our donors and assigning half of them ask amounts based on their HPC and half based on their APC. We used the same ask amounts we typically use with both of these newly sorted groups. For our list, this proved to be an effective method — we increased the action rate by 20% and the total amount raised by over 4%. We successfully raised more money by getting more contributors to give smaller amounts!

The increase in our action rate indicates our ask amounts based on HPC may have been too high. Donors may be inspired by your ask but not always able to match their own highest contribution. By lowering our ask amounts, we kept more donors engaged and decreased the likelihood they would be alienated by receiving ask amounts greater than their ability or willingness to contribute, while still raising more overall.

From this experiment, we’ve continued to build on our results. We are now working on optimizing ask amounts based on our donors’ APC amount and also tested ask amounts based on the type of contribution a donor has given previously. After all, if your goal is to maximize recurring signups, why ask for the highest previous contribution when that is likely to reflect a one-time contribution rather than a recurring amount? It’s also in line with what we know about small-dollar online giving — people want to give when they’re moved to support your cause, and that is often at a lower level multiple times throughout a cycle. That’s a useful lesson for smaller campaigns and organizations who don’t have the list size to run an involved test like this.

To improve recurring signups, we designed a test that was a bit more complex. We took all of our donors and first split them into three groups: those who had only given one kind of contribution (one-off or recurring) and those who had given both kinds of contributions. We then took the group of donors who had given both kinds of contributions and split them in two again: half were assigned ask amounts based on their HPC and half received amounts based on their highest previous recurring contribution (HPR).

ask_amounts_2

 

The impact? We raised nearly twice as much from donors in the HPR group than those in the HPC group. We found that donors were far more likely to give recurring contributions with ask amounts tailored specifically to their recurring contribution amounts.

The results of these tests reinforced a few things: First, how critical it is to question our basic assumptions about email practices to continue to build on our results. And second, how important it is to think about the kind of program you want to build. Do you really want to maximize a person’s donation once? Or are you more worried about building a robust, sustainable recurring program? Is your campaign in the news a lot, and therefore do you have lots of opportunities to engage people who have already donated? All these factors should play into how you think about your ask amounts.

If you’re an admin at a campaign or organization, you can conduct these tests too! We recommend starting by setting up a webhook to collect and analyze more information on your group’s contribution history. If you’re running a smaller campaign and building your list, you may not have enough data yet to test based on your donors’ contribution history. If that’s the case, we recommend using our Smart Recurring tool to ask one-time donors to add a smaller recurring contribution. This will help you build a robust, small-dollar program and provide more insight into how your donors respond to different ask amounts!

If you have any thoughts or questions about setting up a test like this, please reach out to stern@actblue.com!

One-stop shop: Visualize your data and create targeted lists

At ActBlue we’re committed to dreaming up new ways for you to visualize all of your data. In that spirit, our tech team just rolled out a new way for campaigns and organizations to analyze contributor data by tracking HPCs (highest previous contributions) and total contribution amounts by individual donors.

This new visualization provides amazing insight not just for technically skilled data people, but also for smaller campaigns and organizations. The chart allows users to select groups of donors based on their donation history and download their email address and contributor information. That means teams that don’t currently have the ability to segment and target donors can now instantaneously create a segmented email list!

Here’s how it works:

If you navigate to the “Donors” tab of your Dashboard (previously called the “Uniques” tab), you’ll see a graph that looks something like this:

You can also select “Total Contribution Amount” in the “Show your donors by” menu to see this:

You can toggle between the HPC and total contribution views of the graphed data at the top. This chart has a log scale for both its x and y axis. A log scale increases by an order of magnitude, rather than a fixed amount, which allows us to present a clearer picture of your data. What does that mean? If you look at the x-scale, you’ll see there’s more space between 1 contribution and 2 contributions than there is between 8 and 9.  There are far more people who gave just 1 contribution, but on a regular graph all those dots would be stacked on top of each other. So, by choosing a log scale, we’re able to show you more of your actual data.  

The x-axis shows the total number of contributions a donor has made to your campaign or organization in their lifetime on ActBlue. The y-axis shows the donor’s highest previous contribution, or the total amount that they’ve donated, depending on which view of the chart you are looking at.

Values on the y-axis are rounded. For values from $1 to $5, amounts are rounded to the nearest dollar. For $5 to $25, they are rounded to the nearest $5, and from there on, tens are rounded to tens, hundreds to hundreds, and so on.*

For values on the x-axis, contribution numbers above ten are rounded to the nearest ten. It’s unlikely that you have contributions ranging in the hundreds, but in that case they are rounded to the nearest hundred.

The graph itself gives an insight to highest previous contributions for your entire donor base, along with information on how many donations people have made. But it doesn’t end there. Click a dollar value or a number of donations to highlight a row or column. You can switch to the other visualization to see how their total volume corresponds.

More importantly, you can download a CSV of the email addresses and the corresponding contribution data from a column, row, or selected range of the graph. To select a column or row, just click on the corresponding value and click “Download selected” in the upper right hand corner. 

To select a custom set of data, you can drag your mouse to draw a box around your desired values and then download the data.

This allows you to do some pretty sophisticated targeting without needing to do the backend work. You can easily target donors based on their highest previous contribution and frequency of donating without knowing a line of SQL.

For example, we’ve seen a lot of success in our program by targeting donors based on their HPC. For low-dollar donors, we’ll ask them for $5, while higher-dollar donors are asked for $10 or $15. With this chart, you could download a list of $3 and $5 donors and then send a personalized ask to that group. If you have a big enough email list, you could try sending a $5 ask and a $7 ask, to see if donors would be willing to give just a bit more.

You can also toggle the graph to show outliers (people who fall outside the scale of the graph), if you’re interested in targeting those donors.

We hope that this new tool will allow you to get to know your donors in a more nuanced way and run an even better email program.

If you have questions about applications or how to read the graph, we’re happy to answer them. Just drop us a line at info@actblue.com


*We chose this rounding scheme to simultaneously maximize the granularity of useful information and minimize unimportant visual clutter.

Conversion rates by refcode

Our developers have been on a 2015 quest to give users greater access to data and visualizations to help you optimize your fundraising program.

We just rolled out a new feature on page dashboards: conversion rates per reference code (refcodes, for short). The two new columns on the refcode table display the number of people who landed on your form and the percentage of those that actually made a contribution.

Previously, you had to use your email mailer to check how many folks clicked each link to your contribution form and then look on your dashboard to see the number of contributions that were made. This will save you a step.

Just go to any form statistics page and you’ll see it (sample image from our testing):

FYI, we added new tracking functionality to make this work, so there isn’t historic data — only data from yesterday on will show up.

However, this will come in handy when sending emails, particularly if you are testing different versions of an email that are all linking to the same contribution form.

And, in addition to comparing conversion rates among different refcodes on the form, it should be useful to see how conversion rates differ from form to form. Know your organization-wide conversion rate, and pay attention if any form slips below that. Low conversion rates might ensue if there is a disconnect between the email content and what’s included on the form or if there is too much or too little text on the form.

It’s another tool in your toolbox that should help save you time and make better decisions. Speaking of new useful things, don’t miss the Recurring Retention charts that we just rolled out.

Questions? Just drop them in the comments or shoot us an email at info AT actblue DOT com.

When a refcode search box isn’t enough…there’s a chart!

Earlier this month we wrote about the refcode search box on the contribution form’s statistics page. It’s a neat feature that makes it easy to display metrics for refcodes that match your search terms. And now we’ve added a chart to illustrate that same page’s data. Just click on the words “Toggle refcode chart” to display it — if your form has thousands of contributions, it might take a few seconds for the graph to load. Here’s an example from an old ActBlue contribution form:

Like we mentioned in the earlier post, there are two ways to get to that page (and both require you to be logged in): if you’re already on your contribution form, go to the Contributors tab, and click on “View Form Statistics” or just insert your contribution form name into the url below:

secure.actblue.com/pages/CONTRIBUTION_FORM_NAME/statistics

When toggled, the chart displays the top-performing (maximum of 20) refcodes. And when you type in the search box, the chart will update based on your search input.

For instance, if you used one contribution form for the last five months of the election, it might look something like this:

Displayed are the 20 best-performing refcodes during that five-month timespan. Let’s say you want to display only November sends and you had a refcode system in place such that an email send in November 2014 would start like this “201411.” Here’s what that chart would look like after you entered those digits into the search box:

And our developers threw in a bonus capability: the chart also keeps track up to the seconds!

This is great for when you’re monitoring performances of refcodes just minutes after an email send. Here’s a chart displaying a single refcode within the first 15 minutes after a send.

Notice the tiny bumps — those are contributions measured on a second-by-second basis.

Our habit at ActBlue is to refresh this page again and again after sending emails, and this chart is another useful visual aid to keep track of refcode performance.

Spotlight on refcodes

For all those refcode-happy committees, this feature’s for you! Now you search for a specific refcode on the contribution form statistics page.

There are two ways to get to that page (and both require you to be logged in): if you’re already on your contribution form, go to the Contributors tab, and click on “View Form Statistics” or just insert your contribution form name into the url below:

secure.actblue.com/pages/CONTRIBUTION_FORM_NAME/statistics

To try out the new feature, just type in all or part of the refcode you want to see, and you’ll find only those refcodes that match your search:

It doesn’t get simpler than that!

Our developers also threw in a couple of bonus features. The first one allows you to add your search terms to a URL parameter on the contribution form Statistics page: ?refcode=search

This way, when you load the page you’ll only see the search results that you defined in the URL param. Here’s an example of using this feature to find specific refcodes on an old ActBlue contribution form. Back then, we were using the same contribution form for each email send and used refcodes to track the sends, including using year and month digits to keep track of when we sent them. If we wanted to see refcodes for October 2008, the URL param would be ?refcode=200810

The other features will help you find a “starts with…” or “ends with…” match. To only find the refcodes that begin with a certain character string, use the ^ symbol.

For example, if you write “^footer” then results that start with “footer” will be the only ones that pop up.

Adding a $ after the character string will only return refcode results that end with your chosen phrase.

For instance, at ActBlue if we re-send an email, we’ll add a “_r” to the end of our original email’s refcode. Typing “_r$” into the search box helps us quickly find those results without having to scroll through the entire page.

At ActBlue, we generally use upward of 20 unique refcode (and one contribution form) for each email we send. And we assign a different refcode to each fundraising link, which makes it easy to analyze test results. If you’re just getting started using refcodes, check out our tutorial for more information.

This new refcode search feature will let you hone in on the refcodes you care about. Let us know how you’re using this new feature by emailing us at info[at]actblue dot com!

It’s crunch time so optimize those weekly recurring asks!

We’re fewer than six weeks from the election. That means, among other things, that optimal fundraising strategies become even more important than usual. Here at ActBlue, we’ve been running tests on a nearly daily basis on all kinds of Express Lane strategies.

Typically, we see the largest (statistically significant) improvements when optimizing factors related to the Express Lane askblock structure like amounts, number of links, and intervals between the links. For our own list, we find that, statistically speaking, the flashier aspects you see in some fundraising emails — emojis in subject lines, e.g. — do not do much (if anything) to improve donation outcomes. Here’s a tactic we recently tested, though, that’s a bit more on the fun side of things and definitely brought in a lot more money.

A little while ago, we started using our weekly recurring feature to great success. (By the way, if you haven’t tried this feature yet, shoot us an email at info [at] actblue [dot] com and we’ll turn it on for you.) After testing which amounts brought in the most money, we landed on this1:

We wanted to see if we could raise more money by asking for “$7 because there are 7 weeks until the election!” Gimmicky? Sure, but we had a hunch that it would perform well.2 Here’s what it looked like:

So what happened? The segment with the ‘7 for 7’ ask performed much better than the control; it brought in 87.6% more money, a statistically and practically significant improvement.3 Cool!

What’ll be interesting to me is to see when this tactic will lose its optimality. The key factor is that $7 (with gimmick) performed better than $10 (the control and previously optimal ask amount) despite it being a lower dollar amount. Though, at some point, a too-low number-of-weeks-to-election-dollar-ask-amount combination will negate the positive c.p. effect of the gimmick. Based on other testing we’ve done, my guess is that that will be at 4-weeks-$4. We’re doing follow-up testing on this “n weeks until the election!” tactic, so we’ll see!

If you decide to test something similar, send me an email and we can chat! Emails to info [at] actblue [dot] com with my name in the subject line will be directed to me.

P.S. Doing a lot of testing in the election run-up? Want a tool to help you manage your test groups? I wrote something in R for you! I’ll post something on the blog about it soon, but if you want it in the meantime, shoot me a note (emails to info [at] actblue [dot] com with my name in the subject line will be directed to me).

FOOTNOTES:

1 Actually, we built a model that predicts how a given Express user will respond to different types of donation requests based on previous donation information. Using those predicted values, we decide what type of donation ask they receive (of one-time, weekly recurring, monthly recurring) and for how much money they are asked. Math! The point: this is what we landed on for a certain subset of our list.

2 Of course, all else equal, it’s tough to distinguish whether any difference was due to the gimmick or because $7 is lower than $10. The theory would be that with a lower amount, more people would give, and even though the mean donation amount would likely be lower, the increase in number of donors would outweigh the decrease in mean donation size. This is definitely possible, but so is the opposite; it’s all about finding the optimal point.

In fact, we included a segment in the test which received an askblock starting with a lower amount and saw this dynamic in action, though the overall treatment effect was not statistically significantly different from the control. This lends support for interpreting the effect from the gimmick segment as the gimmick per se, but a detailed discussion is excluded from the body of the post for the sake of brevity. More rigorous follow-up testing on this “n weeks until the election!” tactic is already in the field— shoot us an email to chat!

3Pr(>|t|) < .01, controlling for other significant factors, including previous donation history.

Making your A/B test go off without a hitch

This post is the fourth in our blog series on testing for digital organizers. Today I’ll be talking about implementing your A/B test. This post will be full of helpful, quick tips.

So, we’ve discussed some things you might want to test, and some other things you might not want to test. Then, we walked through a simple way to figure out the number of people you’ll need in each of your test groups, which number depends on the smallest difference you’d like to reliably detect.1 Now what?

Well, the short answer is “run the test”, but of course it’s never that simple. Your next specific steps depend on what you’re testing, as well as which platform you’re using to run the test. There are too many possibilities for me to go through each one, but I can provide a few quick tips that should apply to you regardless of your specific situation.

First, make sure you have a reliable method of tracking your variations’ performance (like reference codes or an A/B testing tool (here are instructions for using ours)), and make sure you actually implement that method. This may sound like a no-brainer, but we’ve seen plenty of people start what would otherwise be an excellently set-up test with nothing to measure the variations’ relative performance! Is there a joke here about the “results” of the test?

Groaners aside, pointing out that error isn’t at all to make fun of the people who have committed it. Rather, we’re all busy, and things can get hectic. Having this on your pre-send checklist2 will save you from the realization that a lot of time spent thinking up a test, creating the content, and so on ad nauseam was all for naught.

What’s an example? Well, say you’re testing email content for donations. And of course, you want to use the best online fundraising software in the whole wide world, so you’re using ActBlue. Well, we have a handy feature that allows you to generate reference codes to track donations. We have a full instruction guide for using reference codes on our tutorial, found here. If you’re testing two different versions of your email, you could attach the URL param3 refcode=variation_a to the links in your first email and refcode=variation_b to those in your second email. Then, when you go to https://actblue.com/pages/[YOUR_PAGE_NAME]/statistics, you can measure the performance of each email. The information will also appear in a .csv download of your contribution form donations.

We also allow a handy refcode2 URL param if you want to conveniently subdivide your tracking. Conceptually, it’s the exact same thing as refcode; it’s value lies in the fact that it’s an extra place to store information. Think of a backpack with an extra divider on the inside for sorting your stuff. This is the internet version of that. For example, we use this for tracking link placement in the email. The need for refcode2, however, indicates that your test might be a bit complicated (i.e., there are more than just two variations, so setup and evaluation of the test is a bit outside the scope of the tips in this testing series.) That’s no problem, but you might want to shoot us an email at digital [at] actblue [dot] com to have a chat about test setup and design.

My second tip is related to groups. Taking your list—or some subset of your list—and dividing it up into smaller, randomized groups is a step that you’ll likely do in your CRM or email tool. Unfortunately, I can’t provide detailed instructions for each one. Chances are, though, that your CRM has an instruction page on how to do this within their software.4 In any case, this step is critical: without at least randomizing before conducting your trial, you’re setting yourself up for failure.

Here’s an example of how to do it wrong: let’s say you’re testing two emails, and even though you’re not sure which one is better, you have a hunch that email B is better than email A. So, not wanting to lose out on money, you decide to assign 20,000 people with the highest previous donations to group B and 20,000 people with the lowest previous donations to group A. That way, you can conduct the test to find out which email is definitely better, but not have to lose too much money along the way, right? Well, that’d be great, but unfortunately it’s all wrong. Assigning your groups that way would all but ensure you draw false conclusions about your test–email B is all but certain to bring in more donations, but it’s because it was assigned high-propensity donors, not necessarily because it’s the better email. Make sure you’re at least randomizing (with a proper algorithm, q.v. footnote 4) before splitting your groups and implementing your test.

My third tip is short and sweet. After you do all of this legwork, how do you know that the right variations were sent to the right number of people? What if you’re working with eight groups instead of just two? Well, the answer is that you don’t really. But, that can (and should!) be remedied. Place your own email address in each of the test groups. This won’t significantly affect the results of the test, but it will allow you to be sure that the right variations were sent. “But, I only have one email address, how can I put myself in multiple test groups without the hassle of creating new emails?”, you ask. Use the old email-campaigner’s trick of adding a “+” to your email address if you have a Gmail-based address. For example, if your email address is janesmith@actblue.com, you can add janesmith+test_email_a@actblue.com to group A and janesmith+test_email_b@actblue.com to group B; they’ll both be delivered to your inbox, and you’ll be able to perfectly spot whether the variations were sent correctly.

My fourth and last tip of the day is the most important one of all. Remember going through the process of determining your required sample size? Well, we did that for a (lengthily explained) reason. Don’t deviate from that now. What the hell am I talking about? I’m talking about peaking at the results too early (viz., before you reach your necessary sample size.)

I get it. You spent a lot of time setting up a test for these awesome variations of, say, a contribution form, and even though you know you need to wait until 15,000 people land on the form to see results, you want to check what’s happening? Has either taken an early lead? etc., etc., etc.

You can check what’s happening along the way, but you should definitely not stop the test early because it looks like one variation is performing better.5 This is a really common mistake, but a deadly one. I can’t stress this enough. The more times you test two variations for significance (which we’ll talk about in a future post) before the required sample size is hit, the more likely you are to detect a false positive. In fact, you can pretty quickly render your test effectively useless. So, if you just have to see what’s going on, fine, but promise yourself and statisticians everywhere that you won’t act on what you see!

Ok, that’s it for today! Next we’ll talk about evaluating your results and even more importantly, learning from them!

FOOTNOTES:
1 as well as your tolerance for the probability of getting a false positive and false negative, though using standard values can take some of the difficulty of this decision making away

2 which, if you don’t have a pre-send checklist (we prefer old-fashioned paper, big check boxes, and sharpies!), you should make one ASAP

3 A way of passing messages from the URL back to the website which it can use to customize the display or data recorded.

4 Now, this is generally the most basic possible insurance for proper group setup, as most tools will do nothing more than randomize and divide. There are other steps that should be taken for running anything more complex than a simple A/B test, which steps tend to best be done with a statistical tool such as R. If you think something more complicated is in-line for your program, don’t hesitate to shoot us an email (digital [at] actblue [dot] com)– we’d love to work with you to see if something more complicated is in order, and if so, we’d be glad to help.

5 Saying “definitely” in a conversation about statistics is— if also delightfully ironic— a bit misleading. This is actually a really complicated topic with plenty of proffered solutions, which range from minor adjustments in your calculations to an entirely different philosophical approach to statistics (I mean, who knew, right?). Those are all great discussions to have, but for now, it’s probably best to just assume you shouldn’t repeatedly evaluate your test variations before you hit your required sample size. Ok? Cool.