The Multi-armed Bandit: New and much improved A/B testing tools

The A/B test tool on ActBlue, which allows you to test out Contribution Form titles and pitches, among other variables, has gotten a significant upgrade, just in time for campaign season.

The old A/B testing tool worked great, but it also forced you to wait around for both test variations to get enough traffic to gain statistical significance. If one version was performing way better than the second one, that meant you were losing out on potential contributions in order to gain valuable insight.

This is how most A/B testing tools work, and it’s a good system. But with the new ActBlue testing tools, which use a more advanced statistical algorithm than typical A/B testing, you can still achieve statistical significance without having to sacrifice a ton of traffic to a losing form.

As the test runs and one variation begins performing better, we’ll start sending more traffic to that form, roughly in proportion to how they’re trending. You can see the traffic allocation listed just above each variation on the “A/B Test” tab of your Contribution Form. The traffic allocation will change continuously as donations come in. It’s important to note that if a variation is receiving 75% of the traffic, that does not necessarily mean it’s conversion rate is 3X as high as the other variation(s). If you’re curious what it actually does mean and want to talk complicated stats, you can get in touch with us here.

If there was a false positive and the losing form starts doing better, the traffic allocation will begin to reverse. The test will continue to run indefinitely until you click “Make Winner.” The A/B testing tool will eventually send 100% of volume to the winner if you don’t make either version the winner manually.

The new A/B testing tool makes your tests more efficient, which means you can try out more of them. If you have radically different language you want to try on a form, alongside three more standard pitches, there’s little risk. If it doesn’t work out, we’ll send fewer and fewer people to that losing form.

We wanted to give special thanks to Jim Pugh from ShareProgress for sharing notes on the multi-armed bandit method used in their software and helping us out with building this tool (and for hanging out in the ActBlue office for a week)!

As always, let us know what tests you’re running and what’s working for you at!

Thank your donors AND ask them to take the next step

We’re introducing a feature you’ve been asking for just in time for campaign season! Now you have the option to redirect your Express users to any page you want after they’ve contributed. This gives you the opportunity to send your donors to an action page on your website, like a petition. Or, you can redirect them to a volunteer form and ask them to take the next step in supporting your cause.

To use the redirect feature visit the Edit tab on any Contribution Form and scroll down to “Show Advanced Options.”

Below the “Thanks page blurb” prompt you’ll now see the “Thanks redirect URL” option.

Here you can place the link to your website or any other page you want your Express Users to see after making a contribution.

After contributing, your Express Users will see a popup that says, “Your contribution succeeded, thanks!” for 5 seconds before being directed to the page you’ve chosen.

If a non-Express User has contributed using the form, they’ll still be brought to our thanks page with an Express signup ask.

Express Users are donors who have taken the time to save their information and commit to your cause, and it’s important to continuously cultivate those relationships. By taking the time to redirect these donors to an action page after they’ve contributed, you’re moving them along the ladder of engagement and showing them how much you value their support.

Any questions? Feel free to comment or drop us a line at

On-the-go Campaigning: Contribution Alerts by Form

You’ve long been able to set up email contribution alerts for your campaign or organization, but with campaign season heating up, we’ve added that feature to individual Contribution Forms as well. This will help you track individual fundraising emails closely and keep an eye on any large donations that come in, no matter where you are. And as an added feature, you can set an alert for any refunds that are processed on the form.

To set an alert, navigate to your form’s dashboard and go to the new Email Alerts tab. Click “new contribution form alert” to set up your first alert.

Select the appropriate email address from the drop down menu, add a threshold above which you want to receive an alert, and choose either contribution or refund alerts. If you want to receive an alert for any donation or refund, regardless of amount, put a 0 as the threshold. However, we recommend setting that bar higher so you’re not inundated with emails.

Click “Set up alert” and you’re done. You can set up as many different alerts as you want on the Contribution Form. You’ll receive an email every time someone donates or is refunded, depending on your preferences.

It’s great tool to help your finance director keep tabs on big donations or refunds. It also helps you keep track of any large gifts so that your Executive Director or candidate can personally thank the donor.

Have questions? Let us know at

Conversion rates by refcode

Our developers have been on a 2015 quest to give users greater access to data and visualizations to help you optimize your fundraising program.

We just rolled out a new feature on page dashboards: conversion rates per reference code (refcodes, for short). The two new columns on the refcode table display the number of people who landed on your form and the percentage of those that actually made a contribution.

Previously, you had to use your email mailer to check how many folks clicked each link to your contribution form and then look on your dashboard to see the number of contributions that were made. This will save you a step.

Just go to any form statistics page and you’ll see it (sample image from our testing):

FYI, we added new tracking functionality to make this work, so there isn’t historic data — only data from yesterday on will show up.

However, this will come in handy when sending emails, particularly if you are testing different versions of an email that are all linking to the same contribution form.

And, in addition to comparing conversion rates among different refcodes on the form, it should be useful to see how conversion rates differ from form to form. Know your organization-wide conversion rate, and pay attention if any form slips below that. Low conversion rates might ensue if there is a disconnect between the email content and what’s included on the form or if there is too much or too little text on the form.

It’s another tool in your toolbox that should help save you time and make better decisions. Speaking of new useful things, don’t miss the Recurring Retention charts that we just rolled out.

Questions? Just drop them in the comments or shoot us an email at info AT actblue DOT com.

Evaluate your recurring program with charts!

Building a pool of recurring donors is critical to the long-term success of organizations and campaigns. These sustaining funds are a huge help — budgeting is much easier when you have an idea of how much money will be coming in. Plus, donors appreciate that recurring contributions give them both a sense of investment in and a long-term relationship with the campaigns and organizations they support.

Right now, there are over 150,000 recurring pledges across our platform. Some political organizations have been steadily building up their recurring program over multiple cycles, while others are just now asking their supporters to sign up for recurring contributions. Either way, now is a great time to evaluate your recurring program’s strength. We have introduced two visualizations to help you do so. The election is still far enough away that anything you do now to build a lasting recurring pool will have a big impact.

Our new feature, the graphs found on the “Recurring Retention” tab, provides an overview of the recurring contribution performance for a specific campaign or organization. Admins can navigate there by going to their dashboard and clicking on the “Recurring” header beneath “Metrics.” The new tab is to the right of the “Overview” tab.

The first chart groups recurring contributions together by month. Here’s what ActBlue’s chart looks like:

Rows represent every recurring contribution that began that month. The numbers from zero to thirteen above the columns? Those represent the number of months since the initial donation. The percentages represent the retention rate of those contributions.

You can look left to right to follow the monthly trends of a group of contributions. You can also look top to bottom to figure out what happens to contributions after a specific number of months. Or, you can spot trends. Below is the same ActBlue chart as above, but with red boxes to highlight the November-to-December jump:

Notice that inside the boxes are some major retention rate decreases. That is most likely because many donors only want to give until the election, even though they signed up for an ongoing recurring contribution. This is not out of the ordinary and fairly representative of the cyclical nature of politics.

We’ve also included a graph at the bottom of the “Recurring Retention” tab to help you figure out what is actually normal for your organization. Here’s how that chart looks when you hover over a specific line:

The grey and yellow lines represent the retention rate for a month’s worth of contributions. Another way to think about it is that each row from the chart on top of this page is represented as a single line in this graph. The red line is a trend line, which can help you gauge whether a month’s recurring pledges are over- or under-performing. Lines above the red line are being fulfilled at a higher rate than average. The opposite is true for lines below the trend line.

Here at ActBlue, we’re taking a look at our own recurring program and using these new visualizations to do so. This post is just the first in a series of insights, so stay tuned for our next one when we report back. In the meantime, this new feature can give you a sense of how your organization’s recurring program is doing on a monthly basis. You can start by customizing your recurring receipts1 and determining if that improves retention rates. Don’t forget that now is the time to keep growing your recurring pool! We’re just seven end of quarters away from November 2016.


1: Just go to your entity’s dashboard, click on “Settings” under the “Tools” header, and then scroll down and click on “Blurbs, Disclaimers and Tracking Codes.” Under the “Email Message Blurbs” header, you’ll find the box called “Recurring email blurb”, which is what you want to edit.

Tandem Contribution Forms Just Got WAY Better

Our team is always thinking through ways to make our contribution forms easier to fill out and more streamlined. When donors have too many options and abandon a form, that’s known as choice paralysis. Eliminating that choice paralysis is a big part of building better contribution forms.

Tandem contribution forms list multiple candidates, which require more decisions to be made by donors. But the vast majority of people choose to just split their contribution evenly between all the candidates on the form. That used to look like this:

Too many options and too many boxes for our liking. Do you want to give more to candidate A than organization B? How much do you want to give in total?

We boiled the form down to that last question — how much do you want to give? This made it a lot easier for donors to give (spoiler alert: this A/B test was a huge success).

Now, when you land on a tandem form, you’ll see the normal amount buttons with a note underneath saying who the donation will be split among. You can still click a button to allocate different amounts to each candidate, but donors are less overwhelmed when they land on the page.

Here’s the new form:

So how successful was our A/B test? We saw a 7.16% overall improvement in conversion. That’s unheard-of-huge. We’ve done so many optimizations of our forms that we cheer for a test that leads to a 0.5% increase in conversions.

Part of that overall group consisted of non-Express users (people who haven’t saved their payment information with us) who land on our traditional multi-step form. Among that group we saw a 26% improvement in getting people to move from the first step of the process (choosing an amount to give) to the second step (entering their information).

There are so many candidates and organizations running really thoughtful tandem fundraising campaigns, and this is going to mean a huge bump for them. If you have questions, or want to tell us about a tandem campaign you’ve run, let us know at info AT actblue DOT com. We want to hear from you!

Introducing the Image Awesomifier

Our tech team just rolled out a useful, behind-the-scenes tool to make sure your Contribution Forms load as fast as possible.

We allow campaigns and organizations to use custom HTML and CSS on their branded forms, and we often see campaigns hosting their own images. While we love your customizations, hosting your image on a different site can make the form load slower. It can also make the page appear insecure to browsers, depending on where you choose to host it. And that can lead to security warnings (even though your pages have always been safe), which no one wants to see.

With the new Image Awesomifier (working title), we automatically grab your image and host a version of it on ActBlue’s super fast and secure servers.

This might not make a huge difference in February of an off year, but come 2016, this feature is going to shave precious milliseconds off your Contribution Form load times. And that means more conversions for you and your team. Plus, if your candidate or organization has a big moment (think Wisconsin Dems and Wendy Davis) and your fundraising goes viral, you won’t have to worry about your website crashing because too many people try to load an image hosted there. We’re saving you from yourself.

The best part? You don’t have to do anything different. Just keep adding your images the way you always have.

When a refcode search box isn’t enough…there’s a chart!

Earlier this month we wrote about the refcode search box on the Contribution Form’s statistics page. It’s a neat feature that makes it easy to display metrics for refcodes that match your search terms. And now we’ve added a chart to illustrate that same page’s data. Just click on the words “Toggle refcode chart” to display it — if your form has thousands of contributions, it might take a few seconds for the graph to load. Here’s an example from an old ActBlue Contribution Form:

Like we mentioned in the earlier post, there are two ways to get to that page (and both require you to be logged in): if you’re already on your Contribution Form, go to the Contributors tab, and click on “View Form Statistics” or just insert your Contribution Form name into the url below:

When toggled, the chart displays the top-performing (maximum of 20) refcodes. And when you type in the search box, the chart will update based on your search input.

For instance, if you used one Contribution Form for the last five months of the election, it might look something like this:

Displayed are the 20 best-performing refcodes during that five-month timespan. Let’s say you want to display only November sends and you had a refcode system in place such that an email send in November 2014 would start like this “201411.” Here’s what that chart would look like after you entered those digits into the search box:

And our developers threw in a bonus capability: the chart also keeps track up to the seconds!

This is great for when you’re monitoring performances of refcodes just minutes after an email send. Here’s a chart displaying a single refcode within the first 15 minutes after a send.

Notice the tiny bumps — those are contributions measured on a second-by-second basis.

Our habit at ActBlue is to refresh this page again and again after sending emails, and this chart is another useful visual aid to keep track of refcode performance.

Spotlight on refcodes

For all those refcode-happy committees, this feature’s for you! Now you search for a specific refcode on the Contribution Form statistics page.

There are two ways to get to that page (and both require you to be logged in): if you’re already on your Contribution Form, go to the Contributors tab, and click on “View Form Statistics” or just insert your Contribution Form name into the url below:

To try out the new feature, just type in all or part of the refcode you want to see, and you’ll find only those refcodes that match your search:

It doesn’t get simpler than that!

Our developers also threw in a couple of bonus features. The first one allows you to add your search terms to a URL parameter on the Contribution Form Statistics page: ?refcode=search

This way, when you load the page you’ll only see the search results that you defined in the URL param. Here’s an example of using this feature to find specific refcodes on an old ActBlue Contribution Form. Back then, we were using the same Contribution Form for each email send and used refcodes to track the sends, including using year and month digits to keep track of when we sent them. If we wanted to see refcodes for October 2008, the URL param would be ?refcode=200810

The other features will help you find a “starts with…” or “ends with…” match. To only find the refcodes that begin with a certain character string, use the ^ symbol.

For example, if you write “^footer” then results that start with “footer” will be the only ones that pop up.

Adding a $ after the character string will only return refcode results that end with your chosen phrase.

For instance, at ActBlue if we re-send an email, we’ll add a “_r” to the end of our original email’s refcode. Typing “_r$” into the search box helps us quickly find those results without having to scroll through the entire page.

At ActBlue, we generally use upward of 20 unique refcode (and one Contribution Form) for each email we send. And we assign a different refcode to each fundraising link, which makes it easy to analyze test results. If you’re just getting started using refcodes, check out our tutorial for more information.

This new refcode search feature will let you hone in on the refcodes you care about. Let us know how you’re using this new feature by emailing us at info[at]actblue dot com!

Time to Double Down

With just 13 days to go until the election, ActBlue is doubling down, and your supporters can too.

Double Down (formerly known as Insta-match for those of you with us in 2012) allows you to create a pop-up asking donors to double their donation after their initial contribution is processed.

This is your chance to convey a sense of urgency to your donors and make the conversion, so make sure you use short, compelling content (pictures allowed, too!).

To turn on Double Down, go to the “edit” tab of a form that you have created and scroll down and click “Show ‘Double Your Donation’ Options.”

Check the box to enable the feature and write a powerful sentence or two on why donors should double their contribution to your campaign or organization.

For compliance purposes, doubled donations will show up as two separate line items with the same ID.

We’re always working to make sure ActBlue’s tools suit your needs through the election cycle, and we hope Double Down will help power your campaign to victory. Weekly recurring has always been a super successful feature, but at this point in the year, you need a new way for donors to show their support. Double Down, just like weekly recurring, will end after Election Day.

If you have ideas for other great October-specific tools or questions about setting up or using Double Down, drop us a line at info [at] actblue [dot] com. Happy fundraising!