Archive

Testing

Recurring pledges are like gold. There’s a reason why they’re often called sustaining contributions. Building a base of recurring donors can have a huge impact on the sustainability of any organization, including campaigns.

And now we’re making it easier for you to raise more long-term recurring contributions. Introducing: infinite recurring!

You’ve got a choice: ask people for a recurring contribution for a defined number or months (old standard), or ask them for one with no expiration date (new!). You can also choose not to have a recurring option, but we don’t recommend it (I’ll explain later.)

Here’s how you do it: Go to the edit page of any contribution form. Scroll down till you see this:

recurring toggle

Click on it to expand. It’ll look like this:

recurring options expanded

Select your radio button and then scroll down and hit submit. Yep, that’s it.

ActBlue got it’s start helping candidates raise money for their campaigns, which are built in two year cycles, so we allowed folks to set up recurring contributions for up to 48 months. The assumption was that donors would feel more comfortable signing up for a recurring contribution that would be sure to end at some point. These days, more and more organizations, who are around cycle after cycle, are using ActBlue. Plus, the way people use credit cards has changed and we have a whole system to let you extend/edit/add a new card to your recurring contribution, complete with prompts from us. It doesn’t make a ton of sense to have time-limited recurring contributions anymore.

So we tested it. Would forms with an infinite recurring ask perform the same (or better) as forms with a set number of months? AND would you raise more money if you didn’t have a recurring ask on the form, but asked people with a pop-up recurring box after their contribution was submitted?

We’ve got some answers. Several committees have run tests, confirming that conversion rates on time-limited forms and infinite recurring forms are similar. So if you’re around longer than election day, go ahead and turn on infinite recurring.

Generally speaking, making a form shorter and giving people fewer options leads to higher conversion rates. So theoretically, taking the recurring option off of a form should lead to more donations. We have a pop-up recurring box that campaigns can turn on to try and persuade a one-time donor to make their donation recurring, and there seemed to be a reasonable chance that having no recurring ask on the form would raise more money.

Nope! Turns out that we got a statistical tie on conversion rates between having the recurring option on the form or off. Just having pop-up recurring turned on did not generate as many recurring contributions as having it both on the form and as a post-donation action.

There were slightly more contributions processed on forms without a recurring option, but not enough to generate a statistically significant result. And then add to that the lost revenue from having fewer recurring donations, you end up with a pretty clear take-way: leave the recurring option on the form. Sure, you can turn off the recurring option, but you’ll likely lose money. And nobody wants that.

That’s why recurring contributions have been on every ActBlue contribution form since the beginning. These days we run anywhere from 8-14% recurring, and over $11 million is pledged to thousands of campaigns and organizations.

There is one big question we haven’t answered yet: will you raise more money overall from an infinite recurring contribution than say one with a 48 month expiration date? We’re currently working on a long-term experiment to test exactly that.

The answer might seem self-apparent, but the truth is nobody really knows. Credit cards expire and people cancel their pledges. You never know for sure how much money you’ll raise from a recurring contribution, but if you pay attention to your long-term data, you’ll be able to figure out your pledge completion rate.

If you’re interesting in figuring out a recurring donor strategy, we’re more than happy to give you some (free) advice. Just drop us a line at info@actblue.com.

In the fast-paced digital campaigns world, if you’re not innovating and testing constantly, you’re headed for obsolescence. And, more importantly, you’re letting your users down, especially those in short-term competitive environments (aka elections). At ActBlue, we’re always developing our platform with metrics-driven decision making, aka testing.

The result is that today’s ActBlue isn’t the same as the ActBlue of a month ago, and that’s a great thing. Sometimes our tests fail. Others result in a barely statistically significant bump in conversion rates. But that’s ok because all of those little bumps add up. Occasionally we hit on a big winner that dramatically increases conversion rates. We do it in a methodical, constant way that allows us to identify improvements big and small.

One advantage we have is the sheer volume of contributions we process, which allows us to A/B test small tweaks to the form and get statistically sound results. If one organization tried running an identical test on their own, they’d never be able to identify as many improvements.

We’ve got thousands of campaigns and organizations counting on us to have the best system possible, so they can focus on winning. It drives our work and testing every single day.

Our tech team make changes to the platform daily. Some are minor tweaks, others major changes. They’ve developed a rock-solid platform where we can easily roll out significant feature or a layout change, even in the middle of the crazy busy end-of-quarter period. And that’s no easy feat, but a deliberate design choice so we can be as nimble as the party needs.

Today we thought we’d roll back the curtain just a little bit and break down some of our favorite A/B tests from the past few months.

Test 1: Employer Address Checkbox

We know from our data that a lot of donors mark retired or unemployed on the forms and we wanted to see if we could use that knowledge to increase conversions. Turns out: yes! We A/B tested our normal form with one that has a checkbox they can click if they’re not employed. The checkbox automatically provides us with the information, which fulfills the legal requirement and bumps up conversion rates.

Original:

Checkbox:

We saw a 4.7% improvement in conversions (p < 0.05, for those of you keeping score), so we switched over to the new checkbox version. Bonus points for cutting waaaaay down on customer service questions about the occupation/employer boxes.

Test 2: Shrinking the Contribution Form

Speed is essential in online contributions, so we’re always looking for ways to make the Contribution Form shorter and faster to load, but the rapid increase in mobile donations has made it even more important than ever. We ran a number of tests aimed at shrinking the contribution form, including the following:

- Removed credit card tooltip (which popped up when you click the credit card box) so it would load better on mobile
– Removed “Employment” section header
– Using horizontal employer fields rather than stacking them vertically

All of these tests ended without statistically significant results, but that was a win for us, because it meant we could make our forms less cluttered. If a feature isn’t adding value, that means it’s time to go. And bye bye those three things went on every single form in our system.

You can see the evolution of the Employment section below.

Version 1 (original):

Version 2 (horizontal):

Version 3 (no header, checkbox added):

Test 3: Multi-step Contribution Forms

We already wrote a whole blog post about this test, but it’s worth mentioning again here. This was one of those big wins, with a 25.86% increase in conversion rates with 99% significance. That was after just a few days of running the test. We had tested multi-step Contribution Forms a few years back, and they lost to our standard one page forms, which just goes to show how important it is to test and test again.

One page form (losing version):

Multi-step form (winning version):

We do one thing at ActBlue and we’re the best at it in the business. And the biggest reason is that we’re constantly upgrading our platform. We push changes out to everyone ASAP so that thousands of campaigns and groups big and small can get the best right away.

In a few months when we get down to the crunch of election time, know that we’ve got your backs and you will always be using the most optimized and tested form out there.

This week we officially announced Express Lane, and I’m guessing the fact that it can more than triple your money caught your eye. It can, and the way to raise more money is to learn Express Lane best practices and do your own optimization. We’re here to help you with both.

We’ve done a significant amount of Express Lane testing in our email blasts over the past few months to help you get started on what works– and what doesn’t– with Express Lane. Each email list, of course, is different, so you should probably test and expand upon the the takeaways below with your own list. And definitely let us know the results; we’d love to hear about them. It’d be especially great if you wanted to share your results here on the blog– just like the fantastic folks at CREDO Action were happy to do for this post– so that others can learn from your test results.

Here’s a little bit of background: our own email list consists entirely of donors, therefore it’s a pretty diverse group of folks. Also, we always fundraise to support our own infrastructure, not specific issues or candidates. Further, we spend most of our time optimizing for recurring donations because we’ve found them to be best for our organization, but much of what we say here also applies to one-time donation asks. We are, by the way, totally interested in collaborating with you on testing and optimization efforts– just give us a shout.

For this post, we’re going to discuss the gains you can expect from using Express Lane, results from some of the tests we’ve run on our Express Lane askblocks, and touch on stylistic concerns. Then, we’ll finish up with a summary of our recommendations and where you can go from here.

What to Expect

So, you probably expect to raise a lot more money using Express Lane, but what’s a typical increase? We’ve tested Express Lane vs. non-Express Lane on both recurring and one-time asks among randomly sampled Express users and seen Express Lane bring in more than triple the money for one-time1 asks, and 37.7% more for recurring asks (measured by money donated plus pledged recurring).

That’s quite a big boost, but other partners have seen significant gains, too. For example, here’s a test that was run by our friends at CREDO Action, some of our most sophisticated users. They tested a $5 control ask against a $5, $10, $25 Express Lane askblock. Their Express Lane version brought in 37.4% more donations than the control version. If you don’t see a noticeable increase in your testing, you should definitely reach out.

exp_lane_test_graph

Results from ActBlue’s April 2013 Express Lane test

Askblock Structure

We have an awesome Express Lane Link Creator tool for you, which you can find by clicking the “Express Lane” tab of your fundraising page. It’s really important that you use the language we provide there so that donors know that they’ll be charged instantly and why that’s possible– if you want to deviate from this, you’ll have to get our approval first. We do think, though, that you should stick with this language since it’s clear and concise.

But, how many Express Lane links should you include in the body of your email, and for what amounts? Should the intervals between amounts be equal? The answer to such questions will depend on your email list members but here are some suggestions, based on tests we’ve run, that should help get you on your way to optimizing your own Express Lane askblock structures!

One approach we’ve seen used by organizations in different contexts is what we refer to as a jump structure. The basic idea is that you set a large interval between the lowest link amount (which should be a low amount relative to your list’s average donation amount) and second-lowest link amount. Here’s an example we’ve used:

jump_example_image

Example jump structure

This relatively low-dollar link could encourage a much higher number of donations (if your jump structure amount is, for example, $4 instead of the $5 you’d usually use). This is because it’s a lower absolute dollar amount, but also a lower amount relative to the rest of the structure. Basically, the large jump between the lowest amount and the second-lowest amount makes the first one look small.

We’ve found that in general, this type of jump structure does indeed lead to a higher number of donations, but a lower overall amount of money, than the common structures which we used as controls. While it led to more donations, we didn’t see enough extra donations to outweigh the “cost” of the lower dollar amount and bring in more overall money. If you’re looking to bring in more low-dollar donations in the hopes of larger-dollar donations in the future, however, this might be a good strategy to try.

We’ve also looked at the effect of changing the lowest dollar amount in your ask block. In July, we tested the the following three askblock structures against each other:

Structure "A"

Structure “A”

Structure "B"

Structure “B”

Structure "C"

Structure “C”

Obviously, we were trying to see whether we could increase the total money we raised by increasing the amount of the bottom link2. The risk of this approach is that you might lose a certain number of donations by setting the lowest ask amount to be a little bit higher3.

We found that the by number of donations, A>B>C, but by overall money raised, C>B>A. The askblock labelled “C”, in fact, raised 21.1% more money than “A” (“B” raised 12.1% more than “A”), even though “A” brought in 15.3% more donations than “C”!

structure_test_graph

The “other amount” Link

A great thing about Express Lane is that users’ donations are processed once they click the link in your email body. However, as much as we try to structure our links perfectly, some donors are always going to want to do their own thing, and that’s okay. Enter the “other amount” link.

An “other amount” link doesn’t process the donation right away, it’s simply a normal ActBlue link that takes the user to your contribution page and allows them to choose a custom donation amount and/or recurring length. This is included as a default in our Express Lane Link Creator tool.

We at ActBlue focus on recurring donation asks because over the long run– and our goal is to be the best technology both today and years into the future– they bring in more money than one-time donation asks, even taking into account imperfect pledge completion rates. So, we worried at first that adding an “other amount” link might draw too many people toward giving one-time donations instead of more valuable recurring donations. But, we also know that it’s important to give people the option to choose their own donation amount, lest they not donate at all. This is why every ActBlue contribution page allows people to easily choose between a one-time donation and a recurring donation.

So we decided to test two things. First, we wanted to know whether the presence of an “other amount” link in our email body would lead to more/fewer donations. Actually, we were almost positive that getting rid of the “other amount” link would be a big loss, but we wanted to run the test anyway. That way, we could confirm this and make sure no one else has to lose money on the test. The result: don’t try this at home. The version which included the “other amount” link brought in 88.3% more money (90.6% more donations) than the version which did not. We’ll accept your thanks in the form of chocolate or wine. Just kidding! Our lawyers won’t allow that.

Second, we’ve performed several tests (and several variations thereof) of whether an “other amount” link which indicated that users could instead give a one-time donation would lead to more/fewer donations than an “other amount” link that made no mention of one-time donations. This matters to us because, as we mentioned, we focus mostly on recurring donation asks, and wanted to see whether we could retain people who would give a one-time donation, but might not know that it was possible.

Typically, an “other amount” link which mentions one-time contributions leads to a statistically significantly higher number of donations, but less overall money raised. While this setup might draw in some people who otherwise wouldn’t have given, it also pulls some would-be recurring donors into giving one-time donations, which bring in less money. This doesn’t mean that such language is a bad thing, but you should consider your fundraiser’s goals and organizational priorities while choosing your link language. If, for example your goal is to increase participation rather than raise as much money as possible, then mentioning one-time donations in your “other link” might be a good idea during a fundraiser focused on recurring donations.

No mention of one-time donations

No mention of one-time donations

With mention of one-time donations

With mention of one-time donations

Style

Stylistic elements of an email can often have a huge impact on your ask, and since Express Lane links are new, the presentation of them hasn’t yet been set in stone. We started sending emails with our Express Lane askblock simply as an HTML <blockquote> element. We wanted the Express Lane askblock to stand out and to be easily identified, though, so we devised a simple design to make it pop. We put our Express Lane askblock in a gray box and center-aligned the text4. It looked like this:

We tested this against our original structure among several different link structures, and the results were pretty interesting. Among link structures with 4 or 5 links (including “other amount”), the gray box boosted the amount of money raised by up to 37.7%.

Subtle Express Lane askblock styling

Subtle Express Lane askblock styling

The obvious concern is that some stylistic elements are really subject to novelty effects, and the initial boost in action rate will decline or disappear altogether in time. We think the gray box may be an exception, though. First, the gray box is pretty subtle, almost to the point of being too dull, so I doubt that it caused the fervor of a “Hey” subject line or manic yellow highlighting. Second, the box serves a legitimate function, i.e., to identify this new set of links that’s now appearing in emails as a single entity that stands out from the email content.

Where to go from here

You’ve seen how some slight changes– the link amounts, the intervals between them, the number of links, etc.– can seriously affect the performance of your Express Lane email ask. Hopefully, you’ve picked up some tips about how to structure your asks, as well as picked up a few ideas for testing that might prove fruitful for your own organization.

As progressive organizers, we all know how important participation and collaboration are. In this light, I encourage you to get in touch with us if you’d like to work together on running a test. Moreover, if you run a test with interesting results, we would love to hear from you so that we can share them with the larger ActBlue community.

Footnotes:

1N.B.: some of this money came from people giving recurring donations from the “other amount” link in our one-time ask.

2There could be an additional effect from having one fewer link in “C”, but our other testing indicates that this isn’t a particularly important factor.

3Think about it as if it’s a variation of the classic revenue maximizing problem, where Revenue = Quantity * Donation Amount. Of course, donors can still choose their amount by clicking the “other” link, but the suggested amounts do indeed impact behavior.

4style="background-color:#ECEDF0; padding:1.0em 1.5em; text-align:center;"

Note: This is the first in what we hope will be a regular series on the ActBlue blog sharing our lessons learned from our email program with our larger community of practitioners.

Have you ever wondered if you’d raise more money if you asked your email list for recurring contributions instead of a one-time ask? Yeah, us too.

We’ve tested this from time to time, and usually find for ActBlue and our community members that recurring asks perform better. But we know that the email copy can influence the results, so we decided to test it again. Last Thursday we sent out nearly identical emails to our members, but with two different asks. Here’s an example:

1-time ask

The only way we can do it is if we hit our big goal of raising $75,000 for ActBlue, by the end of the quarter on Sunday. Can you contribute $5, or whatever you can afford, right now to ActBlue, and ensure we’re prepared to help thousands of candidates and organizations raise millions of dollars next fall?

Recurring ask

The only way we can do it is if we hit our big goal of 1,000 donors to ActBlue giving $3 a month, or whatever they can afford, by the end of the quarter on Sunday. Can you contribute $3 a month right now to ActBlue, and ensure we’re prepared to help thousands of candidates and organizations raise millions of dollars next fall?

Each email was sent to 100k random members. We let the test roll for 24 hours before making a call on Friday morning, but it was pretty apparent early on that we were going to have a winner.

One critical piece of important information that we had was the ability to calculate with confidence how much money the recurring contributions would bring in. Recurring donors on ActBlue pick the amount of months they’d like to make their contribution – the maximum is 24 months and we set that as the default when people land on the page through the email parameters. ActBlue Page Dashboards (which are in beta) do the math for you and display exactly how much money your members have signed up to contribute. They’re a tad hidden at the moment due to the beta status, but just add /dashboard/list before your specific page name when you are logged in to see it. For example: https://secure.actblue.com/dashboard/list/offthecharts/. Here’s how it looks for this fundraising page:

But we know that for one reason or another some people don’t complete their pledge. Their credit cards expire and they never updated them with us (even though we ask) or they simply cancel their recurring contribution. That’s why we recently analyzed our pledge completion rate. Since November 2010 the percentage of money that was pledged to ActBlue’s own PAC and received is 88.97% for all recurring contribution pledges that have completed. We did not include people who are still contributing monthly.

Across all the different committees who use ActBlue, we find a sitewide completion rate of 80%, so ActBlue is a bit higher than the average. And our completion rate has grown from where it used to be. All-time we are at a 77.78% pledge completion rate, so clearly we’ve gotten better at getting our folks to finish out their pledges.

Since we had the pledge completion rate, we could calculate how much we could reasonably expect to get in from both the one-time contribution ask and the recurring contribution ask. And here are our results after 24 hours.

One time ask : $2,557 projected
Recurring ask: $4,365.70 projected

We had more contributions on the one-time page than the recurring page, but clearly we were going to end up bringing in more money if we asked the remaining portion of our list to make a recurring contribution. Thus on Friday we sent out the recurring contribution ask to the remaining members of our list, and followed it with a kicker on Sunday, the last day of the quarter.

Our assumption is that ActBlue is viewed by our members as long-term infrastructure that should be supported as such. We’re not a candidate that’s trying to post a great end-of-quarter number, and we’re not trying to raise funds for a specific project like other non-profits. So while our members dig the recurring contributions, your members may not. So test it and find out!

We know people often make different assumptions about recurring revenue, but as you see with this test, the recurring ask is worth significantly more even though fewer people participate. Plus, at this point – 18 months from election day – you’re building a recurring pool so that you’ll have all sorts of money coming in via recurring. That adds up! And it’s a chance to create a long-term bond with your contributors.

We’ll likely run this test again in the not too distant future to ensure that a) our member’s interests/behavior hasn’t changed b) the results were not due to the specific email copy being more suited for a recurring ask. But until then, you’re likely to just be getting recurring contribution asks from ActBlue.

Follow

Get every new post delivered to your Inbox.

Join 40 other followers