Mailchimp is a popular email marketing service with over 7 million users. When reviewing their analytics they found that they had 340,591 failed login attempts in 1 month (from April 12 to May 12, 2012). That’s the total number of times someone couldn’t remember their username and/or password, or simply mistyped.
The team watched several login attempts and thought that the error messaging could be improved. The old message “Your username or password is incorrect” would leave many options for the user to try. Suppose a users has 4 possible usernames and 4 possible passwords – that would be 16 options in total. A more specific error message ‘We could not find an account with that user name” would leave only 3 options.
Hypothesis
Improved error handling – splitting password vs account name error – will reduce the number of failed logins.
Results
The rest ran from June 12-July 12 and the failure rate reduced around 60%.
Learnings
Error messaging can have a big impact on form completion. Note that there is a security tradeoff. Generic messages also makes is harder for hackers to guess usernames and passwords.
Test details
| Test element |
Copy
|
| Page type |
Login page |
| Test date |
October 2012 |
Website information
| Company |
Mailchimp |
| Market |
Marketing, Software (SAAS) |
