Changing error messages (split account or pasword message) reduces failed logins Mailchimp Login page October 2012 -66% failed logins -66% A B Mailchimp is a popular email marketing service with over 7 million users. When reviewing their analytics they found that they had 340,591 failed login attempts in 1 month (from April 12 to May 12, 2012). That’s the total number of times someone couldn’t remember their username and/or password, or simply mistyped. The team watched several login attempts and thought that the error messaging could be improved. The old message “Your username or password is incorrect” would leave many options for the user to try. Suppose a users has 4 possible usernames and 4 possible passwords – that would be 16 options in total. A more specific error message ‘We could not find an account with that user name” would leave only 3 options. Hypothesis Improved error handling – splitting password vs account name error – will reduce the number of failed logins. Results The rest ran from June 12-July 12 and the failure rate reduced around 60%. Learnings Error messaging can have a big impact on form completion. Note that there is a security tradeoff. Generic messages also makes is harder for hackers to guess usernames and passwords. Source: Mailchimp Test details Test element Copy Page type Login page Test date October 2012 Website information Company Mailchimp Market Marketing, Software (SAAS) Test result Test element Copy Page Login page Result -66% failed logins Test Cases from Mailchimp Adding social login options to the signup form Mailchimp +3.4% revenue Changing error messages (split account or pasword message) reduces failed logins Mailchimp -66% failed logins Related AB Tests Adding a message about guest checkout below the… Removing the € sign before the price on the product page Changing the product images from white background to… Change the 'Guest checkout" button text to "Quick… Leave a Reply Click here to cancel reply. You must be logged in to post a comment.