Changing error messages (split account or pasword message) reduces failed logins Mailchimp Marketing, Software (SAAS) Login page Clarity (copy), Persuasion October 2012 -66% failed logins -66% A B Mailchimp is a popular email marketing service with over 7 million users. When reviewing their analytics they found that they had 340,591 failed login attempts in 1 month (from April 12 to May 12, 2012). That’s the total number of times someone couldn’t remember their username and/or password, or simply mistyped. The team watched several login attempts and thought that the error messaging could be improved. The old message “Your username or password is incorrect” would leave many options for the user to try. Suppose a users has 4 possible usernames and 4 possible passwords – that would be 16 options in total. A more specific error message ‘We could not find an account with that user name” would leave only 3 options. Hypothesis Improved error handling – splitting password vs account name error – will reduce the number of failed logins. Results The rest ran from June 12-July 12 and the failure rate reduced around 60%. Learnings Error messaging can have a big impact on form completion. Note that there is a security tradeoff. Generic messages also makes is harder for hackers to guess usernames and passwords. Source: Mailchimp Save to My Case List Company Mailchimp Market Marketing, Software (SAAS) Test data Test element Copy Page Login page Conversion principle Clarity (copy), Persuasion Result -66% failed logins Test Cases from Mailchimp Adding social login options to the signup form Mailchimp +3.4% revenue Changing error messages (split account or pasword message) reduces failed logins Mailchimp -66% failed logins Related AB Tests Adding social login options to the signup form Remove cross selling text on product page Redesigning the homepage to just a signup form Changing pagination to infinite scrolling on the listing… Leave a Reply Click here to cancel reply. You must be logged in to post a comment.