Changing pagination to infinite scrolling on the listing page

Etsy is a peer-to-peer – community driven – e-commerce website focused on handmade or vintage items and supplies, as well as unique factory-manufactured items. These items cover a wide range, including art, photography, clothing, jewelry, beauty products and toys. Dan McKinley – who was leading the optimization efforts at Etsy – wanted to improve the search experience by introducing infinite scroll (more items are shown when scrolling down).

“We thought that it was obvious that more items, faster was a better experience. There is a lot of web lore out there to that effect, based mostly on some findings Google’s made in their own search.”

Hypothesis

Changing the pagination to ‘infinite scroll’ on the search results page, will increase items viewed and eventually purchases, as this is easier for the user.

Results

The results were not what Dan expected …

Metric (Per visitor)	Control	Variation	Change
Items viewed	80	40	-50,0%
Clicks	0,652	0,5811	-10,9%
Items favorited	0,0752	0,0689	-8,4%
Purchases	0,0164	0,0127	-22,6%

The visitors bought fewer items from search, but interestingly not overall. Visitors just stopped using the search to find their items.

Learnings

The test results surprised the team – and initially they thought it was due to a bug in the process. However the results were valid.
The main learning was actually in the process. The underlying assumptions were not properly validated before implementing.

Are more search items actually better?
Are faster results better?

In two simple follow up test it was found these assumptions were not valid. Had the team known this in advance the ‘infinite scroll’ would not have been implemented. Building, bug fixing and testing an infinite scroll features is a lot of work on a high traffic site like Etsy.
Big lesson learned there 😉

Note: Does this mean infinite scroll s**ks on your site too? Not so fast… Would Google be using it on their image search if it would not work? Perhaps for a listing page where people need to read the results (like the google text search page) pagination is better. When people are more skimming – like in Google image search – infinite scroll might be helpful.