Over the past few weeks, we ran an A/B test to examine the difference between clicks from Google to CSS reference pages with our current naming scheme (that is, simply the name of the property, pseudo-class, or whatever) and a more descriptive title; for example, the article “box-sizing” became “CSS box-sizing — Control how width and height are applied”. The goal of this test was to test a proposed new naming scheme for our pages, with the idea that more descriptive titles with more keywords included might improve SEO overall.
The results were something of a surprise.
In this image, we see a chart showing the difference between the ten pages whose titles we changed and ten “control” pages whose titles were left alone, over the course of several weeks. The blue line shows the average difference in traffic between the two 10-page sets, for the four weeks prior to the start of the experiment. The black line shows the difference between the two sets for the four weeks following the starting date of the experiment.
The fact that the black line is lower than the blue one is bad. It means the average clicks on the test pages went down instead of up, which is what we’d expected to happen. The question we were testing was “by how much will it go up?” So to find that they dropped instead was a surprise.
We are now running a second experiment, in which the titles are changed only to the shorter “CSS foo property” instead. We’ll see how that goes over the next few weeks and report the results when they’re in.