I’ve been writing. A lot. For the past few months, I’ve written at least one if not two stories daily as part of the experiment for the 100 Days of Medium journey.
Throughout the journey, one of the things that really stuck out to me as something important to test was whether the length of each story mattered. Most of what I write has a 3 to 5 minute reading time, but occasionally, I’ll write shorter that only takes 1 to 2 minutes to read. From the stats I’ve gathered, I seem to get a lot less views for shorter posts than I do long ones.
So my question is: Do shorter posts on Medium get less views than longer ones?
In order to find out, I decided to run an experiment, and as usual, I’ve got some interesting data to show for it.
But first, some context for this experiment.
Is there an optimal length for stories written on Medium?
To better understand what I was getting myself into, I did some googling on the ‘optimal length for stories on Medium’.
The first relevant result I saw was the article, What makes an article popular on Medium? We analyzed 10,000+ data points to find out, which analyzed the statistics from dozens of authors and their stories before identifying patterns and coming up with a list of recommendations. Their recommendation for the optimal length of a story written on Medium is 6 to 7 minutes. Although I felt that the experiments could’ve been more in-depth, I agree with most of what they’ve said based on the data they’ve given; their recommendations are more or less what I’d also assume to work on Medium.
The second relevant result was actually from Medium’s Data Lab themselves, written by Mike Sall, former Head of Data Science at Medium. The article, The Optimal Post is 7 Minutes, while super helpful in explaining the correlation between the length of the story and the quality of the post based on increased engagement ( number of minutes read), didn’t address my question: Do shorter posts on Medium get less views than longer ones? What I want to know, in particular, is whether or not Medium’s algorithm favors longer posts as opposed to short ones.
Medium’s goal as a platform is to provide relevant results — it’s the basis behind their monetization strategy, and to do so, it makes sense for their algorithm to favor higher quality articles worth reading. If data shows that there’s a correlation to higher quality posts based on the length of the post, wouldn’t Medium naturally choose to feature longer posts more often?
It’s just speculation of course, but it’s what I’d do.
Time to dig deeper.
Analyzing clues based on Medium’s design
One thing I’ve realized as I’ve grown increasingly accustomed to Medium’s platform is that there’s a lot of attention to detail invested in every aspect of the site. Most of the design choices are well-thought-out and intuitively optimized for maximum readability (based on my previous experience as a UX designer).
The best example to Medium’s meticulous design is the different font color used for the author name in the featured section of the homepage and the stories in your feed.¹
My guess as to why there’s two different font colors is because Medium might not want their readers to focus on on the author of stories in the featured section since those are often the stories outside of your network. If users develop a habit of clicking only on stories they know the author to, there’ll be less people reading the featured stories on Medium, so by using a lighter closer to the rest of the information display (number of minutes, the star that indicates paid/non-paid), people are more likely to focus on the actual title of the story instead.
Of course, it could just very well be a design/development mistake and have nothing do with their attention-to-detail, but if that’s the case, why doesn’t Medium show the number of claps on the homepage? If you’ve noticed, the number of claps doesn’t show anywhere on the homepage. The reason for this is because claps provide social proof for a story’s popularity and would significantly reduce the number of people viewing less popular stories.
With all that said, since we now know that Medium puts careful thought and consideration into every design choice made, the very fact that the number of minutes per story is shown on the homepage indicates that the length of the post should matter a lot to readers.
I personally don’t care much about the length of a post; some of the best writing I’ve read from the likes of Ryan Holiday, Seth Godin, Jason Fried are all short but concise. But perhaps where it matters is on a subconscious level, where we’re more likely to click on stories of longer length because our brain draws a correlation between the length and quality of a post.
The question now is, how do we test? Time for an experiment.
¹ As of May 4, 2018, the font colors are no longer the same. There is a more padding (empty space) above the author’s name in the featured section though compared to the featured section.
Analyzing my current data
Let’s take a look at my stats, at the top 3 stories, sorted by highest views and filtered by non-publication.
It could be a coincidence, but it just so happens that the stories with the most views are also the ones at least 3 minutes in length.
Now let’s take a look at my stats again, at the top 3 stories, sorted by lowest views, and filtered by non-publication.
This time, the stories with the least views also happen to be ones with shorter lengths of 2 or less minutes.
It’s honestly hard to draw any kind of conclusion at this point because to accurately test something, you’d generally want at least a minimum of a thousand data points, but simply based on my limited stats alone and our earlier conjecture, it makes sense to continue assuming that shorter posts are less likely to perform as well as longer ones.
That said, it’s time for an experiment.
Since what we’re trying to test here is whether or not Medium’s algorithm favors longer posts as opposed to short ones, the best way to test is probably to rewrite the 3 shorter stories into longer ones.
If I were to republish the articles with exactly the same title, but with different content and a story length of at least 3 minutes, would I get more views than I did before?
If the views for the stories do not increase even after being rewritten, it’ll mean that I’ve assumed incorrectly and that shorter posts can perform just as well as longer ones.
But, if the views for the stories do in fact increase after being rewritten, it’ll mean that what I’ve assumed has an even higher likelihood of being true and that shorter posts don’t perform as well as longer ones.
Let’s see what happens two weeks after republishing the articles.
Without further ado, here are the results.
As we can see, the first story had more or less the same views, while the other two showed a slight increase in views.
Given that there isn’t really much of a difference in views even after rewriting the shorter posts, our previous conjecture is likely incorrect and the length of the story you write on Medium shouldn’t really matter.
As for why those stories had low views, I can only assume the problem lies with the title itself because it’s either unclear (very likely), or unappealing to readers. I’ll most likely run another experiment to test the title in the future by republishing those posts with a different title instead. When I do, I’ll update this article.
And that wraps this experiment up! As always, thanks for taking the time to read. If you’d like to follow along on our journey, please subscribe here.