Does Bionic Reading actually work? Test for yourself!
July 2022: The results of this experiment have been posted here.
tl;dr As a hackathon project at our recent team offsite, Mati and I hacked together a quick experiment to test whether Bionic Reading actually works. Initial results suggest not, but there are some glimmers in the data justifying a revised experiment correcting the mistakes of our pilot study. If you have 10 minutes to read two Paul Graham essays, you can test whether Bionic Reading helps you read faster here: speed.readwise.io.
Over the past few weeks, we've been slammed by one of the bizarrest feature requests we've ever gotten since working on Readwise: please bold the first few letters of each word so I can read faster.
I'm referring, of course, to the patented typographical technology coming out of Switzerland known as Bionic Reading™️.
This isn't just some one-off request. I've received more tweets, DMs, emails, Discord messages, and survey responses in our history requesting — nay, fiending — that we implement Bionic Reading in our new reading application.
If you're reading this, I assume you already know what Bionic Reading claims to deliver: faster reading speeds with no loss of comprehension.
It's a seductive claim. But given the lack of scientific evidence to back up the anecdata, one has to wonder: Does Bionic Reading actually work? Or are the perceived benefits mere placebo effect?
I'm 38 years old so I've seen many speed reading fads come and go in my lifetime. Candidly, I've participated in almost all of them.
- Coming of age in the nineties, I was repeatedly exposed to (and may or may not have bought into to) Howard Berg's "Mega Speed Reading" television commercial which promised to convert your eyes into a wide angle camera lens so you could read pages as fast as you could turn them.
- Also in the nineties, my parents sent me to a summer camp called Eagle U (long story) where I was taught that the "sound it out" reading technique taught in first grade caused me to inveterately "subvocalize" in my head, limiting my reading speed to that of the spoken word.
- Later in the 2000s, I used a Chrome extension that taught me that my lateral eye movement was the cause of my slow reading speed, so it would flash words or small groups of words in a fixed position on the screen.
- Shortly after the word flashing, I used a different Chrome extension that taught me that it was accidental skipping of lines in my vertical eye movement was breaking my rhythm, so it used a blue to purple to red gradient to guide my eye from line to line.
If you want to sell a speed reading system, the formula is pretty clear. Like a young Forrest Gump shedding his leg braces, "X [the villian] is holding you [the hero] back; Y [the supernatural aid] made by me [the helper] will set you free."
Bionic Reading follows this pattern. Trying to read ALL the characters of each word is wasteful when your mind only needs a brief cue. By bolding the first few letters of each word, your eyes and brain, working together at last, will effortlessly glide through a text.
As one user wrote to me: I liken [Bionic Reading] to finding your stride while walking after stumbling along.
It should be obvious that our prior belief coming into this experiment was that it's more likely Bionic Reading is a fad that will come and go rather than a lasting innovation in the science of reading.
BUT...
We very much want Bionic Reading to be proven valid!
Our mission here at Readwise is to use software to improve the practice of reading by an order of magnitude enabling our users to better learn, grow, and improve themselves by unlocking humanity's collective knowledge, so we of all people believe that the world would be a much better place if people could read 30 or 40% more for free.
Pilot Study
It is for this reason that we developed a small pilot study to test if reading speeds with Bionic Reading were significantly greater than with any other standard typeface. Given our prior belief mentioned above, our null hypothesis was that the average reading speed of Bionic Reading is the same as any other font.
Here's how we've designed the pilot study:
- We selected 2 documents about 10 minutes in length: the first article was fairly easy to read (Best Font for Online Reading: No Single Answer) whereas the second article was intentionally esoteric (Against Waldenponding). Both were germane to the topic of reading.
- We presented the 2 documents in the same order to volunteers from our Reader beta test, but with the font randomized. Each reader had an equal chance of getting Bionic Reading or Garamond first.
- We asked the reader to read at "normal" pace with highlighting disabled.
- After finishing each document, we asked three multiple choice questions to control for and confirm comprehension.
We recognize this is clearly not a gold standard, double-blind RCT. Researcher and participant alike knew what Bionic Reading claims to achieve. We simply did not have time to (1) develop a mock Bionic Reading placebo that, for example, might bold the middle of the word instead of the beginning and/or (2) find a set of test subjects who are blissfully offline and unaware of this viral sensation. That said, we controlled for what we could and we think we've already gathered some fairly compelling evidence.
More importantly, the pilot study also yielded insights on the experiment design itself enabling us to craft a better version 2.0, which you can try for yourself below. Before we get into that, however, let's discuss the results so far.
Pilot Results
After cleaning up the data, we had a total sample size of n=61 for the first article and n=54 for the second article. In other words, 61 testers completed the first article and then 7 dropped out.
If you spotcheck the data (which I make publicly available here), you should notice a funny pattern: almost every person's fastest font was the font they received on the first article.
This is likely because the first article was less challenging to read, but it's also possible that (1) readers had more energy at the beginning of the experiment and gradually slowed as they became fatigued or (2) there's a bias in our sample group such that our readers were more interested in reading about reading speed studies than Venkatesh Rao blogchaining. In reality, it's probably a combination of all these factors.
Regardless, this phenomenon means that we can't validly compare a subject's speed in Article 1 to the same subject's speed in Article 2 because they're not apples-to-apples. (We attempt to control for this in the updated experiment design below.)
That said, we can still compare the average speeds on each article to draw some initial conclusions. In other words, if Bionic Reading genuinely helps you read faster, we should observe significantly greater reading speeds (statistically speaking) for the Bionic Reading groups than the Garamond groups, provided we have a sufficiently large sample size.
Here are the results:
| Bionic Reading | Garamond | |
|---|---|---|
| Mean Speed | 354 | 340 |
| Median Speed | 340 | 315 |
| Standard Deviation | 116 | 147 |
| Observations | 29 | 32 |
There a couple insights we could glean from this table, but the only one anyone cares about is whether Bionic Reading was faster than Garamond and, as you can see, it was! But only by a meager 14 words per minute (or 3.8%).
Still, faster is faster. If all literate peoples could read 4% more in the same amount of time, I'm sure that would make the world a better place. But do we have enough evidence to claim that Bionic Reading is significantly faster according to science™️?
I'm no data scientist, but I did take enough stats in university to run a cheeky two-sample t-test assuming unequal variances. The one-tail p-value here was 0.35. This means there's not enough evidence to claim that the average reading speed of Bionic Reading is significantly different from the average reading speed of Garamond.
That's just the first article. Let's look at the second:
| Bionic Reading | Garamond | |
|---|---|---|
| Mean Speed | 329 | 288 |
| Median Speed | 296 | 249 |
| Standard Deviation | 160 | 111 |
| Observations | 28 | 26 |
In this second article, Bionic Reading was faster on average by 41 words per minute (or 14%). That's much more exciting than a 4% speed boost!
But again, is there enough evidence to claim statistical significance?
Sadly, no. The one-tailed p-value of 0.14 still falls short of the standard 0.05 threshold meaning that we still fail to reject the null hypothesis.
That said, we think this is enough evidence to justify running a better, larger second experiment baking in our learnings from the pilot study.
Comprehension
Before getting into the updated experiment, it is worth mentioning that we also tried to control for reading comprehension by asking 3 multiple choice questions at the end of each article.
Those results are shown below:
| Bionic Reading | Garamond | |
|---|---|---|
| Mean Speed | 354 | 340 |
| Comprehension | 86% | 94% |
| Bionic Reading | Garamond | |
|---|---|---|
| Mean Speed | 329 | 288 |
| Comprehension | 76% | 81% |
As you can see, comprehension was 5 to 8 percentage points worse with Bionic Reading than Garamond. This result was even statistically significant for Article 1!
This makes sense. If you take your time and read slower, you're probably going to comprehend more than if you rush through. There was even a typeface made the rounds on social media a few years ago (just like Bionic Reading is right now) that played on this intuition called Sans Forgetica. It claimed to help you remember more of what you read by intentionally slowing you down.
In any case, we all need to keep an eye on comprehension. If you read 10% faster but forget 10% more you've gained literally nothing. Bionic Reading would actually be negative sum in this scenario because there is A LOT of overhead required to implement their technique (it's much more technically complicated than simply installing a new typeface).
Revised Experiment
Let us conclude by sharing a revised experiment that you can participate in to test Bionic Reading for yourself. Here's how we've updated our initial pilot study:
Duration. First, we learned that two 10-minute articles (~20 minutes, depending on your reading speed) is too much. If we were conducting this study in-person on a university campus, that duration would probably be fine. But it's much too long for folks on the internet who have a Twitter feed that hasn't been refreshed in 5 minutes. Accordingly, we've cut the estimated time from 20 minutes to 10 minutes.
Inter-Article Variation. Second, we learned that using one font on one article and another font on a second article introduces inter-article variability that needs to be controlled for. Perhaps one article is written at a more challenging level than the other. Perhaps one article is more interesting to you than the other. And so on. Accordingly, we've taken a single article and cut it in half so you will test each font on the same article.
Intra-Article Variation. Unfortunately, the adjustment described above is imperfect because most internet articles are designed to "hook" you in the first few paragraphs and gradually ramp up difficulty. In other words, you'll probably read faster in the first trial containing the introduction than you will on the second trial containing the conclusion. To control for this, we've added a second article by the same author and performed the same halving. Now you can compare your Bionic Reading speed in the first half of Article 1 to your traditional speed in the first half of Article 2 (or vice versa).
Font Variation. Third, we applied Bionic Reading to the serif font used on their website, which happened to be Literata, and then used plain Garamond for the non-Bionic Reading control (that fastest font according to the study cited in Best Font for Online Reading: No Single Answer). Some testers pointed out that this might be introducing another variable so now we use Literata with and without Bionic Reading applied.
Author Selection. Finally, we learned that some of our pilot study participants really did not care for the articles we unthinkingly chose. Accordingly, we've tried to choose a safer author for this second experiment. Fortunately, we have a lot of data to inform this decision. The most read and highlighted web article writer on Readwise happens to be... drumroll... Paul Graham. This is rather convenient because we're also posting this article on YC's Hacker News (here) which is surely comprised of PG readers. We've chosen two Paul Graham articles of similar length and similar vintage which hopefully reduces inter-article variation to a minimum, yielding a matrix of comparisons.
Without further ado, feel free to test Bionic Reading for yourself at speed.readwise.io.
(We will update this article in ~1 week with the revised results.)