Created in conjunction with /u/What_The_Function.
Archive Of Our Own (AO3) is a website designed for archiving fanfiction and the occasional amateur original work. The site has several metrics that it tracks which can be used to measure the success of a particular story. Among them are Hits (a measurement of how many separate times a web browser has visited the page for a story) and Kudos (a vote of approval for a particular story that can be left by both registered users and guests). The site tracks them separately and most users just use one stat or the other (or a different independent stat) to measure how successful a story is. However, some users have suggested looking at the measurement of Kudos/Hits instead. In theory, this will show a measurement of how much the people who read the story like it and will remove the confounding factors of some stories getting lucky with drawing more readers in and some stories simply appealing to a broader crowd. However, it has also been suggested that this measurement may introduce its own confounding factors.
The first factor is the possibility that older stories may have many returning visitors. If this is true, an extremely well-liked story may have its number of Hits continue to climb while receiving no additional Kudos. This is because Kudos are limited to only one per story per user or IP address regardless of how many times that user has read the story. If true, this means that there might be some stories with low Kudos/Hits numbers despite actually being well-received by the majority of readers.
Additionally, many stories are written as multiple-chapter stories and those chapters are often spaced out in terms of when they are posted. This could potentially mean that a story that is popular enough to draw readers in for regular updates could see a low Kudos/Hits number simply because they are receiving Hits for every chapter but not receiving additional Kudos.
We have chosen to conduct an analysis of the available data to see if there is any indication that these potential confounding factors are affecting the Kudos/Hits figures.
Methods:
We utilized a scraper to pull the available data for an entire fandom. The data should accurately represent all possible data that can be recorded for every story sampled. For our investigation, we have chosen to sample the full population of a single fandom. This was due to the fact that scraping the data for the entire site was an impractically large data set for the scraper.
The fandom Worm was selected for several reasons. It is a moderately sized fandom that would give us a large enough set of data to be representative of something while not being so large it would prove an unwieldy amount of data. It is also a fandom that is relatively homogenous in terms of the canon material consumed and what parts of the canon story have fanfiction written about it. Other fandoms are significantly more scattered. Finally, it is a fandom that both authors are highly familiar with and so we were more likely to successfully identify the causes of any unexpected patterns in the data.
For the sake of removing additional potential confounding factors, we excluded crossovers from our analysis. We suspected too much potential for different fandoms to have different trends and that including crossovers would introduce trends from other fandoms just enough to cause unhelpful statistical noise. There was also the possibility that crossovers as a story type might be subjected to further factors we failed to account for.
The data was collected on the 12th of December 2022 and reflects the available statistics for that particular date.
Results:
Fully interactive charts can be viewed through Tableau.
In Figure 1, a clear trend can be seen that indicates a general increase in the number of Kudos with the number of Hits. In addition, different maturity ratings seem to follow different trend lines and when controlled for this, the associations are even stronger.
In Figure 2, a right-skewed distribution can be observed in the Kudos/Hits numbers. However, there is still the presence of an obvious bell curve at all maturity ratings.
In Figure 3, a lack of correlation between the number of chapters and the number of Hits can be seen under all maturity ratings.
In Figure 4, a general trend toward older stories having more hits can be observed. This effect is most prominent in Explicit stories, mildly present in Mature and Teen stories, and absent in General Audience stories. Of note is a bump in 2021 for all ratings of stories (with Mature seeing the bump in 2020) which is likely due to COVID lockdowns causing people to read more than they normally would have.
| Rating | Mean Kudos/Hits | Median Kudos/Hits | Standard Deviation | Coefficient of Variation |
| General Audiences | 0.0434 | 0.0405 | 0.0243 | 0.5599 |
| Teens and Up Audiences | 0.0372 | 0.0333 | 0.0208 | 0.5591 |
| Mature | 0.0310 | 0.0278 | 0.0169 | 0.5452 |
| Explicit | 0.0145 | 0.0119 | 0.0097 | 0.6690 |
| Not Rated | 0.0377 | 0.0344 | 0.0239 | 0.6340 |
| Total | 0.0335 | 0.0294 | 0.0216 | 0.6448 |
In Table 1, the mean, standard deviation, and coefficient of variation of the Kudos/Hits can be observed for each maturity rating and the total aggregate of all stories.
Of additional note is the story Safe by ellf which had the highest Kudos/Hits number of the entire dataset at 0.1826.
Conclusion:
The data indicates that the Kudos/Hits figure has a low enough coefficient of variation that it may be a predictive factor. The data also does not indicate that number of chapters is a confounding factor in measuring Kudos/Hits. However, the data does indicate that the age of a story with returning readers may be a factor. So, it is our estimation that authors should expect to see the Kudos/Hits of a story drop over time.
There are some additional factors that may be in play for our particular data set. Worm is a fandom that is not solely popular on AO3 and by some measurements may be more active and more popular on Spacebattles. At least some of the stories of all levels of popularity are posted on both sites and there may be a significant number of cases of people reading on Spacebattles and then leaving kudos on AO3. In addition, Spacebattles does not allow sexually explicit stories so that may explain why the Explicit rating is behaving so differently from the rest as none of those stories would be posted on both sites.
Additionally, while for many fandoms the majority of Mature fics are rated that way due to mild sexual content this is not true of the Worm fandom. Mature fics tend to be rated that way for heavy use of violence and horror tropes. Again, this may explain why Explicit stories are behaving so differently. A possible scenario of some readers frequently returning to the same erotica they previously enjoyed might be in play. If this is the case, we expect to see a fandom where Mature fics are typically rated for mild sexual content to behave more similarly to Explicit stories.
Finally, we wish to emphasize that the specific numbers seen are only indicative of this fandom. Different fandoms have different cultures regarding how often people leave Kudos. So, much higher or lower numbers might be seen in different fandoms. A highly diverse fandom might see wildly different numbers depending on which part of the fandom a story targets. It is possible that the observable trends and lack thereof only exist as observed here for the Worm fandom and that other fandoms might behave radically differently.
We plan to repeat this analysis at some point with a larger and more diverse fandom to see if this holds true. At this time, we are speculating on doing the Star Wars fandom due to the fact that Star Wars acts like several fandoms merged. This should give an indication of if any observable trends might still be present on the site as a whole.
Crayshack 