Analysing my PhD thesis with Python

I recently saw someone post about the statistics of their PhD thesis, and I thought I could join in on the fun, but with an emergent communication twist! I will analyse my own thesis with some linguistic-inspired metrics and see what interesting patterns emerge.

In this post I will:

• Start with some simple overall stats
  • I use readability score calculations from textstat and textblob
    • For more explanation on these metrics, visit readability formulas and textblob documentation
  • I also use SpaCy for analysing other parts of the text
    • For explanation on SpaCy, visit their documentation
• Analyse the thesis section by section
  • This is very interesting in my opinion!
  • You can see all the metrics change significantly across different sections
• Compare my work to Shakespeare’s writing
  • Just for fun, to see how we stack up!

I use several readability metrics including “Flesch Reading Ease”, “SMOG Index”, “Flesch-Kincaid Grade Level”, and “Automated Readability Index”, which can be summarized as:

• Flesch Reading Ease: Higher scores indicate easier readability (100 is very easy, 0 is very difficult)
• SMOG Index: Estimates the years of education needed to understand a text
• Flesch-Kincaid Grade Level: Corresponds to US grade level required to comprehend the text
• Automated Readability Index: Similar to Flesch-Kincaid, indicates US grade level needed

Join me on this journey through my thesis statistics! You can also generate your own analysis, with small modifications required, by using the code linked at the top of this post (feel free to leave a comment/suggest changes on GitHub!).

Overall statistics

Let’s see what we’re working with. These are the stats extracted with the pipeline I used for the plots. Interestingly, we get about 6,000 more words with my pipeline than Overleaf’s word count.

Distributions

I also plotted the word distributions in a nice histogram, which some might find easier to read than a table.

Using SpaCy, I could also identify different parts of speech used in my thesis! As you can see from my distribution chart, my thesis is heavy on nouns, which is typical for academic writing - lots of concepts and things to discuss!

Section by section

Let’s start with a simple word count per section. As expected, my literature review is the largest section of all. The other significant spikes correspond to experimental sections and, my favorite, the appendices! My supervisors often commented that I have a tendency to pack the appendices with additional data.

You can also see the word frequency per section, per 1,000 words in heatmap format.

Next, let’s examine how the most frequently used words vary across sections. As expected, “temporal” peaks in the experimental chapters of my thesis where I discuss temporality extensively!

Now, let’s look at the sentiment analysis per section. Interestingly, my chapter on the game of Werewolf appears to be more polarizing and subjective. This could be due to the frequent use of words like “villagers” and “voting,” which sentiment algorithms might associate with political discourse.

We can also examine lexical diversity across the thesis. These metrics may be slightly skewed due to incomplete text sanitization. It’s interesting to see that lexical diversity peaks in the introduction of the chapter based on my NeurIPS paper.

We can quantify the readability of each section using established metrics. Again, the values may be slightly inflated due to text sanitization issues, but I’m pleased to see that the conclusions and introduction score well on readability, as these should be the most accessible sections. The technical sections score lower on readability, as expected, likely due to the large number of technical jargon words.

Academic-ness of my writing

Let’s examine some common academic writing patterns: passive voice usage, sentence length, lexical diversity, and first-person pronouns. As expected, we can see that technical sections tend to use more passive voice. Interestingly, my architecture descriptions appear to have notably shorter sentences!

I also decided (with help from Claude) to analyze whether I overuse specific “academic” terms, including:

• “analysis”
• “research”
• “data”
• “method”
• “results”
• “theory”
• “approach”
• “study”
• “model”
• “framework”

It’s interesting to see that I apparently use “method” frequently in the conclusions of my NeurIPS paper. However, this observation comes again with the caveat that shorter text sections can produce less reliable frequencies.

Comparing my writing with Shakespeare

I thought it might be interesting to compare the statistics from my thesis to Shakespeare’s writing. Fortunately, Andrej Karpathy has compiled a tiny Shakespeare dataset, which I used for this comparison.

Let’s start with sentiment analysis. I’m pleased to see that my thesis is both less polarizing and less subjective than Shakespeare’s works—suggesting at least some degree of scientific rigor!

We can also see that my sentences are somewhat longer, and my passive voice usage is significantly higher than Shakespeare’s.

Somewhat surprisingly, my thesis is less readable than Shakespeare according to all metrics. (Note that for Flesch Reading Ease, higher scores indicate better readability, while for the other metrics, lower scores are better.) This is rather humbling, considering Shakespeare still uses archaic terms like “thou” which aren’t particularly accessible to modern readers. But then again, he was a master playwright after all…

At least I can claim slightly higher lexical diversity! Though I’m concerned this might also be influenced by LaTeX commands that weren’t fully removed during text sanitization.

Finally, let’s examine the top words used by Shakespeare and myself. Unsurprisingly, there’s virtually no overlap. Emergent communication wasn’t particularly popular in the late 16th and early 17th centuries when Shakespeare was writing.

Summary

While this post is primarily for fun, I think it nicely showcases a few important points. First, text sanitization is surprisingly challenging and time-consuming—I now have much greater appreciation for data preparation specialists. Second, there are numerous fascinating approaches to analyzing text and, by extension, language! Exploring these different metrics and seeing how they compare has been super fun. This kind of analysis is precisely what made me pursue emergent communication and linguistics in my PhD, and I hope I’ve managed to convey some of that enthusiasm to you, dear reader!