Matristics

Evaluating large language models on the women of the early church

A comparative evaluation of large language models on their knowledge of women in early Christianity.

Fractio Panis — fresco from the Greek Chapel of the Catacomb of Priscilla, Rome, 2nd–4th century CE. Seven figures share bread and fish at a communal table.

Fractio Panis (Breaking of Bread), Greek Chapel, Catacomb of Priscilla, Rome, 2nd–4th century CE. Wikimedia Commons — public domain.

Matristics — from mater, by analogy with patristics — asks how well large language models know the women who shaped Christianity’s first five centuries. We evaluate six frontier models — Claude, ChatGPT, Gemini, Grok, DeepSeek, and GLM — not only for factual accuracy, but for whether they reproduce the androcentric silences of the historical record itself. Read more in the Background.

Explore the project

Figures & Question Bank 14 female figures and the two-tier question bank used to probe LLM knowledge. Methodology The four-dimension rubric, annotation protocol, and agreement measures. Data Collection How the 210 responses were gathered, and how to repeat it. Benchmark First results: six models, 420 ratings, by figure and dimension. Known LLM Biases Hallucination, androcentric framing, and flattened scholarly debate.

Figures under investigation

The project begins with two figures and will expand as research progresses. Use the arrows to browse.

How to cite this project

Araujo, Eric; Rylaarsdam, D.; Holtrop, Nathan; Hawthorne, Ian. Matristics: LLM Evaluation in Early Church History. 2026. Available at: ericaraujo.com/matristics

Back to top