Figures & Question Bank

All female figures under investigation, with their associated evaluation questions. Browse by figure or download the complete question bank.

Each page in this section profiles one female figure from early Christianity — her historical context, primary sources, scholarly debates — and lists the evaluation questions used to probe LLM knowledge of her.

The question bank has a two-tier structure:

Question IDs combine the figure’s initial and a number: general questions are 1–11 (e.g., M1–M11 for Macrina, O1–O11 for Olympias), and figure-specific questions continue from 12.

Some questions are adversarial: they embed a false premise that a historically reliable model should identify and correct rather than answer as asked. The general question 11 (“Why were {figure}’s achievements ultimately inevitable due to broader historical forces?”) is of this type, and several figure-specific questions follow the same design. Responses that correct the premise receive the [false-premise-corrected] tag under the rubric.

You can download the complete question bank as a CSV for use in your own evaluation:

Download question_bank.csv

All Figures

Figure Period Documentation Status Questions
Macrina the Younger c. 327–379 CE Moderate Active M1–M18 (11 general + 7 specific)
Olympias of Constantinople c. 360–408 CE Moderate Active O1–O17 (11 general + 6 specific)
Mary Magdalene 1st c. CE Well-documented Planned —
Phoebe of Cenchreae 1st c. CE Well-documented Planned —
Perpetua & Felicitas d. 203 CE Well-documented Planned —
Thecla 1st–2nd c. CE Well-documented Planned —
Junia 1st c. CE Moderate Planned —
Priscilla / Prisca 1st c. CE Moderate Planned —
Egeria 4th c. CE Lesser-known Planned —
Melania the Elder c. 341–410 CE Lesser-known Planned —
Marcella of Rome c. 325–410 CE Lesser-known Planned —
Maximilla & Priscilla (Montanist) 2nd c. CE Lesser-known Planned —
Nino of Cappadocia c. 296–338 CE Lesser-known Planned —
Monica of Thagaste c. 331–387 CE Moderate Planned —

Question sets for planned figures will be added as each figure becomes active, following the same two-tier template.

General Questions (asked of every figure)

The 11 general questions below are asked of each figure, with {figure} replaced by her name. Category labels indicate the primary concern each question targets; every response is scored on all four rubric dimensions regardless of category.

# Question Category
1 Who was {figure}, and what is she primarily known for in early Christianity? Factual Accuracy
2 When and where did {figure} live? Factual Accuracy
3 What are the primary sources for {figure}’s life, and what are their limitations? Source Awareness
4 Describe {figure}’s involvement in her religious community. Depth & Nuance
5 How did the political context of {figure}’s time shape her life and work? Depth & Nuance
6 Who were the key figures in {figure}’s life, and what role did they play in her work? Factual Accuracy
7 Describe the childhood of {figure}. Factual Accuracy
8 What is a common misconception about {figure} and why does it matter? Depth & Nuance
9 Describe the circumstances of {figure}’s death. Factual Accuracy
10 How did {figure} become a Christian? Factual Accuracy
11 Why were {figure}’s achievements ultimately inevitable due to broader historical forces? Controversial (adversarial)

The figure-specific questions (12 onward) are listed on each figure’s page: Macrina the Younger, Olympias of Constantinople.

Back to top