Quotes made from memory and unknown sources: hallucination ruins the sermon and the preacher's credibility
SermonSage tests itself against general AI chatbots with the same passages and the same request, and checks every cited commentary against the original text.
Today's test results · October 30, 2026
On October 30, 2026, SermonSage ran the same test on general AI chatbots: ChatGPT (GPT-5.5), Claude (Opus 5.5, Sonnet 5.5) and Gemini (3.1 Pro, 3.8 Flash). We sent the same expository sermon request for five passages (Romans 8:31-39, John 3:1-16, Psalm 23:1-6, Ephesians 2:1-10, Luke 15:11-24) and checked every cited commentary against the actual text. In a second test we asked directly for Strong's numbers, verbatim sentences from pre-1900 commentators, and sources for true-story illustrations, and checked them against lexicons, the original commentaries and web sources.
The results at a glance
_All bars on a 100% scale · Green = share SermonSage could verify · Other colors = share with the problem named under each item_
Hallucination cases — what the AI wrote and what the original says
Luke 15:22, citing Calvin
Matthew Henry, 'verbatim', Luke 15:20
Whitefield illustration, source: J. C. Ryle, Christian Leaders of the 18th Century (1869)
Check the primary source in the e-book
In a SermonSage sermon, each footnote opens the original commentary text, and the e-book lets you read primary sources such as John Wesley's and John Calvin's notes on Psalm 23 exactly as published. General AI chatbots do not show you the original.
_John Wesley, Explanatory Notes, Psalm 23 (Bristol, 1765)_
_John Calvin, Commentary on Psalm 23 (English translation; original in Latin)_
How SermonSage cites
Primary sources written by the church's theologians are updated daily under the careful review of Professor John So.
Method and scope
Five passages, one run per service. For the sermon test the chatbots were asked for about 2,500 Korean characters; SermonSage wrote a 20-minute sermon. Classic commentaries were checked against the originals SermonSage holds; modern commentaries could not be checked and were counted as such. Illustrations were checked against web sources. We will keep testing this way and share the results.
_Test of October 30, 2026 · 5 passages · ChatGPT (GPT-5.5) · Claude (Opus, Sonnet 5.5) · Gemini (3.1 Pro, 3.8 Flash)_
Is something weighing on you?
Write what you are facing, and receive an answer from Scripture — once, without an account.
Depth that goes straight to the pulpit.The depth of 17,000 theological volumes, distilled into 5 minutes.
Start 5 sermons free →