Another Conversation with Magisterium AI

Faith, Reason, and Artificial Intelligence

Last summer, the U.S.-based tech company Longbeard released the artificial intelligence (AI) program Magisterium AI. As I noted at the time:

Similar to “large language models” (LLMs) like OpenAI’s ChatGPT and Google’s Bard, Magisterium AI is “trained” on a huge database of written documents and programmed to analyze those documents in order to answer queries from human users. Unlike other LLMs, however, Magisterium AI is trained using official documents of the Catholic Church and is designed specifically to answer questions regarding the Catholic faith.

The creators of Magisterium AI made ambitious claims about what the program could be used to do, such as providing information about the Catholic faith useful for catechesis, helping priests prepare homilies, and even scholarly tasks like analyzing ancient texts. In the post cited above, I asked Magisterium AI several prompts intended to test some of these claims and to explore the program’s capabilities and limitations.

I found that Magisterium AI is pretty successful at providing succinct summaries of Catholic teaching on a variety of issues. It also did a good job of analyzing the meaning of Scripture passages, at least with a little work finding the right question. On the other hand, the program had a bias toward more recent magisterial documents—for example, if I asked it about the views of a historical figure like St. Thomas Aquinas on an issue like religious liberty, it would provide an answer that reflected current Church teaching on the issue rather than an accurate summary of the historical figure’s viewpoint.

I also pointed to the general problem with LLMs that they do not truly reason or know things, but rather mimic the language they find in the documents on which they are trained. Magisterium AI’s creators have claimed that the fact that it is only trained on magisterial documents means that it is less likely than other LLMs to hallucinate, or respond with false information, but this problem of mimicry means that it is nevertheless not immune from hallucinating.

In the meantime, I have received a few emails from the makers of Magisterium AI with notifications of updates to its features and capabilities. Among the most notable include:

• The addition of a Scholarly Browsing Mode that draws on a library of historical theological resources like the writings of Thomas Aquinas, Augustine, and others, intended to provide responses to more historical questions rather than questions about current magisterial teaching.

• The ability to hear Magisterium AI’s responses by means of an automated voice feature (with both male and female options), rather than reading text.

• The development of Vulgate, a separate AI program focused on the digitization and analysis of historical texts.

• Improvements to Magisterium AI’s algorithm that are meant to provide better structured, more reasonable, and faster responses.

In light of these updates, I thought it was time to give Magisterium AI another try and report on my findings. In my earlier post, I included my prompts and Magisterium AI’s responses using screenshots, but this time I will use Substack’s block quote feature. Also, to simplify the formatting, I’ve deleted the citations included by Magisterium AI, but I reference them when relevant to the discussion.