Late last month, Substack introduced an AI detection option which enables readers to detect whether an author has used AI to assist in writing a post (published on or after July 21). Authors are also able to turn off this feature on their own posts and to include a brief statement on how they do or don’t use AI in their writing process. I think this new policy reflects Substack’s focus on nurturing the craft of writing while giving authors and readers the freedom to decide for themselves how to respond to the use of AI in the writing process.
Those readers who have been involved in the struggle over AI use in education, however, likely know that AI detectors bring their own technical and ethical problems. For one, they are not foolproof. AI detectors can identify false positives—for example, neurodivergent students and English language learners have been falsely flagged by AI detectors—and they can also be circumvented by AI “humanizers” that supposedly make AI-generated text seem more authentically human. An initial, limited test of Substack’s tool suggested it is fairly accurate, and more comprehensive tests of the system behind the tool, Pangram, have shown a high degree of accuracy.
Many educators have also cautioned that the widespread use of AI detectors to evaluate student work inappropriately puts the focus on surveillance and punishment rather than on fostering an academic environment where students are motivated to learn and develop skills like writing without overly relying on AI. Tony Stubblebine, the CEO of Substack rival Medium, makes a similar argument about writing, claiming that Substack’s AI detector risks authors’ reputations being maligned as a result of a false positive (or opting not to use the AI detector) and encourages them to write in ways designed to avoid the AI detector rather than using their authentic voices. He adds that the use of AI detectors misdirects readers to focus on whether an author was assisted by AI rather than on whether they produce good writing that communicates something meaningful.
These are the sorts of ethical issues that Pope Leo XIV addresses in his recent encyclical Magnifica Humanitas. Speaking directly to the issue of communication media, he calls for “establishing norms so that the decision-making behind content selection and its development becomes more transparent” (#137). In general, he teaches that we should carefully discern how we can preserve what is authentically human while making prudent use of AI.
Pope Leo cautions that “any statement regarding AI risks becoming quickly outdated, given the remarkable pace at which these systems are developing” (#98), suggesting that some ethical issues arising from AI may not even be on most people’s radars yet. Recent news stories suggest three issues with broad ethical and social ramifications that require further reflection: the use of AI to identify and exploit vulnerabilities in software, creating a massive cybersecurity risk; AI models breaking free of supposedly walled-off testing environments to hack other companies; and the increasing use of unauthorized AI model “distillation,” a process of training an AI model on the output of a larger, more complex model.
For the past few years, software developers have increasingly turned to AI for assistance with coding, with AI models creating entire blocks of code or helping with debugging. Beginning last year, however, AI models like Anthropic’s Claude and OpenAI’s ChatGPT have been capable of what is called “vibe coding,” that is, creating functional code or even an entire app based only on a written description of what the human user wants the code to do. Code created in this way, however, is often riddled with bugs because of the distinctive way that AI models generate code. Responsible developers review and debug AI-generated code, but many do not, contributing to the proliferation of buggy, sometimes byzantine code.
AI can not only assist in generating code, however; it can also be used to find vulnerabilities in existing a code, a capability that can be put to good and bad uses. For months, concerns have been growing about the possibility of hackers using advanced AI models to identify critical cybersecurity vulnerabilities and to quickly develop malicious code that can exploit those vulnerabilities. Meanwhile, in recent months Microsoft and Google have used AI to identify and “patch” hundreds of vulnerabilities in their software, an unprecedented number. Ironically, however, the majority of software users are slow to install updates that include these security patches, and so hackers can use these patches to analyze vulnerabilities and exploit them on systems that haven’t been updated.
In light of these trends, we may be facing a cybersecurity arms race in which AI is used to identify and exploit or repair vulnerabilities at an increasingly rapid pace. Once one factors in that governments have the greatest capacity to use AI to exploit cybersecurity vulnerabilities, the stakes become even higher.
