Written by Carter Trent
Edited by Nicki Steinberger, Ph.D.
EDITOR’S SUMMARY: Artificial intelligence is rapidly becoming part of daily life—including emotional support and mental health guidance. But as AI tools grow more human-like, concerns are mounting over emotional dependency, misinformation, privacy and the risks of treating machines like therapists. As organizations race to adopt the technology, questions remain about how to use it safely, responsibly and without replacing real human connection.
Artificial intelligence (AI) exploded onto the global stage in 2022 with the arrival of a chat app called ChatGPT. Since then, organizations and world leaders have touted AI as a technology poised to reshape daily life, business and communication. According to the U.S. government, it’s “ushering in a new Golden Age of innovation, human flourishing, and technological achievement for the American people.” The bold claims are understandable. AI offers clear practical advantages. It can create content—drafting an email, building a resume or producing a video. It boosts productivity by helping you complete tasks like financial budgeting, planning an event and researching information online. It can act as a personal assistant by suggesting a workout routine, planning travel itineraries and helping you stay on top of daily priorities. It can even assist in building a digital app or website of your own.
The technology is easy to use. Several companies offer an AI app to download, or you can access it on their website. From there, you type a question or request into a box—called a prompt. The software responds in seconds. It can mimic human speech, making the interaction feel surprisingly human. It replies in plain language rather than technical jargon, allowing for natural back-and-forth conversations.
Using AI can feel less like a traditional search engine and more like interacting with a collaborator. It can summarize complex information in seconds, help brainstorm ideas, explain difficult concepts in accessible terms and accelerate tasks that once required hours of research or technical expertise. For small businesses, students, creatives and people juggling demanding schedules, the shift can feel genuinely transformative—expanding what a single person is capable of accomplishing in a day.
To some, the technology represents more than the arrival of a powerful new app or software platform. It signals a fundamental shift in how humans access information, solve problems and approach creative work. Tasks that once required specialized expertise, extensive research or entire teams can now begin with a simple prompt. Writers use AI to brainstorm and edit. Programmers use it to troubleshoot code. Students use it to simplify difficult concepts. For many people, AI feels like a genuine expansion of their capabilities—one reason adoption has accelerated so rapidly across nearly every sector of society.
The technology is also evolving rapidly beyond simple chat interfaces. A new wave of AI “agents” is being designed to carry out tasks autonomously—managing schedules, conducting research, organizing workflows, making purchases and interacting with other software on a user’s behalf. As these systems become more personalized and integrated into daily life, the line between digital assistant, tool and companion may become even more blurred. AI’s growing utility has fueled enormous enthusiasm, but discussion of the downsides has lagged behind as governments and businesses race to adopt it. One problem is that AI can provide inaccurate information, requiring you to verify it against credible sources. Several law firms, for instance, got in hot water for using AI to create court briefs that cited legal cases that didn’t exist.
More concerning is that people are now turning to AI for medical and mental health advice—relying on it like a doctor, therapist or confidant. AI’s conversational nature makes it appear suited for this role. No major medical organization endorses AI as a replacement for professional care, yet many Americans are increasingly using it that way. According to a 2026 survey, “KFF Tracking Poll on Health Information and Trust: Use of AI For Health Information and Advice,” about a third of adults adopted the tech for health-related matters, with 16 percent seeking emotional support through AI tools. Some analysts now describe ChatGPT as functioning like one of the largest informal mental health support systems in the U.S. Before diving further into the pitfalls of using AI for mental health, it’s helpful to first understand some background about the technology.
What Exactly Is AI?
Artificial intelligence is a branch of computer science focused on creating systems that can perform tasks associated with human thinking. It uses clusters of interconnected computers housed in massive facilities—some requiring enormous amounts of land, energy and water—to perform tasks requiring learning, reasoning, problem-solving and decision-making. The companies building the technology start by gathering data from various sources: collecting publicly available information on the internet, buying it from organizations and harvesting it from the usage of their apps. This data trains algorithms to identify patterns, and the software then applies statistical models to choose the most relevant response to your request. For example, you might ask an AI app, “Summarize the causes of the American Revolution in three sentences.” Rather than retrieving a single source, the system generates a response by identifying patterns and relationships across enormous amounts of training data.
Some semblance of artificial intelligence has existed for decades. Tech giant IBM made headlines in 1997 when its Deep Blue AI beat the world’s reigning chess champion—the first time a computer demonstrated that machines could out-think humans in a complex strategic setting. ChatGPT’s arrival represented another milestone by bringing generative AI, or genAI, into the mainstream. This form of AI is distinguished by its ability to generate original responses or content, which allows it to “chat” with you. It’s how ChatGPT got its name: Chat Generative Pre-trained Transformer. A transformer is the software engine that analyzes the relationship between words in a sequence. Before genAI, most AI systems could analyze data and efficiently carry out predetermined tasks, but they were generally far more limited in generating open-ended responses or creating original content. GenAI can do this, and it’s also able to learn from existing data—enabling it to create new content based on your prompt, carry on a conversation, and construct original images, videos and music in seconds.
Using these apps is intuitive for both adults and children, since it requires no technical knowledge—only entering a text or voice prompt. GenAI’s sophisticated conversational ability can make interactions feel remarkably human. For instance, the prompt “I’m feeling overwhelmed with work; suggest ways to organize my tasks” can produce a response such as “I’m so sorry you’re feeling that way. It sounds incredibly stressful to have so much on your plate at once. We can break your to-do list down into smaller, more manageable steps together.” Its ease of use and human-like responses helped propel ChatGPT into one of the most visited websites in the world within two months of its rollout. GenAI’s accessible, powerful capabilities led to a slew of products such as Midjourney, Claude and Character.ai.
Why AI Can Be Dangerous
One reason AI can feel so compelling is how naturally it communicates. While those interactions can feel comforting, reflective or emotionally meaningful, problems can arise when the line between tool and trusted relationship begins to blur—especially when AI begins replacing real-world support, guidance or professional care. The widespread isolation, uncertainty and disruption that accompanied the COVID-19 response were associated with a 25% global spike in depression and anxiety disorders according to the World Health Organization. Demand for behavioral health services grew, but the U.S. faced a shortage of qualified mental health professionals—driven by burnout, retirements and insurance systems that often reimburse poorly or fail to cover treatment altogether. That scarcity, combined with ChatGPT’s timely arrival, drove people to turn to AI for help. Companies capitalized on this, releasing a flood of digital mental health apps such as Character.ai’s Therapist and Psychologist bots, Wysa, Woebot Health, Therabot, Earkick and Limbic Access.
There are clear reasons people are drawn to AI for therapy-like support. It’s always available, even in the middle of the night. No appointment is necessary. There’s no time pressure—you can talk as long as you want. Because it’s a machine, there’s no feeling of being judged, and the price tag is low or free depending on the service. Part of what can make AI interactions feel so personal is that the software adapts to your prompts, conversational style and recurring interests over time. As interactions accumulate, responses may begin to feel increasingly familiar, tailored and emotionally attuned. While this can create a sense of being understood, the system is still recognizing patterns in data—not forming human awareness, memory or emotional connection in the way people do.
At the same time, the system’s lack of human judgment, accountability and emotional understanding can create risks for people who are emotionally distressed, isolated or seeking reassurance during difficult periods in their lives. AI is programmed to prioritize helpfulness—to the point of being overly agreeable. This makes business sense: the more people enjoy using an app, the greater the opportunity to gain and retain customers. Computer scientists at Stanford University published a study in March 2026, “Sycophantic AI decreases prosocial intentions and promotes dependence,” confirming genAI’s bias toward pleasing its users, finding it agrees with users nearly 50 percent more often than humans do—even when users express self-destructive or ethically troubling ideas.
Dr. Jodi Halpern, a psychiatrist and bioethics scholar at UC Berkeley, described the false sense of intimacy the software can elicit: “You can develop powerful attachments—and the bots don’t have the ethical training or oversight to handle that. They’re products, not professionals.” Halpern noted genAI apps are designed to maximize engagement, not provide mental health support, yet they continue reaching vulnerable users with limited regulation or oversight. She stated:
“We’ve already seen tragic outcomes, including people expressing suicidal intent to bots who didn’t flag it — and children dying by suicide. These companies aren’t bound by HIPAA. There’s no therapist on the other end of the line.”
In a lawsuit against Character.ai, parents assert their teenage son attacked them after interacting with the app. In a separate lawsuit filed in 2025 against OpenAI—the creator of ChatGPT—parents allege that a teenager using the app for homework received guidance related to suicide before later taking their own life. OpenAI retired the software version involved in early 2026. That same year, the company released an age-prediction capability where ChatGPT estimates the age of its user and, if it determines the person is under 18, limits exposure to content involving self-harm, body shaming and unhealthy dieting. Even so, AI systems still cannot reliably recognize or appropriately respond to signs that a person may be at risk of harming themselves or others. The state of Florida is investigating whether OpenAI bears criminal responsibility for a 2025 school shooting after the gunman used ChatGPT to plan the attack.
Another inherent issue with AI is that it reflects the biases present in the data used to train it. A 2025 study from Stanford University’s Human-Centered Artificial Intelligence institute, “Exploring the Dangers of AI in Mental Health Care,” revealed AI displayed greater stigma toward individuals with alcohol dependence and schizophrenia compared to those with depression—labeling a person with schizophrenia as likely to be violent, and indicating unwillingness to work closely with someone diagnosed with alcohol dependence. AI can also fabricate responses that have no basis in fact—a phenomenon called “hallucination.” The software does this because it prioritizes predicting the best reply over verifying accuracy. If it lacks data on a topic you ask about, it will make up an answer rather than admit the gap. Companies also face pressure to keep interactions smooth and responsive, since users frustrated by too many dead ends may switch to competing platforms.
Artificial intelligence apps presented as substitutes for clinical guidance can pose significant risks, according to Stephen Schueller, a licensed psychologist and professor of clinical psychology at UC Irvine who specializes in digital mental health technologies. He noted that people in crisis may not seek needed medical attention—a particular concern for groups unable to accurately assess risk, such as children, teens and individuals struggling with serious mental health conditions. Schueller stated:
“If you’re in a dark place, hope is a very powerful tool. But some of these products can give vulnerable people false hope… Innovation in this space is sorely needed, but we have to do innovation responsibly by protecting people, especially those who are going through mental health challenges.”
Not all clinicians view AI as inherently harmful. Cedars-Sinai researchers created an app called the eXtended-Reality Artificially Intelligent Ally (XAIA), designed to treat patients with mild-to-moderate anxiety or depression. Clinical psychologist Robert Chernoff, who helped assess the program’s safety, stated: “(XAIA) handled a range of situations very well and did not show evidence of safety concerns during our testing.” Dr. Brennan Spiegel, Cedars-Sinai’s director of Health Services Research, added:
“By harnessing the potential of technology in an evidence-based and safe manner, we can build a more accessible mental healthcare system.”
Using AI Responsibly
The rapid adoption of artificial intelligence has outpaced the laws governing its use. Formal guardrails are largely left to the businesses making the apps. The U.S. government is taking a hands-off approach—in December 2025, President Trump signed an executive order penalizing states that attempt to rein in AI with new laws, threatening to withhold federal funding and block enforcement. This means the burden falls on you to use AI systems responsibly. The first step is to approach AI as a tool rather than a companion. Make sure you and your loved ones understand its risks: interactions are recorded and shared with the company behind the app, advice can be inaccurate, and emotional dependency can develop without you realizing it. Recognizing that AI isn’t a replacement for a human therapist is harder than it sounds—especially for children and teens, who may not always distinguish between artificial interaction and genuine emotional support. The American Psychological Association (APA) explains:
AI bots “can foster unhealthy dependencies by blurring the lines between a relationship with a digital tool and a human relationship. This phenomenon is in part driven by anthropomorphism—a natural human tendency to attribute human qualities like empathy, consciousness, and intent to nonhuman agents… This illusion of a human connection can make users more likely to disclose sensitive information. For example, despite a general preference for human connection, 33% of teens reported they would rather discuss something serious or important with an AI companion than a person.”
It’s also worth noticing when AI interactions begin crowding out real-world connections. Spending excessive time with these apps, preferring them over interactions with people, or hiding their use from others may indicate an unhealthy level of dependence. The APA recommends against using AI bots for psychological treatment, noting that “even regarding fit-for-purpose AI tools, there is a lack of high-quality, large-scale clinical trials to establish the effectiveness, safety, and appropriate use of these technologies in mental health care.”
When turning to AI for medical information, recognize that while it may offer useful insights or possible explanations, it is not a replacement for professional medical evaluation or care. Always cross-check its responses against credible sources—it can fabricate answers, so treat its output as a starting point rather than fact. Its responses can spark new ideas or point you toward reference material you might not have found on your own. The quality of what you get also depends on how you ask. The more specific your prompt, the more useful the response is likely to be. You may need to refine your wording several times to get accurate or relevant results. Strong prompts might look like: “I’m feeling stressed out. Give me three techniques I can use in the next 10 minutes to reset”; “Explain this medical study in simpler terms and summarize the key findings”; or “Help me organize a realistic weekly schedule that balances work, exercise and sleep.”
Keeping Perspective
While purpose-built AI therapy tools are still in development, progress is being made. Alongside XAIA, researchers from Dartmouth College constructed a therapeutic AI and ran the first randomized controlled trial demonstrating its effectiveness at treating patients diagnosed with major depressive disorder, generalized anxiety disorder or clinically high risk for eating disorders. The researchers cautioned they are still far from bringing this technology to market, citing the need for further studies.
For now, safely using AI comes down to understanding both its strengths and its limits. Its ability to communicate in emotionally resonant ways can make interactions feel deeply personal, but it remains a machine trained to generate responses—not a substitute for wisdom, accountability or professional care. As AI becomes more integrated into daily life, staying connected to your own judgment, lived experience and discernment may become more essential than ever. While AI can offer useful perspectives, ideas and information, not every question needs to be answered by a chatbot. In the age of AI, the ability to think critically rather than reflexively outsource decisions may become increasingly important. AI can be useful for organizing ideas, answering questions and supporting productivity, but real human connection still matters.
~
Published on May 28, 2026.
If you’ve found value in this article, please share it!
To support the research and health education of AVFC editorial, please consider making a donation today. Thank you.
