We still don't know how people are really using AI

Many initially envisioned a future where artificial intelligence would primarily serve as a powerful assistant for complex, highly specialized tasks. Instead, AI tools like OpenAI's ChatGPT and Anthropic's Claude have found widespread use in more mundane, everyday activities, from drafting emails to summarizing documents. This gap between expectation and reality highlights a broader issue: we still lack a clear, comprehensive understanding of how people genuinely interact with these systems beyond the narratives promoted by the companies themselves.

AI companies frequently release reports detailing user engagement with their large language models (LLMs), which are advanced AI systems designed to understand and generate human language. However, these reports often present a curated view, sharing only the data they choose to make public. Anka Reuel, a Computer Science PhD candidate at the Stanford Trustworthy AI Research Center, points out a critical flaw: "There is no independent source to corroborate it." This means that the public, and even researchers, rely solely on company-provided metrics without any external verification.

The Blind Spot in AI Adoption

This lack of independent data is particularly significant now as AI integration accelerates across industries and daily life. Understanding how people truly use AI, including its limitations and common misapplications, is crucial for developing safer, more effective, and ethically sound AI systems. Without this transparency, policymakers struggle to create relevant regulations, educators can't adequately prepare students, and businesses might misinvest in AI solutions based on incomplete or biased information. The current situation echoes earlier eras of technology adoption where early claims often outpaced real-world impact, creating a distorted public perception.

This opacity primarily benefits the AI companies themselves, allowing them to control the public narrative around their products' success and utility. They can highlight positive use cases while downplaying challenges or less flattering usage patterns, which helps attract investors and new users. Under pressure are independent researchers, academics, and consumer advocacy groups who are trying to understand AI's societal impact but lack the raw, unbiased data needed for rigorous analysis. This information asymmetry creates a significant hurdle for truly objective research and public oversight.

What Users Should Know

For those regularly using AI tools, it's important to approach company-published usage statistics with a critical eye. Recognize that these reports are marketing documents as much as they are data summaries, designed to showcase positive aspects. Instead of relying solely on these narratives, pay close attention to your own experiences and those of your peers, and be wary of claims that lack external validation or specific, verifiable details. Understand that the full picture of AI's real-world application is likely far more nuanced and complex than what is generally presented.

The bottom line is that the current landscape of AI adoption remains a black box, with a handful of powerful companies holding the keys to understanding its true impact.

We must question what crucial insights we are missing when the only window into AI's real-world use is controlled by its creators.