Gabriela Ramos, co-chair of the working group focused on social inequality and financial disclosure, has held notable positions including Assistant Director-General for Social and Human Sciences at UNESCO, where she oversaw the drafting of the “Recommendation on the Ethics of Artificial Intelligence.” Emilia Stoymenova-Douh, an electrical engineering professor at the University of Ljubljana, serves on the board of Globethics and previously held the role of Slovenia’s Minister for Digital Transformation.
A significant shift in the U.S. approach to technology governance was marked by President Donald Trump’s directive appointing Treasury Secretary Steven Mnuchin to oversee artificial intelligence (AI). This move stands in contrast to both the AI industry and regulatory bodies, which have largely maintained a narrative suggesting that regulation stifles innovation. Amidst the geopolitical race to outpace China through advanced AI, this new focus from the Treasury Department introduces a welcomed institutional perspective on the issue.
With national security, cybersecurity, and financial stability at stake, the regulatory framework surrounding AI must evolve from the hands-off, free-market philosophy that has characterized U.S. policy thus far. Mnuchin’s recent proposal signals this shift clearly. According to Bloomberg, the Trump administration is considering establishing an agency akin to the Financial Industry Regulatory Authority (FINRA), a self-regulatory organization funded by industry that oversees brokers and dealers with Congressional authorization under the Securities and Exchange Commission.
For the first time, a U.S. administration is seriously contemplating a dedicated body aimed at safeguarding public interests by evaluating advanced AI models prior to their launch. However, the proposed agency resembles FINRA, raising concerns over conflicts of interest, as it would be governed by the same industry it aims to regulate. This model, long supported by leaders such as Demis Hassabis, CEO of Google DeepMind, is flawed at its core: it is ill-equipped to identify systemic risks, as was painfully evident during the 2008 financial crisis, which cost the U.S. economy an estimated $22 trillion and led to reforms including the establishment of the Financial Stability Oversight Council.
Ultimately, an agency modeled after FINRA is unsuitable for overseeing AI. What is truly needed is for governments to build independent expertise in AI, relying on public oversight mechanisms that were introduced following the emergence of generative AI. One such measure came from President Joe Biden, who mandated that developers of leading AI models share their safety testing results with the federal government before making their systems public. In the same year, U.K. Prime Minister Rishi Sunak convened an AI Safety Summit and established a mandate for the first international report on AI safety, paving the way for the creation of AI safety institutes.
Following the unveiling of the Mythos model from Anthropic—currently the most powerful AI system—the Trump administration issued its own executive order. This order, however, is limited because it makes pre-launch assessment voluntary. Such an approach disrupts the necessary balance: increasingly powerful AI systems require independent public institutions that possess both the expertise and authority for effective oversight, neither of which is provided by Trump’s executive order.
This disparity is also reflected in how governments address AI-related risks. Often viewed primarily as a national security issue, resources allocated to understanding its impacts on employment, healthcare, education, and other facets of daily life remain disproportionately low. Even the European Union’s AI Act, despite requiring assessments of societal risks, focuses more significantly on cybersecurity.
One notable exception is the growing recognition of the risks posed by AI to children online, which has prompted legislative and judicial measures in several countries, including the U.S. However, more extensive societal consequences—including cognitive development, women’s economic opportunities, and privacy—have yet to elicit a cohesive strategic response.
Such skewed priorities stem from framing AI development as a geopolitical race. However, leading in technology means little if it comes at the expense of worker health, social resilience, and the ability for individuals to lead fulfilling lives. A nation that develops highly efficient AI systems while neglecting workforce skills and allowing public discourse to be marred by misinformation ultimately achieves nothing worthy of praise.
Addressing these societal challenges does not contradict competitiveness; rather, it is essential for both national security and economic viability. Success in the AI era will hinge not only on the capabilities of a nation’s premier AI models but also on the strength of its institutions that safeguard citizens’ well-being—its schools, healthcare systems, social services, and independent judiciary.
These institutions, however, now struggle to keep pace with technological advancements. In education and labor markets, the OECD has found that the benefits of AI are unevenly distributed, exacerbating existing inequalities: students who are already advantaged have better access to AI-supported learning, while skilled workers adapt more easily compared to those with fewer qualifications.
Similar trends can be observed in healthcare, social services, and the justice system. As decision-making is increasingly delegated to AI, challenging errors becomes more difficult, particularly for those least able to advocate for themselves. The AI Hallucinations Database managed by Damien Charlotte from HEC Paris has documented over 1,800 instances globally where judges encountered fabricated evidence created by AI, often presented by litigants unable to afford legal representation.
The implications for judicial integrity are profound. Courts rely on the ability to verify facts and evidence, yet AI complicates this task, disproportionately burdening those already in vulnerable positions. In July, for instance, India’s Supreme Court annulled two judgments based largely on nonexistent precedents, warning that reliance on unverified AI-generated rulings compromises the rule of law.
While many societal effects of AI cannot be evaluated pre-launch in the same manner as cybersecurity risks, this underscores the need for ongoing public oversight. Rather than creating another industry-managed agency, what the United States and other nations truly require is an equivalent to the Financial Stability Oversight Council for AI: a regulatory body tasked with identifying systemic risks that threaten education, health, labor markets, and the judicial system. We have already established such entities to protect financial stability and national security; it is time to create equivalents focused on safeguarding the institutions essential to society.
