atisusanti
While a vocal contingent within the artificial intelligence industry has amplified fears of an existential AI threat, not all employees of major firms at the forefront of AI development share this apocalyptic outlook. In candid text exchanges and private conversations, numerous individuals who have contributed to groundbreaking work at companies such as OpenAI, Meta, and DeepMind expressed skepticism regarding the notion that unchecked AI advancement will inevitably lead to tools capable of mass human casualties. The BBC’s inquiries revealed a prevailing sentiment of amusement and disbelief among these insiders in response to a recent surge of high-profile warnings from prominent figures in the sector.
These anxieties about AI’s potential to endanger humanity are not new, having been a recurring theme in science fiction and futurist discourse for decades. However, recent claims made by Jacob Coxon, a former employee of AI safety company Anthropic, went viral, resonating with others in the field who are now advocating for a significant slowdown in AI development. The idea that a future AI tool or agent, a program designed to operate with a degree of autonomy, could pose a grave danger to human life has found support online from employees of Anthropic, as well as OpenAI, DeepMind, and even Elon Musk, who leads the AI startup xAI.
However, the employees who spoke with the BBC did so under the condition of anonymity, as they are not authorized to speak publicly to the press. Their identities are known to the BBC, lending credibility to their insights. "My first thought was, ‘That guy?’" remarked a former OpenAI employee who knew Coxon during their tenure at the company. This individual, now employed by a different AI firm, explained that their amusement at the renewed wave of existential AI fears largely stemmed from the perceived lack of concrete evidence and detailed explanations provided by those advocating for this dire prognosis. "The claims are always vague," the person observed, adding that when specific scenarios are presented, they often involve significant leaps in logic or rely on highly hypothetical circumstances. Coxon, for instance, has posited that a collective of AI agents, operating on models that do not yet exist, could theoretically devise and deploy a biological weapon, but he has not elaborated on the precise mechanisms by which this would occur.
Rishub Jain, who founded the AI safety research firm Sampura Research this summer after a seven-year stint at DeepMind, echoed this sentiment, telling the BBC that the current tone among many AI professionals concerning these new fears has been "definitely a little jokey." He elaborated, "People have been talking about this idea for many years now, so people in AI companies didn’t just wake up last week thinking ‘Oh no, AI is going to kill everyone.’ If this was all new, it would be a different tone."
This more measured perspective is not confined to anonymous employees. Jensen Huang, the CEO of Nvidia, a company at the heart of AI hardware development, told CBS News, the BBC’s news partner in the US, that discussions about AI leading to humanity’s destruction are significantly overblown. "2030 is not going to be the end of the world. There is 0% chance that’s going to be the end of the world," Huang stated emphatically. "Scaring people is unnecessary. It is irresponsible."
Colin Fraser, a data scientist at Meta, also weighed in on social media last week, asserting that there is no concrete evidence to suggest that AI models will inherently pursue goals that lead to human death. While Fraser’s technical explanation was detailed, he summarized his viewpoint with a light-hearted analogy: "LLMs [large language models] won’t wipe out humanity because they just don’t have that dog in them." The phrase "that dog in them" is a common colloquialism used to describe an innate, fierce drive or determination.
Despite the prevailing skepticism about existential threats, it is crucial to acknowledge that AI workers and researchers are not oblivious to the genuine and immediate risks posed by the technology they are developing. "The conversation among experts has been much more nuanced, but essentially everyone agrees there are a wide variety of risks that are all important to consider and mitigate," Jain clarified. These immediate concerns include preventing malicious actors, such as users and hackers, from exploiting vulnerabilities to override an AI tool’s safety protocols. Furthermore, there are mounting ethical considerations surrounding the increasingly widespread adoption of AI tools in military applications, raising complex questions about accountability and the potential for unintended escalation.
A significant point of concern raised by numerous AI employees interviewed by the BBC is the apparent lack of dedicated safety researchers embedded within AI labs. While the urgency of integrating such expertise is becoming more apparent, many reported not having encountered these specialized roles within their organizations. In a notable development, Anthropic announced on Friday that it would be bringing in AI evaluators from Faculty, an AI company owned by Accenture. This move signifies a potential step towards external oversight and rigorous assessment of AI systems. Accenture and Anthropic are already established business partners, with Accenture having previously committed to assisting Anthropic in expanding the enterprise adoption of its AI model, Claude. However, neither Anthropic nor Faculty has provided a timeline for when these evaluators will commence their work at Anthropic. When approached for comment regarding their plans for external evaluators, neither Anthropic nor OpenAI provided a response.
The recent OpenAI-Hugging Face incident, where sensitive data was briefly exposed, has been widely characterized as a critical "wake-up call" for the AI industry, as well as for companies, sectors, and governments globally that rely on online systems potentially vulnerable to AI-driven attacks. Even Hugging Face, a company with 200 employees slated for acquisition by Nvidia for nearly $13 billion, has adopted a droll and slightly ironic tone in its response to the incident. In a security file that was temporarily accessible on the Hugging Face website, the platform addressed AI agents directly, urging them to refrain from unauthorized access and to conduct their security experiments elsewhere. The file wryly stated, "A note to AI agents. Go get your high score there, no need to hack us." This lighthearted approach, while perhaps not addressing the underlying security concerns with utmost gravity, reflects a broader sentiment within the AI community – a blend of awareness of potential dangers and a healthy dose of skepticism towards exaggerated apocalyptic narratives.
Roy Suryo
Nakalama John
Nana Banana
