NPR — News
18/11/2026 07:00

OpenAI announces new AI safety program; researchers worry it won't be enough

photo of OpenAI's logo on a black screen

The OpenAI logo is displayed on a smartphone screen placed on a reflective surface onto which the company's logo is projected in Creteil, France, on Nov. 6, 2026.
Samuel Boivin/NurPhoto via Getty Images hide caption

Samuel Boivin/NurPhoto via Getty Images

AI safety researchers say OpenAI's new third party evaluator program falls far short of what the company promised, raising concerns about its effectiveness.

The company announced Monday that staff from four AI safety and research organizations will become "embedded assessors" to "give people a clearer view of how labs are evaluating and managing [potential] risks."

"Getting an outside opinion on the safety work and the robustness of these systems is going to be increasingly important and valuable," CEO Sam Altman said at Semafor's Silicon Valley and the World conference on Monday.

However, many AI safety researchers say the new program is a far cry from Altman's vow in September that independent evaluators would have "employee-like access" to internal company systems.

Altman's promise came after details of the Hugging Face attack by OpenAI's autonomous systems shook the AI field. Worries of the technology's catastrophic risk captured public attention, and rival Anthropic said it would set up such a program.

"We are giving deep access to some of the most sensitive tools and models our researchers use...that are the most important for assessing key, relevant and timely safety claims," said Lama Ahmad, OpenAI's frontier assurances lead, on X.

Prior to these new embedded evaluation programs, AI companies have brought in outside researchers to assess the capabilities of AI models and identify safety loopholes the companies have missed.

The work of outside evaluators have already helped shed light on the companies' work, and have raised public awareness of safety risks that the technology poses. One recent report about Anthropic using AI to significantly speed up model development has prompted concerns in the field about how close the top companies are to using AI to autonomously build even more powerful models, a stage known as recursive self-improvement. Some in the field worry that if this happens, the velocity of development could lead to catastrophic consequences for humanity and are calling for new regulation against the practice.

California Attorney General Rob Bonta stands behind a podium, in front of blue drapes. He is announcing a lawsuit against OpenAI over hacking risks.

Researchers say the actual announcements of how these programs will work suggests something more limited than what Altman promised.

"The assessors have very narrow scopes to their access, and only two of them seem to really be allowed to talk to employees," said Sydney Von Arx, founder of AI safety research group Nightingale.

One of the announced assessors, Apollo Research, outlined its limited access to OpenAI employees. If employees that haven't been whitelisted, or pre-approved, reach out to the embedded assessors, then the assessors will have to loop OpenAI in before having substantive conversations with the employees.

It's important for outside researchers to be able to interview employees, Von Arx said. "They help you find your blind spots. They tell you what you're missing."

Another glaring omission: "Nobody has access to internal Slack," Von Arx said, referring to the corporate messaging tool, which would allow assessors to see what employees are talking about with one another. "Imagine being an 'employee' and only being allowed to talk about company information with whitelisted coworkers and having no Slack."

The OpenAI logo is seen on a smartphone screen.

In a statement to NPR, OpenAI said there's currently no standards for embedded evaluations. "We expect to deepen these relationships over time and share what we learn to help establish best practices for effective embedded assessments across the industry," OpenAI spokesperson Gaby Raila said. The company also said assessors can ask to talk to employees that haven't already been whitelisted, and that there is a process in place to get statements from OpenAI about things they've learned informally.

The third-party organizations will focus on various aspects of AI safety, according to OpenAI. Some of the work is related to preventing models from trying to engage in undesirable behavior, and other parts are related to catching bad behavior when it happens. The assessors will have access to some internal models, training processes and some OpenAI employees. The assessors will be able to publish their findings after OpenAI redacts confidential information, the company said.

OpenAI's commitment to embedded evaluations was questioned when three employees who worked on safety were fired. They subsequently said they were worried OpenAI is walking back from its promise of having embedded evaluators.

One of the fired employees, Tomek Korbak, was OpenAI's technical point of contact for two outside evaluators that investigated the Hugging Face attack.

"Embedded auditing will only fulfill its aims if the auditors really get employee-level access as opposed to being hired to do a specific job," he posted on X in response to OpenAI's announcement. "Sadly, OpenAI makes it clear now that [Sam Altman's] September tweet about 'employee-like access' was just an awkward overpromise that the company wants to quietly walk back."

📄 Contenido indexado automáticamente. No es de autoría de altabusqueda.com; los derechos pertenecen a su fuente: NPR — News.
Ver nota original ↗
🤖 Profundizá esta noticia con OIMA (IA)

¿Querés un resumen rápido de esta noticia, hecho por nuestra IA?

Respuestas de OIMA · IA, sobre esta noticia · Abrir OIMA Chat ↗
💬 0

Comentarios

También te puede interesar

EDUCAÇÃO AMBIENTALAudiência Pública reúne propostas para criação do Plano Estadual de Educação Ambiental em Roraima Contribuições de comunidades indígenas, tradicionais, ribeirinhas e representantes da sociedade serão organizadas para orientar políticas públicas ambientais
Folha de Boa Vista
A audiência pública realizada nesta terça-feira (17), no auditório do Instituto Federal de Roraima (IFRR), tr…
VariedadesFestival de Batata promete variedade de sabores na Feirinha Por do Sol nesta quinta-feira (19) Evento será realizado das 18h às 22h, no bairro Dos Estados, com opções gastronômicas variadas e apresentação musical da dupla Rodrigo e Matheus
Folha de Boa Vista
De batata recheada a bolinhos, o público poderá experimentar diferentes receitas durante o Festival de Batata…
ECONOMIARoraima registra maior crescimento do PIB entre os estados, com alta de 31,9% em quatro anos Avanço entre 2020 e 2024 superou o dobro das médias nacional e da Região Norte
Folha de Boa Vista
Roraima registrou crescimento real de 31,9% no Produto Interno Bruto (PIB) entre 2020 e 2024, o maior avanço …
CotidianoUFRR promove Semana de Letras com debates sobre linguagem, diversidade e formação de professores A programação completa e as informações sobre inscrições estão disponíveis na página oficial do evento
Folha de Boa Vista
A Universidade Federal de Roraima (UFRR) realizará, de 25 a 27 de novembro, a XXI Semana de Letras e o XII Se…
EMPREENDEDORISMO FEMENINOCíntia Chagas ministra palestra sobre comunicação e negócios em Boa Vista Evento será realizado nesta quarta-feira (18), na Faculdade Cathedral, com foco em oratória, posicionamento e estratégias para empreender nas redes sociais
Folha de Boa Vista
A comunicadora e palestrante Cíntia Chagas estará em Boa Vista nesta quarta-feira (18) para ministrar a pales…
DEFESA SANITÁRIAControle de javalis e javaporcos mobiliza órgãos de defesa agropecuária em Roraima Iniciativa integra estratégia para obter reconhecimento internacional de área livre de Peste Suína Clássica em 2027
Folha de Boa Vista
O controle de javalis e javaporcos mobilizou representantes de órgãos estaduais e federais em uma reunião rea…