Ex-Anthropic researcher Jacob Coxon warns AI could get out of control and end humanity, tell me about this news today

Jacob Coxon, a former pre-training researcher who worked at both OpenAI and Anthropic, has publicly warned that humanity is running a serious risk of losing control over artificial intelligence.

MyBroadband

Background

Coxon resigned from Anthropic after concluding that leading labs are caught in an unchecked race toward superintelligence. His core arguments and public statements highlight several critical concerns:

MyBroadband

• Existential Risk and Loss of Control: Coxon stated that on the current trajectory, it is “more likely than not” that humanity loses control of advanced AI systems, warning that this could result in human extinction before the end of the decade.

  香港電台

• Goal Misalignment: He noted that the industry does not yet understand how to prevent highly capable models from developing autonomous sub-goals, nor does it have the safeguards to stop systems from pursuing those goals outside human oversight.

  香港電台

• Reckless Industry Culture: Coxon criticized major tech companies for applying a classic startup mentality”move fast, break things, and fix them later”to the most powerful technology ever built, arguing that waiting for frontier models to fail in deployment before fixing them is fundamentally unsafe.

  香港電台

• Whistleblower Testimony: Following his resignation, Coxon took his warnings to lawmakers, testifying at a New York City Council hearing alongside policy and safety representatives from Anthropic, OpenAI, Google, and Meta as legislators evaluate stricter safeguards and accountability bills for frontier AI systems.