The AI safety bottleneck: Save the world or get the bag

What happened: Rishub Jain left Google DeepMind's safety team to cofound nonprofit Sampura Research, saying he wanted autonomy to pursue deprioritized AI alignment research. Safety researchers have also departed OpenAI and Anthropic. Key numbers: Open AI safety roles rose 91% on LinkedIn over the past year. Why it matters: The field's constraint is talent, not funding, and safety nonprofits and startups must compete with large pay packages at major labs. Researchers are divided on whether they are more effective inside companies or outside.

Rishub Jain left Google DeepMind's safety team to cofound nonprofit Sampura Research, saying he wanted autonomy to pursue deprioritized AI alignment research. Safety researchers have also departed OpenAI and Anthropic.

The field's constraint is talent, not funding, and safety nonprofits and startups must compete with large pay packages at major labs. Researchers are divided on whether they are more effective inside companies or outside.