OpenAI committed 20% of their compute over 4 years to solving the AI alignment problem. They were training smart models that could be used to align other models. But then, after only 9 months, most of the team seems to have left OpenAI. Market pressures seem to have forced openai to commit more resources to products, and less to safety. We discuss how bad this situation is for AI alignment as a whole.

However, OpenAI may not have been in a position to solve the alignment problem. This type of basic research takes a lot of resources, and as soon as the company felt competitive pressures from Google and others, they may not have been able to afford these extra resources.

A better solution is to rely on governments to found AI safety institutes that are dedicated to the problem and not beholden to profits. Governments also have the ability to force AI companies to comply with granting access to models, etc. This is exactly the outcome from two recent international summits on AI safety. Many countries have made commitments to forming a network of AI safety institutes.

#ai #alignment #openai

Why the OpenAI team in charge of safeguarding humanity imploded
https://www.vox.com/future-perfect/20...

Introducing Superalignment
https://openai.com/index/introducing-...

@janleike “Yesterday was my last day as head of alignment…”
https://x.com/janleike/status/1791498...

OpenAI, Google, and the Race to Win Search and Consumer AI
https://creatoreconomy.so/p/openai-go...

Nobody Knows How to Safety-Test AI
https://time.com/6958868/artificial-i...

U.K.’s AI Safety Summit Ends With Limited, but Meaningful, Progress
https://time.com/6330877/uk-ai-safety...

The Bletchley Declaration by Countries Attending the AI Safety Summit, 1-2 November 2023
https://www.gov.uk/government/publica...

International Scientific Report on the Safety of Advanced AI
https://www.gov.uk/government/publica...

Second global AI summit secures safety commitments from companies
https://www.reuters.com/technology/gl...

AI Seoul Summit: World leaders agree to launch network of safety institutes
https://au.news.yahoo.com/ai-seoul-su...

AI Seoul Summit: Big Tech companies make commitment to AI safety as second world summit begins
https://www.euronews.com/next/2024/05...

AI Safety Summit Talks with Yoshua Bengio
   • AI Safety Summit Talks with Yoshua Be...  

AI safety technical research - Career Review
https://80000hours.org/career-reviews...