AI Safety Study Group

[wdoug]

Why

Machine Learning and Artificial Intelligence are already extremely powerful tools that are having a massive impact on society. These tools are only becoming more powerful and in the future could potentially even lead to Artificial General Intelligence (AGI) or Superintelligence. While there is amazing potential for these technologies to positively impact the future there is also a huge risk that they could have an enormous negative impact (including possible extinction) either through accidental misuse or through malicious intent. Meanwhile, despite the potential risks, AI Safety is a very neglected area of research which means that any work done on the topic can have an outsized impact. Taking this into account, 80,000 Hours estimates that working on AI Safety is one of the most effective ways to have a positive impact on the world.

What is the idea

I'd like to start a study group to learn more about the issues and see if we might be able to contribute to any of the directions of research.

Potential Partner(s)?

• For local contacts, we might want to reach out to:
  • DU Senior Research Fellow Heather Roff
  • CU Senior Research Associate Seth Herd
• Outside Colorado some organization options are:
  • OpenAI
  • The Machine Intelligence Research Institute (MIRI)
  • The Future of Humanity Institute (FHI)
  • Global Catastrophic Risks Institute (GCRI)
  • The Center for the Study of Existential Risk (CSER)
  • Center for Human-Compatible AI (CFHCA)

Additional Background Context

Further reading:

• Why AI Safety
• No Time Like the Present for AI Safety Work

Details

Champion: [To be filled out during exploration stage]
Repo: [To be filled out during exploration stage]
Waffle Board: [To be filled out during exploration stage]

[tylerperkins]

I just watched an excellent talk about these issues and the state of the art given by Max Tegmark (MIT) just three weeks ago. It's Max Tegmark - How Far Will AI Go? Intelligible Intelligence & Beneficial Intelligence. He proposes three goals to "win the wisdom race":

• Invest in AI safety research
• Ensure that AI-generated wealth makes everyone better off
• Ban lethal autonomous weapons

He also mentions an excellent resource, Victoria Krakovna's page, Deep Safety.

[provmusic]

I would be very interested in joining a conversation centered around this. I've spent a quite a bit of time pondering this space. My current peak area of interest is the effects Ai can, and is currently having on culture.