Technical Distance to AGI

July 11th, 2017
airisk, ea
I think the largest piece of the disagreement between ML people over how to prioritize AI risk may turn out to be: how likely is it that we get AGI soon that looks a lot like systems we have now?

Specifically, here are two things I'd guess most of the people I've talked to would agree with (though I haven't run this by any of them):

  • If there were a 10% chance of current systems getting us to AGI within the next 10 years, then the kind of questions posed in Concrete Problems in AI Safety (pdf) would be very important from a safety perspective.

  • If the chance were less than 1%, then while those are still interesting research questions that look useful for today, they're not altruistically important.

What I'm planning to do next is write back to all of the people I've been talking to in order to find out:

  • Does this actually capture their views?
  • What do they put the chance at, and why?

Before I reach out to people again, though, I wanted to post this here for feedback. Any thoughts?

(This is a lot like the more general question of AI timelines. For that, Open Phil's What Do We Know about AI Timelines? gives a good overview and summary, with Katja Grace and others' recently published When Will AI Exceed Human Performance? Evidence from AI Experts (pdf) giving more recent information. But (a) that's about time and not about technical distance (would AGI look like what we have now) and (b) my main takeaway from the surveys is that people are giving pretty off-the-cuff answers without thinking very hard.)

Referenced in:

Comment via: google plus, facebook, substack

Recent posts on blogs I like:

Parenting standards and the village

Cartoons Hate Her has written about the village nobody wants. In principle, parents usually want a community to help take care of their kids: someone to bring meals when they get sick, keep an eye on their children while they play, and talk to about paren…

via Thing of Things December 19, 2025

Opinionated takes on parenting

This post is a collection of parenting takes that sometimes go through my head, based on my experience raising our two boys (5 and 2 years old). All of this is based on my experience and might not apply to others (see the law of equal and opposite advice)…

via Victoria Krakovna December 16, 2025

How to Make a Christmas Wreath

Yesterday, I made a Christmas wreath. Here's how to make one. First, find an evergreen tree near your house. Clip off a few branches from the tree. Try to have as many leaves or needles on the branches as possible. Next, bring them home. What I usu…

via Anna Wise's Blog Posts December 6, 2025

more     (via openring)