AI Advances and Detection Strategy

April 19th, 2025
airisk, nao

Cross-posted from my NAO Notebook.

This is an internal strategy note I wrote in November 2024 that I'm making public with some light editing.

In my work at the NAO I've been thinking about what I expect to see as LLMs continue to become more capable and get closer to where they can significantly accelerate their own development. I think we may see very large advances in the power of these systems over the next few years.

I'd previously thought that the main impact of AI on the NAO was through accelerating potential adversaries, and so shorter timelines primarily meant more urgency: we needed to get a comprehensive detection system in place quickly.

I now think, however, that this also means the best response involves some reprioritization. Specifically, AI will likely speed up some aspects of the creation of a detection system more than others, and so to the extent that we expect rapid advances in AI we should prioritize the work that we expect to bottleneck our future AI-accelerated work.

One way to plan for this is to imagine what would be the main bottlenecks if we had a far larger staff. Imagine if each senior person had AI support equivalent to all the smart junior people they could effectively manage. Or even (but my argument doesn't depend on this) AI systems that are as capable as today's experienced researchers. I think if in a year or two we found ourselves in this situation we would wish that:

  • We had collected a lot more data, because with a very large virtual computational staff future-AI-assisted-NAO can wring insights out of data far more efficiently than present-NAO.

  • We had started large-scale collection sooner, because, even if AI accelerates sequencing's price decreases and we can collect a lot more data in the future, it can't give us historical data.

  • We had a lot more partnerships for bringing in samples and data, because these take real-human time to scale up.

While I don't think this is the only way things could play out, I think it's likely enough that we should be taking these considerations very seriously in our planning.

April 2025: since initially drafting this we've started an ambitious effort to scale up our pilot system.

Referenced in: Personal AI Planning

Comment via: facebook, lesswrong, mastodon, bluesky

Recent posts on blogs I like:

Book Review: Daygame Mastery

the dark, forbidden, secret truth that women like men with boundaries

via Thing of Things April 18, 2025

Which Came First, the Chicken or the Egg?

When I thought about this question it was really hard to figure out because the way it's phrased it's essentially either a chicken just pops into existence, or an egg just pops into existence, without any parent animals involved. I thought about t…

via Lily Wise's Blog Posts April 13, 2025

Advice for time management as a manager

have accurate expectations of yourself • prioritize ruthlessly • unemploy your future self • a five-step “help, I’m overwhelmed” checklist • carve out focused time

via benkuhn.net April 1, 2025

more     (via openring)