Zstd Window Size

At work we've recently been using zstd as a better-compressing alternative to gzip, and overall I've been pretty happy with it. A minor documentation gripe, though, is that the behavior around multithreaded compression is a bit unclear. I understand it's chunking the work and sending chunks to different threads to parallelize the compression process, and this means that I should expect to see better use of threads on larger files because there are more chunks to spread around, but what is the relationship?

When I look in man zstd I see that you can set -B<num> to specify the size of the chunks, and it's documented as "generally 4 * windowSize". Except the documentation doesn't say how windowSize is set.

more...
Spending on Ourselves

The Effective Altruism community has encouraged a range of different approaches to doing good over time. Initially there was more focus on frugality as a way to increase how much you could donate, which was mostly supplanted by emphasis on earning more. In late 2015 this started to shift towards doing things that are directly useful, which accelerated in 2021. Then the market fell in 2022, FTX turned out to be a fraud, and there haven't been new donors near the scale of Open Phil / Good Ventures. Among many changes, people are thinking more about frugality again: the less you can live on, the more you can stretch a given amount of funding. [1]

To encourage myself to live more frugally and to give an example of what I thought was a pretty fulfilling life at relatively low cost for the US, I used to calculate numbers for how much we spent on ourselves. This included housing, food, transportation, medical, etc but not donations, taxes, or savings. At one point there were some news stories comparing our spending to our income, and it was nice to have a simple number to point at.

I was thinking it might be nice to start calculating these numbers again, but when I looked back at why I stopped it's mostly that it's actually a pretty tricky accounting question and I'm not sure there are ways to draw the lines that make much sense. For example:

more...
AI Advances and Detection Strategy

Cross-posted from my NAO Notebook.

This is an internal strategy note I wrote in November 2024 that I'm making public with some light editing.

In my work at the NAO I've been thinking about what I expect to see as LLMs continue to become more capable and get closer to where they can significantly accelerate their own development. I think we may see very large advances in the power of these systems over the next few years.

I'd previously thought that the main impact of AI on the NAO was through accelerating potential adversaries, and so shorter timelines primarily meant more urgency: we needed to get a comprehensive detection system in place quickly.

I now think, however, that this also means the best response involves some reprioritization. Specifically, AI will likely speed up some aspects of the creation of a detection system more than others, and so to the extent that we expect rapid advances in AI we should prioritize the work that we expect to bottleneck our future AI-accelerated work.

more...
British and American Connotations

As an American who works with some people who speak British English, the language differences are usually not a problem. Most words mean the same thing, and those that don't are usually concrete enough not to cause confusion (ex: lift, flat, chips). The tricky ones, though, are the ones that differ primarily in connotations. For example:

  • In American English (AE), "quite" is an intensifier, while in British English (BE) it's a mild deintensifier. So "quite good" is "very good" in AE but "somewhat good" in BE. I think "rather" works similarly, though it's less common in AE and I don't have a great sense for it.

  • "Scheme" has connotations of deviousness in AE, but is neutral in BE. Describing a plans or system as a "scheme" is common in BE and negative in AE.

  • "Graft" implies corruption in AE but hard work in BE.

These can cause silent misunderstandings where two people have very different ideas about the other's view:

more...
Host Keys and SSHing to EC2

I do a lot of work on EC2, where I ssh into a few instances I use for specific purposes. Each time I did this I'd get a prompt like:

$ ssh_ec2nf
The authenticity of host 'ec2-54-224-39-217.compute-1.amazonaws.com
(54.224.39.217)' can't be established.
ED25519 key fingerprint is SHA256:...
This host key is known by the following other names/addresses:
    ~/.ssh/known_hosts:591: ec2-18-208-226-191.compute-1.amazonaws.com
    ~/.ssh/known_hosts:594: ec2-54-162-24-54.compute-1.amazonaws.com
    ~/.ssh/known_hosts:595: ec2-54-92-171-153.compute-1.amazonaws.com
    ~/.ssh/known_hosts:596: ec2-3-88-72-156.compute-1.amazonaws.com
    ~/.ssh/known_hosts:598: ec2-3-82-12-101.compute-1.amazonaws.com
    ~/.ssh/known_hosts:600: ec2-3-94-81-150.compute-1.amazonaws.com
    ~/.ssh/known_hosts:601: ec2-18-234-179-96.compute-1.amazonaws.com
    ~/.ssh/known_hosts:602: ec2-18-232-154-156.compute-1.amazonaws.com
    (185 additional names omitted)
Are you sure you want to continue connecting (yes/no/[fingerprint])?

The issue is that each time I start my instance it gets a new hostname (which is just derived from the IP) and so SSH's trust on first use doesn't work properly.

Checking that "185 additional names omitted" is about the number I'd expect to see is ok, but not great. And it delays login.

I figured out how to fix this today:

more...
Risers for Foot Percussion

The ideal seat height for foot percussion is significantly higher than a typical chair. I've tried a few things over the years:
  • Stacking chairs: works great, until you come to a venue without any stackable chairs.

  • Drum stool: good (unless you buy a very cheap wobbly one like I did) but heavy and not good for flying with.

  • Adjustable booster: ok, though unavoidably bulky and the one I made is also too heavy for flying.

At Hashdance Weekend Kelsey had a set of furniture risers she let me borrow, and I liked them a lot! I decided to get a pair.

They do slide around a bit, though, because they're hard plastic on the bottom:

more...
More Posts