Debug Headers with GET

November 17th, 2020
tech
HTTP offers a convenient way to download only the headers: send a HEAD request:
$ telnet www.example.com 80
Trying 93.184.216.34...
Connected to www.example.com.
Escape character is '^]'.
HEAD / HTTP/1.1
Host: www.example.com

HTTP/1.1 200 OK
Content-Encoding: gzip
Accept-Ranges: bytes
Age: 325063
Cache-Control: max-age=604800
Content-Type: text/html; charset=UTF-8
Date: Tue, 17 Nov 2020 02:29:50 GMT
Etag: "3147526947"
Expires: Tue, 24 Nov 2020 02:29:50 GMT
Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
Server: ECS (dcb/7F82)
X-Cache: HIT
Content-Length: 648
Of course you wouldn't usually manually type into telnet, you'd use something like curl:
$ curl -I http://www.example.com
HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 326121
Cache-Control: max-age=604800
Content-Type: text/html; charset=UTF-8
Date: Tue, 17 Nov 2020 02:47:38 GMT
Etag: "3147526947"
Expires: Tue, 24 Nov 2020 02:47:38 GMT
Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
Server: ECS (dcb/7EC9)
X-Cache: HIT
Content-Length: 1256
It's defined in RFC 7231:
The HEAD method is identical to GET except that the server MUST NOT send a message body in the response (i.e., the response terminates at the end of the header section). The server SHOULD send the same header fields in response to a HEAD request as it would have sent if the request had been a GET, except that the payload header fields MAY be omitted.

Unfortunately, HEAD is a trap. When you are trying to debug strange server behavior, it is much safer to send GET requests and throw away the body (ex, ex). Not only is "SHOULD" just a recommendation, but even if this were a "MUST" you could bet some servers would mishandle it. Counterfactuals are hard!

While differences are rare, always debugging by requesting the body like a normal client would, and then discarding it, means one fewer way that your debug request differs from a real one:

$ curl -sS -D- -o/dev/null http://www.example.com
HTTP/1.1 200 OK
Accept-Ranges: bytes
Age: 326124
Cache-Control: max-age=604800
Content-Type: text/html; charset=UTF-8
Date: Tue, 17 Nov 2020 02:47:41 GMT
Etag: "3147526947"
Expires: Tue, 24 Nov 2020 02:47:41 GMT
Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
Server: ECS (dcb/7EC9)
Vary: Accept-Encoding
X-Cache: HIT
Content-Length: 1256

Going farther in the same direction, it's even better to start with "Copy as cURL":

And then add the -sS -D- -o/dev/null to get the headers if that's all you want.

Comment via: facebook, lesswrong

Recent posts on blogs I like:

Book Review: Why They Do It

my quest to get my readers not to commit fraud

via Thing of Things March 3, 2025

Breakfast Over February Break

Over February break I made breakfast for me and Nora three days in a row. Normally, my dad makes me, Nora, and Lily breakfast, but my dad and Lily were on a trip and my mom wasn't up yet so I was basically the only one in the house who was awake and cou…

via Anna Wise's Blog Posts March 1, 2025

2024-25 New Year review

This is an annual post reviewing the last year and setting intentions for next year. I look over different life areas (work, health, parenting, effectiveness, travel, etc) and analyze my life tracking data. Overall this was a pretty good year. Highlights …

via Victoria Krakovna January 15, 2025

more     (via openring)