You can use curl to see the http headers from an email link, or webpage is going to send you to a known bad page. If you want to see the contents of the page, without risk of infection (look for base64 encoding is a good sign of an infected page, but common sense will prevail) – just curl the http://address without parameters. But if you want to get the headers from an http request using curl instead of your browser – perhaps to troubleshoot http response codes, ssl certificates or other website problems, then this is a technique that will help you…
- 1. Standard in Linux, or Install http://cygwin.com and curl (cygwin has most of the unix binaries available to run on windows, and is free)
- 2. Use “Curl” as explained below
Examples of Using CURL to See HTTP Headers
Example of looking at “proxy”, but it could be any url you are unsure about:
$ curl -sI http://proxy/
HTTP/1.1 302 Found
Location: http://proxy/?cfru=aHR0cDovL3Byb3h5Lw
Cache-Control: no-cache
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Proxy-Connection: close
Connection: close
Content-Length: 1097
http://proxy does a temp redirect (302) in this example to: http://proxy/?cfru=aHR0cDovL3Byb3h5Lw
1097 Bytes are sent and there are no cookies dropped.
The following example is a program I wrote to manage shortcuts for myself, it uses easy mnemonics on clickable.biz to help me remember complicated or fre URLs.
Bit.ly is a good example of a service that does this too. This curl http header example shows how to go through a local proxy to see what the redirector service is doing:
$ curl -x proxy:80 -U james http://clickable.biz/keywords -Is
Enter proxy password for user 'james':
HTTP/1.1 302 Found
Date: Thu, 30 Sep 2010 23:14:45 GMT
Server: Apache
Location: https://adwords.google.com/select/KeywordToolExternal?forceLegacy=true
Content-Type: application/octet-stream
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
curl http headers are saying that clickable.biz/keywords goes directly to the google keyword tool link as a 302.
standard 301 redirect looks like…
$ curl -x proxy:80 -U james http://google.com
Enter proxy password for user 'james':
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
Same 301 redirect url, but header instead of content
$ curl -x proxy:80 -U james http://google.com -Is
Enter proxy password for user 'james':
HTTP/1.1 301 Moved Permanently
Location: http://www.google.com/
Content-Type: text/html; charset=UTF-8
Date: Thu, 30 Sep 2010 23:20:20 GMT
Expires: Sat, 30 Oct 2010 23:20:20 GMT
Cache-Control: public, max-age=2592000
Server: gws
X-XSS-Protection: 1; mode=block
Content-Length: 219
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Curl can be used for many reasons – and curl is much more than what I’m showing here too.
Showing http headers with curl is simply one good use that helps you to investigate questionable emails from Aunt Mildred about “click here for your virtual card shorturl.go/1771 ” type stuff. Or “you HAVE to watch this movie: http://bit.ly/funny
Normally you’d either delete the email, or take a chance and hope you don’t get infected… now you don’t have to guess. You can see what the actual content is going to be before you click it and load up the infections because you can see the http headers using curl.
wget can be used in a similar fashion, however – this article is about using curl to get http headers!