cURL is one of the most useful and powerful CLI tools for sending and receiving HTTP(S) requests. It has over 200+ CLI options, but there’s a few that come in handy on a daily basis for network engineers. Here’s a breakdown of the most common configurations.
On it’s own, cURL fetches the URL and displays the response to CLI:
root@linux:/home/user# curl theserpent.co.uk <html> <head><title>301 Moved Permanently</title></head> <body> <center><h1>301 Moved Permanently</h1></center> <hr><center>nginx</center> </body> </html>
Most of the time, the content being returned isn’t necessary, it’s the response headers that usually matter:
root@linux:/home/user# curl -s -D - -o /dev/null https://www.theserpent.co.uk HTTP/2 200 server: nginx date: Thu, 24 Feb 2022 18:59:22 GMT content-type: text/html content-length: 20182 last-modified: Tue, 15 Feb 2022 18:53:15 GMT x-frame-options: SAMEORIGIN x-xss-protection: 1; mode=block strict-transport-security: max-age=31536000 accept-ranges: bytes
-ssuppress the progress bar
-Ddump the headers, using
-dumps them to standard output
-odump the response somewhere, in this case to
Sometimes it’s useful to see the request headers being sent too:
root@linux:/home/user# curl -v -s -D - -o /dev/null https://www.theserpent.co.uk
-venables debugging, which outputs request headers.
cURL can also tell you a lot about the TLS connection, though if you’re dealing ith self-signed certificates, you’ll need the following:
You might always want to control the exact TLS version in use:
Now you can access sites via TLS, regardless of the certificate status.
Sometimes it’s useful to override the headers being sent, or add custom ones:
root@linux:/home/user# curl -H "X-Custom:some value" https://www.example.com
cURL also supports proxy usage, there’s a number of arguments for proxy support, but you can also combine them into a simple command:
root@linux:/home/user# curl -x http://192.168.1.1:80 https://www.example.com curl: (56) Received HTTP code 407 from proxy after CONNECT
If your proxy requires authentication, pass it over using
root@linux:/home/user# curl --proxy-user jeff:passw0rd -x http://192.168.1.1:80 https://www.example.com
There are other commands you can use to break down the particular proxy request you want to make:
--proxy-insecuredon’t check the proxy certificate when doing HTTPS (basically the same as
--proxy-basicuse BASIC authentication (‘Proxy-Authorization’ header)
--proxy-headerpass custom headers to the proxy
Putting it all together
All these commands get get quite cumbersome, so using configuration files really makes things easier. Create a file and add the commands, e.g.
# curl config file # -I Header information only, makes HEAD request # -s No progress bar # -o /dev/null Dump response body (use null for Windows) # -D - Display headers to stdout # -H "X-Blah: Cheese" Add\overwrite header # On-prem primary proxy -D - --proxy http://192.168.1.10:80 --proxy-basic --proxy-user static:passw0rd --proxy-insecure --insecure -s -o null
Then you simply reference the configuration file using
|Posted February 24, 2022||Tweet|
|Written by John Payne|