Linux Commands

How to Set User Agent in curl

A user-agent string describes various info to the server you’re trying to connect to, for example, browser type, browser version, operating system, etc. Whenever you’re browsing the web, your web browser automatically includes these data in the HTTP header. Depending on the user agent, the server may respond differently.

The curl command is a powerful tool for developers to transfer data to and from servers. It’s based on the libcurl development library, compatible with most other libraries. We can use the curl command to imitate different user agents when communicating with a server.

Curl user-agent

The curl command sends its own user agent in the HTTP request by default. The curl user-agent follows the following structure:

$ curl/<version_number>

To get the version number of curl, use the following command:

$ curl --version

As the output reports, it’s v7.68.0. So, the default user agent of curl would look like this:

$ curl/7.68.0

Setting custom user agent in curl

We can tell curl to send a different user agent string instead. To do so, use the flag “-A” to describe the new user agent. The command structure will look something like this:

$ curl -A "<custom_user_agent>" <url>

The flag “-A” is an abbreviation for the option “–user-agent”. We can also use that for better clarification:

$ curl --user-agent "<custom_user_agent>" <url>

Another method we can use is the flag “-H.” This flag allows tweaking a single parameter of an extra header to be embedded into the curl request. The command structure would look like this:

$ curl -H "User-Agent: <user_agent>" <url>

Getting custom user-agent

Now, it’s a question of getting a custom user agent. Any user agent consists of multiple variables (operating system, browser version, browser type, etc.), each with multiple possible values. This easily pushes the number of possible user agents to millions.

If you’re interested, check out this WhatIsMyBrowser database on numerous user agents. The massive database records millions of unique user agents. For demonstration purposes, I will be using the following user agent string:

$ Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.74 Safari/537.36

This user-agent describes the following info:

  • Operating system: Linux (x86_64)
  • Browser: Chrome
  • Browser version: 99.0.4844.74

Note that there’s no established standard for defining a user agent. This leads to a very wide (and wild) range of user-agent strings.

Sending custom user agents using curl

Now that we have our hand on the desired user agent, we will demonstrate how to implement it. In the following example, we’re going to send a request to example.com using our custom user agent string:

$ curl --user-agent "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.74 Safari/537.36" example.com

Using the verbose mode, we can see additional info curl is sending to the remote server:

$ curl --verbose --user-agent "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.74 Safari/537.36" example.com

Final Thoughts

This guide demonstrates setting a custom user agent in curl. Depending on the user-agent, the requesting service may respond differently. The curl command sends its own user agent by default. However, it allows specifying a different user agent for its web requests.

This is just a single way of using the curl command. Check out this guide on curl in Linux that demonstrates many ways of using curl (with examples). As always, it’s also recommended to take a look at the man page:

$ man curl

Happy Computing!

About the author

Sidratul Muntaha

Student of CSE. I love Linux and playing with tech and gadgets. I use both Ubuntu and Linux Mint.