Making HTTP Requests With Curl

by Ukeje Chukwuemeriwo Goodness

If you are developing or testing an API, making HTTP calls is a basic requirement. Several tools can help with this, and this article will show you how to use a well-known one: curl.

---

Session Replay for Developers

Uncover frustrations, understand bugs and fix slowdowns like never before with OpenReplay — an open-source session replay suite for developers. It can be self-hosted in minutes, giving you complete control over your customer data.

Happy debugging! Try using OpenReplay today.

---

HTTP (Hypertext Transfer Protocol) is a protocol for data exchange between servers and clients. The client sends HTTP requests, and the server responds to requests.

You can make these requests using various tools. One of the popular ones is curl (client URL), a command-line tool for sending and receiving data via the HTTP, HTTPS (Hypertext Transfer Protocol Secure), and SFTP (Secure File Transfer Protocol) protocols.

Developers love curl since you can automate repetitive tasks with scripts for debugging and testing.

Getting Started With Curl

curl comes preinstalled on popular operating systems like Windows, Linux, and macOS. You can verify your installation by checking the version you have with this command:

curl --version

If you encounter any issues, you can download and install the tool from the official download page.

After successfully installing curl, you can use this command to overview the commands and options you can use with the tool.

curl --help

Here's the result you should expect on running the command:

Making HTTP Requests With curl

Curl is primarily used for making HTTP requests. You can request a server with all the HTTP verbs, and curl provides rich support for custom HTTP behaviors.

Making an HTTP Get Request

You can make GET requests with curl to source from data and resources over HTTP.

Here's how you can make a simple GET request for a webpage.

curl https://www.example.com

When executing the command, here's the data you should expect (the HTML of the example.com webpage).

You don't need to specify the HTTP verb in this case, and on default, curl sends a GET request; however, if you need to specify an HTTP verb, you can add the -X option to the command like this:

curl -X GET https://www.example.com

This command works the same as the other, except that you've specified an HTTP verb in this case.

Making an HTTP POST Request

POST requests are for uploading data/resources to servers and usually create entries on the server. You'll add the data/resource in the body of the POST request.

Here's the format for sending POST requests via curl:

curl -X POST -d "key=value" <URL>

The -d flag specifies that there's a data entry that you wish to send accompanying the request. Here's an example:

curl -X POST http://example.com -d "param1=value1&param2=value2"

The -X POST specifies that you want to use the POST method. The -d flag specifies that the payload is in the request body.

Similarly, by adding a -H and a format specification header flag, you can use curl to send JSON and payloads of other resource types in a request. Here's how to send an image in a POST request.

curl -X POST https://www.example.com/api/users/image \
-H "Content-Type: image/jpeg" \
-d' image.jpg'

In this case, the "Content-Type: image/jpeg" specifies that you're sending an image in the jpeg format.

Making an HTTP PUT Request

You'll send PUT requests to update existing resources on the server. The format for sending PUT requests is similar to sending POST requests. Here it is:

curl -X PUT <URL>
-H "Content-Type: [content-type]"
-d "data"

Here's an example of a PUT request that modifies the image on a server:

curl -X PUT https://www.example.come/users/image
-H "Content-Type: image/jpeg"
-d "image.jpg"

On sending this request, depending on the server's configurations, it should modify the image or return an error.

Sending the same PUT request multiple times returns the same output since it is an idempotent operation

Making an HTTP PATCH request

You can send PATCH requests to partially modify existing resources on the server. The format for sending PATCH requests is similar to sending PUT requests. Here it is:

curl -X PATCH <URL>
-H "Content-Type: [content-type]"
-d "data"

Here's an example of a PATCH request that modifies the image on a server:

curl -X PATCH https://www.example.come/users/image
-H "Content-Type: image/jpeg"
-d "image.jpg"

On sending this request, depending on the server's configurations, it should modify the image or return an error.

Sending the same PATCH request multiple times returns the same output since it is an idempotent operation

Making an HTTP DELETE Request

You'll use a DELETE request to request that the server delete a resource. Here's the format for a DELETE request.

curl -X DELETE <URL>

Authorization with curl

Some servers may require authentication of authorization for you to access some services for access to specified resources.

You can add an authentication/authorization header to curl requests, similar to adding a payload to a request. You can use the -H flag to add an authorization token.

You can use the -H flag to add authorization headers to your curl request.

curl -X <HTTP VERB> <server entry URL>
     -H "Authorization: <Bearer YOUR_ACCESS_TOKEN>"

The <HTTP VERB> placeholder will hold the HTTP verb (could be POST, GET, DELETE, PUT, PATCH, etc.), and the <Bearer YOUR_ACCESS_TOKEN> placeholder will hold the authorization data the server requires.

Here’s an example authorization request:

curl -X GET https://www.example.com/user/image
     -H "Authorization: Bearer YOUR_ACCESS_TOKEN"
     -H "Content-Type: image/jpeg"
     -d "image.jpg"

In this case, you’re sending a GET request to the [example.com](http://example.com) server to access the image.jpg image from the /user/image route. If the access token is valid, the server will grant your request.

Overviewing Advanced Curl Methods

Some handy curl request methods aren't popular, but you may find them handy as you use the curl tool.

Here's an overview of some of them:

The HEAD Method

You'll use the HEAD method to request the HTTP header without the response body. You'll need the -I flag to specify like this:

curl -I <URL>

You'll replace the <URL> placeholder with the HTTP URL for effect.

The OPTIONS Method

You'll use the OPTIONS method to request information about available communication options on the target source. The OPTIONS method may return data in headers, which may include supported HTTP methods (GET and POST) and supported content types, but doesn't return the actual content of the resource. Responses to the OPTIONS method aren't stored as cache data but might be stored by the server based on its configuration.

Here's a syntax of how the OPTIONS method is used:

curl -X OPTIONS <URL> -i

In this case, the -i flag includes the response header in the command output on execution.

The TRACE Method

The TRACE method is a diagnostics tool that you can use to reveal changes to intermediate servers. It performs a message loop-back test along the path to the resource.

curl -X TRACE <URL>

Many web servers have the TRACE method disabled because it can leak sensitive information when mishandled.

Common Curl Errors and Their Solutions

You may encounter many errors when using curl, from firewall issues to timeouts, an insufficient PHP memory limit, and more.

Here are some common errors you may encounter:

• Connection Timeout: Connection timed out is a common curl error you'll experience when the client fails to connect with the server in the specified time frame.

• Firewall Issues: You'll experience a firewall issue when the firewall of the resource you're trying to access flags some methods as suspicious.
• Incorrect SSL Configuration occurs when the resource server's Secure Socket Layer (SSL) protocol blocks the process to ensure a secure connection.
• Outdated Software: Your client's configuration may need to run on the compatible PHP and curl library versions, which may cause time-outs and other errors.

Here's how you can fix these errors:

• Connection Timeout: You can fix your connection time issues by adding the --max-time flag followed by the maximum time you want to allow for (in seconds)

• Firewall Issues: This involves configuration changes on the server. You can discuss this with the network admin to fix this issue.
• Incorrect SSL Configuration: You can bypass SSL checks using the -k flag after the curl command. Also, update CA certificates to recent ones.
• Outdated Software: Try updating PHP and curl to fix this error. You can also try adjusting the memory limit in the php.ini file if your issue concerns memory.

Conclusion

You've learned how to make HTTP requests to servers with the curl client right from your command line. You also learned how to use curl methods, errors you may encounter, and how to debug them.

Curl is a versatile client, which is why it's popular. In your next steps, you can use curl to automate repetitive tasks and fetch and use data easily.