Want some programming help? [close]

curl Tutorial

Introduction

curl (https://curl.haxx.se/) is an incredibly useful and powerful command-line tool and library. The latest version at the time of this writing is 7.68.0 released January 8, 2020. You can download it from https://curl.haxx.se/download.html and the source code is available at https://github.com/curl/curl.

It's primary purpose is transferring data over network protocols like HTTP and HTTPS. It supports a large number of other protocols including: FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMB, SMBS, SMTP, SMTPS, Telnet and TFTP. It also supports SSL/TLS encryption, cookies, authentication, and proxies.

You can use it as a command-line tool by calling curl or you can use it as a library inside C, C++, or other applications. We will look at several things in this tutorial including:

  • Basic curl usage for making HTTP requests
  • Downloading files
  • Compiling from source
  • Using curl in a C++ application

Install curl

Most Linux distributions come with curl already installed. If not, you can usually use the package manager for your distribution to easily install it. For example, in Ubuntu:

sudo apt install curl

In Windows, you can download the executable from https://curl.haxx.se/download.html or use the Windows Subsystem for Linux (WSL).

Mac users can use the Homebrew formula for curl.

To learn how to build from source, see the "Compiling curl from source" section below.

View curl documentation

You can view the manual page online at https://curl.haxx.se/docs/manpage.html or by running:

man curl

You can also view the available flags and options with -h or --help:

curl --help

Basic usage

By default, curl will make an HTTP request and output the results to standard out (typically the terminal).

# Make an HTTP request and print out the response
curl https://www.devdungeon.com

Common curl options

When I use curl, there are a few options I find myself using frequently. For example:

  • Viewing verbose output to inspect HTTP headers
  • Saving the output to a file instead of printing to stdout
  • Ignoring SSL certificate errors and self-signed certificates

View verbose output

With the -v or --verbose, curl will output detailed information including the HTTP header information and SSL handshake steps.

curl -v https://www.devdungeon.com

Save the output as a file

Use the -o or --output option to tell curl to save the output as a file instead of printing to standard out:

# Save output to `devdungeon.html`
curl https://www.devdungeon.com --output devdungeon.html

There is another shortcut with a capital O -O. This option will save the file using the name of the file based on the URL. This is handy if you don't want to specify the name and just have it automatically choose the name. For example, this command will download the file and name it TcpNull-1.0.jar.

curl -O https://www.devdungeon.com/sites/default/static/projects/TcpNull/TcpNull-1.0.jar

Alternatively, but less recommended, you could use redirection to pipe the standard output to a file:

# Save output to `index.html`
curl https://www.devdungeon.com > index.html

Ignore SSL certificate errors

You can ignore SSL certificate errors by using the -k or --insecure flags which is useful if you have a self-signed certificate.

# Ignore SSL certificate errors
curl -k https://www.devdungeon.com

Compiling curl from source

Clone or download the source from https://github.com/curl/curl or https://curl.haxx.se/download.html.

git clone https://github.com/curl/curl

For compiling, the main files are in the src/ directory. It comes with a CMakeLists.txt file if you want to use CMake. It also comes with a Makefile.

I am using Windows 10 and CMake for my build. If you are using Mac or Linux, CMake can output the appropriate type of build. You simply need to run cmake on the root directory with the CMakeLists.txt file.

cmake C:\path\to\curl\

It will output several files in whatever directory you are currently in. The primary file of interest is the CURL.sln file which is a Visual Studio solution.

Open the CURL.sln file with Visual Studio (the full Visual Studio, not VS Code). Choose whether you want to build the Debug or Release version and run Build Solution.

If you chose release mode, the .dll file will be located at curl\lib\Release\libcurl.dll. This is the file you will be linking against when you compile your application that uses curl.

C++ curl example

You will need the compiled libcurl.dll (e.g. curl\lib\Release\libcurl.dll) file from the previous build step and the .h files in the include directory curl/include/.

You can find many examples in C and C++ from the official curl website examples page.

Here is a simple example of making an HTTP request:

// main.cpp
// Adapted from https://curl.haxx.se/libcurl/c/https.html
#include <iostream>
#include <curl/curl.h>

int main(void)
{
  CURL *curl;
  CURLcode result;

  curl_global_init(CURL_GLOBAL_DEFAULT);

  curl = curl_easy_init();
  if (curl) {

    // Set URL
    curl_easy_setopt(curl, CURLOPT_URL, "http://www.example.com/");
    // If you want to set any more options, do it here, before making the request.

    // Perform the request which prints to stdout
    result = curl_easy_perform(curl);

    // Error check
    if (result != CURLE_OK) {
      std::cerr << "Error during curl request: " 
                << curl_easy_strerror(result) << std::endl;
    }

    curl_easy_cleanup(curl);
  } else {
    std::cerr << "Error initializing curl." << std::endl;
  }

  curl_global_cleanup();

  return 0;
}
# Compile with g++, linking to libcurl, specifying the include directory and the path to the library(dll)
g++ main.cpp -lcurl -Iinclude -Llib\Release\

This will create an a.exe file that will perform the HTTP request.

Note that you will need to specify the path to the DLL during compile/link time, but after you have the executable, you might need to move the DLL file. The libcurl.dll needs to be in a directory that is in your PATH or in the same directory as the executable.

Conclusion

After following this guide you should know how to install and use curl to perform common tasks like HTTP requests. You should also know how to compile curl from source and use it in a simple C or C++ application.

References

Advertisement