Explaining CURL in Detail

In some PHP tutorials, we already covered how to make HTTP requests using file_get_contents and stream_get_contents – but if your host supports it, you might also be able to use cURL – not that it will matter which method you use, as long as it gets the job done in your situation.

cURL is powered by libcurl for all transfer-related features. libcurl is free, thread-safe, IPv6 compatible, feature-rich, well supported and fast.

Using cURL, you can:

• Implement payment gateways’ payment notification scripts.

• Download and upload files from remote servers.

• Login to other websites and access members-only sections.

Once you’ve compiled PHP with cURL support, you can begin using the cURL functions.

The curl_init function initializes the request, usually assigning it to a variable – in this case, $ch – we are going to use this when we add options to the request, such as whether we want the transfer/result returned directly, and the user agent string we want to use.

The curl_setopt function is used to set many different options – not covered in this tutorial.

The curl_exec function performs the request with whatever options we have set. By default, it should only return true or false – according to the manual – so we set the CURLOPT_RETURNTRANSFER option to true, in order to make it return the content instead of just true; it should still return false if something went wrong with the request.

Using CURL for HTTP Requests

Using CURL to make HTTP Requests is not much different than using file_get_contents, the principles are the same.

A working example script to perform a GET request is included below:

<?php
// URL to fetch
$url = "http://example.com/";

// Initialize cURL session
$ch = curl_init($url);

// Option to Return the Result, rather than just true/false
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

// Perform the request, and save content to $result
$result = curl_exec($ch);

// Close the cURL resource, and free up system resources!
curl_close($ch);

// Shows the result!
echo $result;
?>

It can be a good idea to close – curl_close – the session if you are looping, unless you are going to overwrite the same session with a new one, or unless you are sure the script is done. In any case, all open sessions will be closed automatically when the script stops, so often it doesn’t matter.

HTTP Post Requests with cURL in PHP

Creating a POST Request with cURL can be somewhat complicated, because a lot of unexpected things might happen, in the light of this, it seems more streight forward to use file_get_contents – anyway, lets get started!

The important parts when making a post request, is the CURLOPT_POST and CURLOPT_POSTFIELDS – the below script should be fully working.

$url = "http://brugbart.com/Examples/http-post.php"; // The POST URL
// The POST Data
$postdata  = "name=Jacob";
$postdata .= "&bench;=150";

$ch = curl_init($url);

// Set the request type to POST
curl_setopt($ch, CURLOPT_POST, true);
// Pass the post parameters as a naked string
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);

// Option to Return the Result, rather than just true/false
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

// Perform the request, and save content to $result
$result = curl_exec($ch);

// Show the result?
echo $result;

As mentioned ealier, the important part in the above script is the part where you set the request-type to POST, and pass the data to be posted. I.e.

// Set the request type to POST
curl_setopt($ch, CURLOPT_POST, true);
// Pass the post parameters as a naked string
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);

The variable $postdata contains a string, where each parameter is separated by unencoded ampersands. To easily add and remove parameters in a very readable way, you can do like we did in this script. I.e

// The POST Data
$postdata  = "name=Jacob";
$postdata .= "&bench;=150";

This is just as clean and readable as using an array.

Problems with cURL and POST Requests

To begin with, cURL in PHP seems to be poorly documented, but this tutorial hopefully clears that up a bit.

One problem you may encounter while using cURL is that it sends the POST as multipart/form-data with a boundary, instead of sending it as application/x-www-form-urlencoded. The reason this is happening is that curl, for some reason, does this if you pass the post parameters as an array.

We have been unable to get this to work properly – even when setting headers manually and using http_build_query on the array – maybe it is character-dependent? In any case, there is no reason to use an array. Using an array seems even more messy than just passing the data as a naked-untreated string.

Setting Request Headers with cURL in PHP

Setting custom headers in cURL is done through the use of the CURLOPT_HTTPHEADER option, which can be set with the curl_setopt function. To add headers to your HTTP request, you can add the headers to an array, which you can then pass to the cul_setopt function, as demonstrated in the below example.

$User_Agent = 'Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.43 Safari/537.31';

$request_headers = array();
$request_headers[] = 'User-Agent: '. $User_Agent;
$request_headers[] = 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8';

If you are setting headers using the CURLOPT_HTTPHEADER function, you may consider setting all headers this way, rather than using cURL functions – unless there is some specific advantage to doing it the other way.

After setting all the HTTP request headers that you want, you may pass the array to the curl_setopt function. Your request can then be executed as you would normally, using the cURL curl_exec function.

curl_setopt($ch, CURLOPT_HTTPHEADER, $request_headers);

In this cURL tutorial, we have now demonstrated how you can set the User-agent and Accept headers – but cookies are also delivered in the headers of your request – this is done with the Cookie header. I.e.

$cookies = 'CookieName1=Value;CookieName2=Value';
$request_headers[] = 'Cookie: '. $cookies;

Some headers can be set with cURL functions. This goes for the User-agent and Cookie headers. For example, this can be done by using the CURLOPT_USERAGENT and CURLOPT_COOKIE options respectively – but there is rarely any advantage to doing things with cURL vs just doing them manually. I.e.

$User_Agent = 'Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.43 Safari/537.31';
$cookies = 'CookieName1=Value;CookieName2=Value';
curl_setopt($ch, CURLOPT_USERAGENT, $User_Agent);
curl_setopt($ch, CURLOPT_COOKIE, $cookies);

Using Proxy Servers with cURL in PHP

Setting up a proxy server to be used with cURL and PHP is relatively simple. It mostly depends on the server that you are using, and the authentication method (if any). The HTTP authentication method is controlled by the CURLOPT_PROXYAUTH option. The default method is CURLAUTH_BASIC. If the proxy requires authentication, a username and password can be set in the [username]: [password] format, using the CURLOPT_PROXYUSERPWD option.

For now we’ll just focus on using a proxy that doesn’t require any authentication. Setting a proxy server and a port number in PHP for cURL can be done using the CURLOPT_PROXY option, like shown in the below example:

curl_setopt($ch, CURLOPT_PROXY, '128.0.0.3:8080');

As shown in the above example, you can set the a proxy with the IP:PORT syntax in PHP using cURL. But if you prefer to keep the ip seperated from the port, you can also use the CURLOPT_PROXYPORT option, which would result in the below PHP code:

curl_setopt($ch, CURLOPT_PROXY, '128.0.0.3');
curl_setopt($ch, CURLOPT_PROXYPORT, '8080');

After setting a proxy server, you will be able to perform the request using the curl_exec function. I.e.

$ch = curl_init($url);
$url = "http://brugbart.com/Examples/http-post.php";
curl_setopt($ch, CURLOPT_PROXY, '128.0.0.3');
curl_setopt($ch, CURLOPT_PROXYPORT, '8080');

// Perform the request, and save content to $result
$result = curl_exec($ch);
echo $result;

Setting cURL Proxy Type

cURL supports two proxy types, the default is HTTP, and the other option is SOCKS5. You can set the proxy type using the CURLOPT_PROXYTYPE option. I.e.

curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5);

You really only need to set the type of the proxy, if you are not using a HTTP proxy.

Setting Authentication Method

As mentioned in the beginning of the tutorial, setting the authentication method of a proxy server can be done using the CURLOPT_HTTPAUTH option. To make this work properly, we will also need to provide a username and password for the proxy server. This is all accomplished in the below script, in which we are just using a BASIC authentication method.

$ch = curl_init($url);
$url = "http://brugbart.com/Examples/http-post.php";
curl_setopt($ch, CURLOPT_PROXY, '128.0.0.3');
curl_setopt($ch, CURLOPT_PROXYPORT, '8080');

curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);

// The username and password
curl_setopt($ch, CURLOPT_PROXYUSERPWD, 'USERNAME:PASSWORD');

// Perform the request, and save content to $result
$result = curl_exec($ch);
echo $result;

Other authentication methods include the following:

• CURLAUTH_BASIC
• CURLAUTH_DIGEST
• CURLAUTH_GSSNEGOTIATE
• CURLAUTH_NTLM
• CURLAUTH_ANY
• CURLAUTH_ANYSAFE

Note.The vertical bar | (or) operator can be used to combine methods. If this is done, cURL will poll the server to see what methods it supports and pick the best.