An in-depth overview of PHP And Curl

by Hyder on August 6, 2010

Tweet

Tweet

cUrl is a utility that can be used to do network-related tasks. It stands for Client URL. Curl can be used with any programming language and work with a variety of protocols, including HTTP, HTTPS, FTP, FTPS, GOPHER, LDAP, DICT, TELNET and FILE. You can use it to access Websites, Upload files on FTP server,create proxy application, Content scrapers ,Link Checkers,search Engines ,and so on .PHP also supports libcurl, a cURL library that allows you to develop php application. Below are some examples where and how you can use php/Curl to solve common problems. These examples start with the most easiest one to the most complex one .

Example 1 : Downloading A Web Page using Php/Curl

For some reason or another , you might want to fetch a webpage content and parse the HTML returned to get only certain values from a certain website .Of course ,you might tell yourself there are other ways to to fetch a web page with PHP like :

$content  =  readfile("http://youhack.me");
// or
$content = file_get_contents("http://youhack.me");
// or
$content = file("http://youhack.me");

So why use curl , if it can be done in much simple way without curl. What if the web page that needs to be fetched, require to login to this website to get its content? There is a lack of flexibility and Error Handling with these functions that is why Curl is more suitable since you can handle cookies ,Form posts ,file upload ,authentication and so on .Below is a simple example how to fetch a web page using curl .

//Start Curl Transaction
$ch = curl_init();
//The url that needs to be fetched
curl_setopt($ch, CURLOPT_URL, "http://youhack.me/");
//Return the content of the transaction
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//Exucute The transaction
$output = curl_exec($ch);
echo $output;
curl_close($ch); //Close the Transaction

Example 2 : Login to a website with Apache HTTP Authentication

You might want to access a protected page which has Apache HTTP Authentication . Below is the code how you can achieve this with curl .

// HTTP authentication
$url = "http://localhost/blog/authen/";
$ch = curl_init();

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $url);
//Specify The username and Password for the protected Page
curl_setopt($ch, CURLOPT_USERPWD, "admin:admin");
$result = curl_exec($ch);
curl_close($ch);
echo $result;//Output of page will be displayed here

Below is a screenshot of a page which is protected using Apache HTTP Authentification if you have no idea what it looks like .

Apache HTTP Authentification

You can use this tool to generate username and password and test the code or if you have Cpanel at your webhosting provider ,you can use the “Password Protect Directories” Option .

Example 3 : Fetch a webpage by using proxies

If you want your PHP Application to use a proxy so that your server IP is not identified , you can use a proxy to access that page .This is pretty useful also when accessing pages that ban IP Addresses if there are too many request from your application .You can simply build a list of proxies that your PHP Applicattion will pick up randomly to avoid getting banned .

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://youhack.me');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
//Tunnel through a given HTTP proxy.
curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
//The HTTP proxy to tunnel requests through,It should be in format IP:Port
curl_setopt($ch, CURLOPT_PROXY, 'IP_Address:Port');
//Specify Username and Password to connect through the proxy
curl_setopt($ch, CURLOPT_PROXYUSERPWD, 'Proxy_username:Proxy_Password');
$result   = curl_exec($ch);//Execute The Transaction
echo $result;
curl_close($ch);

Example 4 : Send a File via FTP

You can also upload a file on a FTP server using Curl .You might want to create a web based FTP Client or a script that upload back up files on a remote FTP server .The code below will surely help you to achieve your goal :)

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Upload on FTP Server using PHP/Curl</title>
</head>

<body>
<form action="upload.php" method="post" enctype="multipart/form-data">
<div>
<label for="upload">Select file</label>
<input name="upload" type="file" />
<input type="submit" name="Submitted" value="FTP File !" />
</div>
</form>

</body>
</html>

if (isset($_POST['Submitted'])) {//If button is clicked
    if (!empty($_FILES['upload']['name'])) {//if file name is not empty
        $ch = curl_init();//Start Curl Process
        $localfilename = $_FILES['upload']['tmp_name'];//The File Name
        $fp = fopen($localfilename, 'r');//Open File
        //The Url To send Data to - In this case the FTP URL
        curl_setopt($ch, CURLOPT_URL, 'ftp://FTP_username:FTP_Password@ftp.domain.org/' . $_FILES['upload']['name']);
        curl_setopt($ch, CURLOPT_UPLOAD, 1);//Prepare File for Upload
        curl_setopt($ch, CURLOPT_INFILE, $fp);//The file that the transfer should be read from when uploading.
        curl_setopt($ch, CURLOPT_INFILESIZE, filesize($localfilename));//The Expected Size (in Bytes)of the file when uploading
        curl_exec($ch);//Execute The Transaction
        $error_no = curl_errno($ch);//Get Error Code if Any

        if ($curl_error_no==0) {//0 Means that there's no error
            $error_msg = 'The File '.$_FILES['upload']['name'].' has been uploaded succesfully.';
        } else {
            $error_msg = 'Ooop we could not Upload that file.. Curl Error Code : '.$curl_error_no;
        }
    } else {
        $error_msg = 'You should select a file to upload !';
    }

echo $error_msg;
    curl_close($ch);
    exit();
}

Example 5 : Downloading a file with Curl

You can use curl to download files from other servers . There are many scripts out there such as Rapidleech that allows you to download file from file hosting website like Rapidshare,hotfile,Megaupload etc . Actually Curl is used in these scripts to do remote transfer of files .

$ch = curl_init();
// Specify The URL of file to download
curl_setopt($ch, CURLOPT_URL,'http://youhack.me/demo/contact%20form%20css3/contact%20form%20css3.zip');
//Write the file
$fp = fopen('contact%20form%20css3.zip', 'w');
//The file that the transfer should be written to .
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_exec ($ch);//Execute Transaction
curl_close ($ch);
fclose($fp);

After running the code , the downloaded file should be in the same folder as the script .

Example 6 : Login Programatically on a website

In the past , i had to deal with websites(Forums,Blogs ) that require login to be able to access it’s content . The code below demonstrate how you send data via POST to login to a website and fetch its content .For this example , i am using a Login Form that i created some month ago about “Building A registration System with Email verification in PHP” .


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
 <title>Using cURL</title>
</head>
 <body>
 <?php

 // Identify the URL:
 $url = 'http://youhack.me/demo/Registration%20System/login.php';

 // Start the process:
 $curl = curl_init();

curl_setopt($curl, CURLOPT_URL, $url);

 // Tell cURL to fail if an error occurs:
 curl_setopt($curl, CURLOPT_FAILONERROR, 1);

 //Species the filename where the cookies should be stored .
curl_setopt($curl, CURLOPT_COOKIEJAR, "/temp/cookie.txt");

//Send Cookies Back to the server - If this is not set , no cookies will be sent to the server
curl_setopt($curl, CURLOPT_COOKIEFILE, "/temp/cookie.txt");

//Ignore all previous cookies (if any ) and set a new cookie session
curl_setopt($curl, CURLOPT_COOKIESESSION, true);

 // Allow  redirects if there is any
 curl_setopt($curl, CURLOPT_FOLLOWLOCATION, 1);

 // Assign the returned data to a variable:
 curl_setopt($curl, CURLOPT_RETURNTRANSFER,1);

 // Use POST method :
 curl_setopt($curl, CURLOPT_POST, true);

 // Set the POST data that will be sent to the server
 curl_setopt($curl, CURLOPT_POSTFIELDS, 'e-mail=test%40youhack.net&Password=1&formsubmitted=TRUE');

 // Execute the transaction:
 $result = curl_exec($curl);

if(curl_errno($curl))//If there is any error
{
    echo 'Curl error: ' . curl_error($curl);
exit();
}

 echo '<h2>The code below is fetched from the protected page over <a href="http://youhack.me/demo/Registration%20System/login.php" target="_blank">here :</a></h2>  <pre>' . htmlentities($result) . '</pre>';
 curl_close($curl);
  // Close the connection:
 ?>
 </body>
 </html>

Below is an online demo for the last example :

Demo

Summary

Through the above examples, I have made use of the most common functions for Curl. This should be helpful enough to start working with curl and php. If you think I missed something or need any help, do not hesitate to drop a comment below.

Get Tips Emailed to You !

Recent Posts

Recent Comments