使用PHP的cURL庫可以簡單和有效地去抓網頁。你只需要運行一個腳本,然後分析一下你所抓取的網頁,然後就可以以程序的方式得到你想要的數據了。無論是你想從從一個鏈接上取部分數據,或是取一個XML文件並把其導入數據庫,那怕就是簡單的獲取網頁內容,cURL 是一個功能強大的PHP庫。
PHP中的CURL函數庫(Client URL Library Function)常用函數如下:
<?php // 初始化一個 cURL 對象 $curl = curl_init(); // 設置你需要抓取的URL curl_setopt($curl, CURLOPT_URL, 'http://www.cmx8.cn'); // 設置header curl_setopt($curl, CURLOPT_HEADER, 1); // 設置cURL 參數,要求結果保存到字符串中還是輸出到屏幕上。 curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); // 運行cURL,請求網頁 $data = curl_exec($curl); // 關閉URL請求 curl_close($curl); // 顯示獲得的數據 var_dump($data); ?>
接受兩個表單域,一個是電話號碼,一個是短信內容。
<?php $phoneNumber = '13812345678'; $message = 'This message was generated by curl and php'; $curlPost = 'pNUMBER=' . urlencode($phoneNumber) . '&MESSAGE=' . urlencode($message) . '&SUBMIT=Send'; $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, 'http://www.lxvoip.com/sendSMS.php'); curl_setopt($ch, CURLOPT_HEADER, 1); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_POST, 1); curl_setopt($ch, CURLOPT_POSTFIELDS, $curlPost); $data = curl_exec(); curl_close($ch); ?>
<?php $ch = curl_init(); curl_setopt($ch, CURLOPT_URL, 'http://www.cmx8.cn'); curl_setopt($ch, CURLOPT_HEADER, 1); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1); curl_setopt($ch, CURLOPT_PROXY, 'proxy.lxvoip.com:1080'); curl_setopt($ch, CURLOPT_PROXYUSERPWD, 'user:password'); $data = curl_exec(); curl_close($ch); ?>
模擬登錄 discuz 程序。
<?php
/**
* Curl 模擬登錄 discuz 程序
* 尚未實現開啟驗證碼的的論壇登錄功能
*/
!extension_loaded('curl') && die('The curl extension is not loaded.');
$discuz_url = 'http://www.lxvoip.com';//論壇地址
$login_url = $discuz_url .'/logging.php?action=login';//登錄頁地址
$get_url = $discuz_url .'/my.php?item=threads'; //我的帖子
$post_fields = array();
//以下兩項不需要修改
$post_fields['loginfield'] = 'username';
$post_fields['loginsubmit'] = 'true';
//用戶名和密碼,必須填寫
$post_fields['username'] = 'lxvoip';
$post_fields['password'] = '88888888';
//安全提問
$post_fields['questionid'] = 0;
$post_fields['answer'] = '';
//@todo驗證碼
$post_fields['seccodeverify'] = '';
//獲取表單FORMHASH
$ch = curl_init($login_url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$contents = curl_exec($ch);
curl_close($ch);
preg_match('/<input\s*type="hidden"\s*name="formhash"\s*value="(.*?)"\s*\ />/i', $contents, $matches);
if(!empty($matches)) {
$formhash = $matches[1];
} else {
die('Not found the forumhash.');
}
//POST數據,獲取COOKIE
$cookie_file = dirname(__FILE__) . '/cookie.txt';
//$cookie_file = tempnam('/tmp');
$ch = curl_init($login_url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_fields);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file);
curl_exec($ch);
curl_close($ch);
//帶著上面得到的COOKIE獲取需要登錄後才能查看的頁面內容
$ch = curl_init($get_url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 0);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file);
$contents = curl_exec($ch);
curl_close($ch);
var_dump($contents);
?>