README

QueryList 插件：Coroutine

安装

composer require mingfei/querylist-coroutine

API

• Coroutine coroutine(int $size=0)：使用协程 QueryList。

class Coroutine

• Coroutine size($size)：设置最大并行数。
• Coroutine add(string $option)：添加选项。
• Coroutine success(callable $callable)：成功回调。
• Coroutine wait($post)：等待请求。

用法

• 绑定协程

use QL\QueryList;
use QL\Ext\Coroutine;

$ql = new QueryList();
$ql->use(Coroutine::class);
//or Custom function name
$ql->use(Coroutine::class,'coroutine');

• 您的规则如下

$rule = [
    'package' => ['.col-sm-9>h4>a','text'],
    'link' => ['.col-sm-9>h4>a','href'],
    'desc'=>['.col-sm-9>p:last','text'],
    'language'=>['.col-sm-9>p:eq(0)','text'],
    'star'=>['.col-sm-3 span:eq(1)','text'],
];
$range = "li .row";

• 使用协程

$coroutine = $ql->range($range)->rules($rule)->coroutine();
$coroutine->add("https://packagist.org.cn/explore/popular");
$data = $coroutine->wait();
print_r($data->all());

• 使用带成功回调的协程

$coroutine = $ql->range($range)->rules($rule)->coroutine();
$coroutine->add("https://packagist.org.cn/explore/popular");
$coroutine->success(function($item){
    $item["star"]  = preg_replace('/\D/s', '', $item["star"] );
    return $item;
});
$data = $coroutine->wait();
print_r($data->all());

输出

Array
(
     [0] => Array
        (
            [package] => symfony/polyfill-mbstring
            [link] => /packages/symfony/polyfill-mbstring
            [desc] => Symfony polyfill for the Mbstring extension
            [language] => PHP
            [star] => 7373
        )

    [1] => Array
        (
            [package] => psr/log
            [link] => /packages/psr/log
            [desc] => Common interface for logging libraries
            [language] => PHP
            [star] => 9976
        )
	...
)

• 使用带大小的协程

$coroutine = $ql->range($range)->rules($rule)->coroutine(1000);
$range = range(1,10000);
foreach($range as $i){
    $coroutine->add("https://packagist.org.cn/explore/popular?page=".$i);
}
$data = $coroutine->wait();
print_r($data->all());
//爬取速度受packagist.org并发限制，这里大约需要等几分钟

输出

use 424.99214887619s