68publishers/crawler-client-php

维护者

详细信息

github.com/68publishers/crawler-client-php

源代码

问题

安装: 320

依赖项: 0

建议者: 0

安全: 0

星标: 1

关注者: 2

分支: 0

开放问题: 0

v1.2.0 2023-07-10 23:51 UTC

Requires

Requires (Dev)

proprietary bcb06a8a712d670d9f582a4b938e63c8c9848881

  • Tomáš Glawaty <tomasglawaty.woop@icloud.com>

scrapercrawlercrawlingscraping68publisherscrawler-client

This package is auto-updated.

Last update: 2024-09-11 02:15:51 UTC

README

Crawler Client Logo

Crawler Client PHP

PHP 客户端，用于 68publishers/crawler

Checks Coverage Status Total Downloads Latest Version PHP Version

安装

$ composer require 68publishers/crawler-client-php

客户端初始化

客户端实例通过调用静态方法 create() 简单创建。

use SixtyEightPublishers\CrawlerClient\CrawlerClient;

$client = CrawlerClient::create('<full url to your crawler instance>');

使用 Guzzle 库与 Crawler API 进行通信。如果您想向 Guzzle 的配置传递一些自定义选项，请使用第二个可选参数。

use SixtyEightPublishers\CrawlerClient\CrawlerClient;

$client = CrawlerClient::create('<full url to your crawler instance>', [
    'timeout' => 0,
]);

向 Crawler API 发出的请求必须始终进行身份验证，因此我们必须提供凭证。

use SixtyEightPublishers\CrawlerClient\CrawlerClient;
use SixtyEightPublishers\CrawlerClient\Authentication\Credentials;

$client = CrawlerClient::create('<full url to your crawler instance>');

$client = $client->withAuthentication(new Credentials('<username>', '<password>'));

应指出，客户端是不可变的 - 调用 with* 方法总是返回一个新的实例。这对于客户端正常工作就足够了。您可以在高级选项 页面上了解有关其他选项的更多信息。

Nette 框架集成

有关与 Nette 框架集成的说明，请参阅此链接。

与场景一起工作

场景由 ScenarioController 处理。

use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ScenariosController;

$controller = $client->getController(ScenariosController::class);

列出场景

/**
 * @param int $page
 * @param int $limit
 * @param array<string, string|array<string>> $filter
 * 
 * @returns \SixtyEightPublishers\CrawlerClient\Controller\Scenario\ScenarioListingResponse
 * 
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\BadRequestException
 */
$response = $controller->listScenarios(1, 10);

$filteredResponse = $controller->listScenarios(1, 10, [
    'name' => 'Test',
    'status' => 'failed',
])

获取场景

/**
 * @param string $scenarioId
 * 
 * @returns \SixtyEightPublishers\CrawlerClient\Controller\Scenario\ScenarioResponse
 * 
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\BadRequestException
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\NotFoundException
 */
$response = $controller->getScenario('<id>');

运行场景

/**
 * @param \SixtyEightPublishers\CrawlerClient\Controller\Scenario\RequestBody\ScenarioRequestBody $requestBody
 * 
 * @returns \SixtyEightPublishers\CrawlerClient\Controller\Scenario\ScenarioResponse
 * 
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\BadRequestException
 */

作为场景配置，我们可以传递一个普通数组或使用准备好的值对象。这两种选项都是有效的。

 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\RequestBody\ScenarioRequestBody;

$requestBody = new ScenarioRequestBody(
    name: 'My scenario',
    flags: ['my_flag' => 'my_flag_value'],
    config: [
        'scenes' => [ /* ... */ ],
        'options' => [ /* ... */ ],
        'entrypoint' => [ /* ... */ ],
    ],
)

$response = $controller->runScenario($requestBody);
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\RequestBody\ScenarioRequestBody;
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValueObject\ScenarioConfig;
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValueObject\Entrypoint;
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValueObject\Action;

$requestBody = new ScenarioRequestBody(
    name: 'My scenario',
    flags: ['my_flag' => 'my_flag_value'],
    config: (new ScenarioConfig(new Entrypoint('<url>', 'default')))
        ->withOptions(/* ... */)
        ->withScene('default', [
            new Action('...', [ /* ... */ ])
            new Action('...', [ /* ... */ ])
        ]),
)

$response = $controller->runScenario($requestBody);

验证场景

/**
 * @param \SixtyEightPublishers\CrawlerClient\Controller\Scenario\RequestBody\ScenarioRequestBody $requestBody
 * 
 * @returns \SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValidateScenarioResponse
 */

作为场景配置，我们可以传递一个普通数组或使用准备好的值对象。这两种选项都是有效的。

 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\RequestBody\ScenarioRequestBody;

$requestBody = new ScenarioRequestBody(
    name: 'My scenario',
    flags: ['my_flag' => 'my_flag_value'],
    config: [
        'scenes' => [ /* ... */ ],
        'options' => [ /* ... */ ],
        'entrypoint' => [ /* ... */ ],
    ],
)

$response = $controller->validateScenario($requestBody);
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\RequestBody\ScenarioRequestBody;
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValueObject\ScenarioConfig;
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValueObject\Entrypoint;
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValueObject\Action;

$requestBody = new ScenarioRequestBody(
    name: 'My scenario',
    flags: ['my_flag' => 'my_flag_value'],
    config: (new ScenarioConfig(new Entrypoint('<url>', 'default')))
        ->withOptions(/* ... */)
        ->withScene('default', [
            new Action('...', [ /* ... */ ])
            new Action('...', [ /* ... */ ])
        ]),
)

$response = $controller->validateScenario($requestBody);

中止场景

/**
 * @param string $scenarioId
 * 
 * @returns \SixtyEightPublishers\CrawlerClient\Controller\Common\NoContentResponse
 * 
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\BadRequestException
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\NotFoundException
 */
$response = $controller->abortScenario('<id>');

与场景调度器一起工作

场景调度器由 ScenarioSchedulersController 处理。

use SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\ScenarioSchedulersController;

$controller = $client->getController(ScenarioSchedulersController::class);

列出场景调度器

/**
 * @param int $page
 * @param int $limit
 * @param array<string, string|array<string>> $filter
 * 
 * @returns \SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\ScenarioSchedulerListingResponse
 * 
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\BadRequestException
 */
$response = $controller->listScenarioSchedulers(1, 10);

$filteredResponse = $controller->listScenarioSchedulers(1, 10, [
    'name' => 'Test',
    'userId' => '<id>',
])

获取场景调度器

/**
 * @param string $scenarioSchedulerId
 * 
 * @returns \SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\ScenarioSchedulerResponse
 * 
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\BadRequestException
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\NotFoundException
 */
$response = $controller->getScenarioScheduler('<id>');
$etag = $response->getEtag(); # you need Etag for update

创建场景调度器

/**
 * @param \SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\RequestBody\ScenarioSchedulerRequestBody $requestBody
 * 
 * @returns \SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\ScenarioSchedulerResponse
 * 
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\BadRequestException
 */

作为场景配置，我们可以传递一个普通数组或使用准备好的值对象。这两种选项都是有效的。

 use SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\RequestBody\ScenarioSchedulerRequestBody;

$requestBody = new ScenarioSchedulerRequestBody(
    name: 'My scenario',
    flags: ['my_flag' => 'my_flag_value'],
    active: true,
    expression: '0 2 * * *',
    config: [
        'scenes' => [ /* ... */ ],
        'options' => [ /* ... */ ],
        'entrypoint' => [ /* ... */ ],
    ],
)

$response = $controller->createScenarioScheduler($requestBody);
$etag = $response->getEtag(); # you need Etag for update
 use SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\RequestBody\ScenarioSchedulerRequestBody;
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValueObject\ScenarioConfig;
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValueObject\Entrypoint;
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValueObject\Action;

$requestBody = new ScenarioSchedulerRequestBody(
    name: 'My scenario',
    flags: ['my_flag' => 'my_flag_value'],
    active: true,
    expression: '0 2 * * *',
    config: (new ScenarioConfig(new Entrypoint('<url>', 'default')))
        ->withOptions(/* ... */)
        ->withScene('default', [
            new Action('...', [ /* ... */ ])
            new Action('...', [ /* ... */ ])
        ]),
)

$response = $controller->runScenario($requestBody);
$etag = $response->getEtag(); # you need Etag for update

更新场景调度器

/**
 * @param string $scenarioSchedulerId
 * @param string $etag
 * @param \SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\RequestBody\ScenarioSchedulerRequestBody $requestBody
 * 
 * @returns \SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\ScenarioSchedulerResponse
 * 
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\BadRequestException
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\PreconditionFailedException
 */

作为场景配置，我们可以传递一个普通数组或使用准备好的值对象。这两种选项都是有效的。

 use SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\RequestBody\ScenarioSchedulerRequestBody;

$requestBody = new ScenarioSchedulerRequestBody(
    name: 'My scenario',
    flags: ['my_flag' => 'my_flag_value'],
    active: true,
    expression: '0 2 * * *',
    config: [
        'scenes' => [ /* ... */ ],
        'options' => [ /* ... */ ],
        'entrypoint' => [ /* ... */ ],
    ],
)

$response = $controller->updateScenarioScheduler('<id>', '<etag>', $requestBody);
$etag = $response->getEtag(); # you need Etag for next update
 use SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\RequestBody\ScenarioSchedulerRequestBody;
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValueObject\ScenarioConfig;
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValueObject\Entrypoint;
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValueObject\Action;

$requestBody = new ScenarioSchedulerRequestBody(
    name: 'My scenario',
    flags: ['my_flag' => 'my_flag_value'],
    active: true,
    expression: '0 2 * * *',
    config: (new ScenarioConfig(new Entrypoint('<url>', 'default')))
        ->withOptions(/* ... */)
        ->withScene('default', [
            new Action('...', [ /* ... */ ])
            new Action('...', [ /* ... */ ])
        ]),
)

$response = $controller->updateScenarioScheduler('<id>', '<etag>', $requestBody);
$etag = $response->getEtag(); # you need Etag for next update

验证场景调度器

/**
 * @param \SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\RequestBody\ScenarioSchedulerRequestBody $requestBody
 * 
 * @returns \SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\ValidateScenarioSchedulerResponse
 */

作为场景配置，我们可以传递一个普通数组或使用准备好的值对象。这两种选项都是有效的。

 use SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\RequestBody\ScenarioSchedulerRequestBody;

$requestBody = new ScenarioSchedulerRequestBody(
    name: 'My scenario',
    flags: ['my_flag' => 'my_flag_value'],
    active: true,
    expression: '0 2 * * *',
    config: [
        'scenes' => [ /* ... */ ],
        'options' => [ /* ... */ ],
        'entrypoint' => [ /* ... */ ],
    ],
)

$response = $controller->validateScenarioScheduler($requestBody);
 use SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\RequestBody\ScenarioSchedulerRequestBody;
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValueObject\ScenarioConfig;
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValueObject\Entrypoint;
 use SixtyEightPublishers\CrawlerClient\Controller\Scenario\ValueObject\Action;

$requestBody = new ScenarioSchedulerRequestBody(
    name: 'My scenario',
    flags: ['my_flag' => 'my_flag_value'],
    active: true,
    expression: '0 2 * * *',
    config: (new ScenarioConfig(new Entrypoint('<url>', 'default')))
        ->withOptions(/* ... */)
        ->withScene('default', [
            new Action('...', [ /* ... */ ])
            new Action('...', [ /* ... */ ])
        ]),
)

$response = $controller->validateScenarioScheduler($requestBody);

激活/停用场景调度器

/**
 * @param string $scenarioSchedulerId
 * 
 * @returns \SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\ScenarioSchedulerResponse
 * 
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\BadRequestException
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\NotFoundException
 */
 use SixtyEightPublishers\CrawlerClient\Controller\ScenarioScheduler\RequestBody\ScenarioSchedulerRequestBody;

# to activate the scenario scheduler:
$response = $controller->activateScenarioScheduler('<id>');

# to deactivate the scenario scheduler:
$response = $controller->deactivateScenarioScheduler('<id>');

删除场景调度器

/**
 * @param string $scenarioSchedulerId
 * 
 * @returns \SixtyEightPublishers\CrawlerClient\Controller\Common\NoContentResponse
 * 
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\BadRequestException
 * @throws \SixtyEightPublishers\CrawlerClient\Exception\NotFoundException
 */
$response = $controller->deleteScenarioScheduler('<id>');

许可证

该软件包在 MIT 许可下分发。有关更多信息，请参阅LICENSE。