m6web/roboxt

此包已废弃，不再维护。未建议替代包。

用于解析robots.txt文件的库

维护者

详细信息

github.com/M6Web/roboxt

源代码

问题

安装: 572

依赖: 2

建议者: 0

安全: 0

星标: 19

关注者: 15

分支: 7

开放问题: 5

1.1.0 2014-12-08 22:06 UTC

Requires

Requires (Dev)

MIT 2097a848f101f87fec16c143f38d16ee910ca128

  • Benjamin Grandfond <benjaming.woop@theodo.fr>

This package is not auto-updated.

Last update: 2021-09-27 00:06:57 UTC

README

Roboxt是PHP的robots.txt文件解析器。

使用方法

    # Create a Parser instance
    $parser = new \Roboxt\Parser();

    # Parse your robots.txt file
    $file = $parser->parse("http://www.google.com/robots.txt");

    # You can verify that an url is allowed by a specific user agent
    $tests = [
        ["/events", "*"],
        ["/search", "*"],
        ["/search", "badbot"],
    ];

    foreach ($tests as $test) {
        list($url, $agent) = $test;
        if ($file->isUrlAllowedByUserAgent($url, $agent)) {
            echo "\n ✔ $url is allowed by $agent";
        } else {
            echo "\n ✘ $url is not allowed by $agent";
        }
    }

    # You can also iterate over all user agents specified by the robots.txt file
    # And check the type of each directive
    foreach ($file->allUserAgents() as $userAgent) {
        echo "\n Agent {$userAgent->getName()}: \n";

        foreach ($userAgent->allDirectives() as $directive) {
            if ($directive->isDisallow()) {
                echo "  ✘ {$directive->getValue()} \n";
            } else if ($directive->isAllow()) {
                echo "  ✔ {$directive->getValue()} \n";
            }
        }
    }

安装

推荐通过 Composer 安装Roboxt

$> composer require m6web/roboxt

运行测试

Roboxt使用PHPSpec进行单元测试

$> composer install --dev

$> ./vendor/bin/phpspec run

致谢

许可证

Roboxt在MIT许可证下发布。