通过 
搜索
搜索innmind / crawler-app
爬取网络并将图发布到API
1.5.2
2020-10-25 17:28 UTC
Requires
- php: ~7.4
- innmind/amqp: ~3.0
- innmind/cli: ~2.0
- innmind/cli-framework: ^1.2
- innmind/crawler: ~6.0
- innmind/genome: ^3.0
- innmind/homeostasis: ~4.0
- innmind/installation-monitor: ~3.0
- innmind/ipc: ~3.0
- innmind/json: ^1.1
- innmind/logger: ~2.0
- innmind/operating-system: ~2.0
- innmind/rest-client: ~8.0
- innmind/robots-txt: ~5.0
- innmind/silent-cartographer: ~2.0
- innmind/stack: ^1.0
- monolog/monolog: ~2.0
- symfony/dotenv: ~5.0
Requires (Dev)
- giorgiosironi/eris: ^0.11.0
- innmind/debug: ~2.0
- phpunit/phpunit: ~8.0
- roave/security-advisories: dev-master
- vimeo/psalm: ~4.0
Provides
- innmind/genome-genes: 3.0
This package is auto-updated.
Last update: 2024-08-29 04:42:44 UTC
README
这是一个爬取互联网并将资源属性发布到图书馆的应用。
安装
composer install docker-compose up -d
将 config/.env.dist 复制到 config/.env 并根据需要调整amqp服务器的url。
使用方法
bin/crawler consume crawler
这将启动一个消费者来读取要爬取的url
bin/console crawl http://the.url/to/crawl https://innmind_library.host/
这将爬取 http://the.url.to/crawl，提取资源属性并将它们发布到图书馆 https://innmind_library.host/。它将自动检测要发布的API资源。