FreeBSD.software
Home/www/py311-protego

py311-protego

0.1.16_1

Pure-Python robots.txt parser

Protego is a pure-Python robots.txt parser with support for modern conventions. It is mainly used by Scrapy application when crawling websites.

Origin: www/py-protego
Category: www
Size: 202KiB
License: BSD3CLAUSE
Maintainer: dbaio@FreeBSD.org
Dependencies: 2 packages
Required by: 1 packages
Website: pypi.org/project/Protego
$pkg install py311-protego

Dependencies (2)

python311py311-six

Required By (1 packages)

py311-scrapy

More in www

py311-requests2.32.5
Python HTTP for Humans
p5-libwww6.81
Perl5 library for WWW access
p5-HTML-Parser3.83
Perl5 module for parsing HTML documents
php84-session8.4.16
The session shared extension for php
p5-Catalyst-Runtime5.90132_1
Elegant MVC Web Application Framework (Runtime)
py311-django424.2.29
High-level Python Web Framework
apache242.4.66
Version 2.4.x of Apache web server
p5-HTTP-Message7.01
Representation of HTTP style messages
p5-Template-Toolkit3.102
Extensible template processing system
p5-Plack1.0051
Perl extension of PSGI reference implementation and utilities