Scrapy project

scrapy Public
Scrapy, a fast high-level web crawling & scraping framework for Python.

Python 42,560 9,491 495 (31 issues need help) 288 Updated Jan 22, 2022
scrapyd Public
A service daemon to run Scrapy spiders

Python 2,406 539 49 20 Updated Jan 13, 2022
scrapyd-client Public
Command line client for Scrapyd server

Python 636 132 10 1 Updated Jan 6, 2022
w3lib Public
Python library of web-related functions

Python 340 89 15 (3 issues need help) 10 Updated Jan 3, 2022
scrapy.org Public
The scrapy.org website

HTML 45 148 1 (1 issue needs help) 1 Updated Dec 23, 2021
parsel Public
Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

Python 736 110 28 (1 issue needs help) 14 Updated Dec 22, 2021
itemadapter Public
Common interface for data container classes

Python 30 BSD-3-Clause 6 2 2 Updated Oct 12, 2021
protego Public
A pure-Python robots.txt parser with support for modern conventions.

DIGITAL Command Language 24 BSD-3-Clause 18 1 (1 issue needs help) 1 Updated Oct 7, 2021
cssselect Public
CSS Selectors for Python

Python 246 53 13 10 Updated Aug 30, 2021
queuelib Public
Collection of persistent (disk-based) and non-persistent (memory-based) queues for Python

Python 228 49 2 2 Updated Aug 26, 2021

Popular repositories