Scrapy project

Scrapy, a fast high-level web crawling & scraping framework for Python.

Library to populate items using XPath and CSS with a convenient API

Common interface for data container classes

Parsel lets you extract data from XML/HTML documents using XPath or CSS selectors

Python library of web-related functions

A service daemon to run Scrapy spiders

The scrapy.org website

Command line client for Scrapyd server

CSS Selectors for Python

A pure-Python robots.txt parser with support for modern conventions.