crawler

Python爬虫代理IP池(proxy pool)

A Powerful Spider(Web Crawler) System in Python.

News, full-text, and article metadata extraction in Python 3. Advanced docs:

一些非常有趣的python爬虫例子,对新手比较友好,主要爬取淘宝、天猫、微信、微信读书、豆瓣、QQ等网站。(Some interesting examples of python crawlers that are friendly to beginners. )

Incredibly fast crawler designed for OSINT.

Python脚本。模拟登录知乎， 爬虫，操作excel，微信公众号，远程开机

基于搜狗微信搜索的微信公众号爬虫接口

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Intelligent proxy pool for Humans™

实战🐍多种网站、电商数据爬虫🕷。包含🕸：淘宝商品、微信公众号、大众点评、企查查、招聘网站、闲鱼、阿里任务、博客园、微博、百度贴吧、豆瓣电影、包图网、全景网、豆瓣音乐、某省药监局、搜狐新闻、机器学习文本采集、fofa资产采集、汽车之家、国家统计局、百度关键词收录数、蜘蛛泛目录、今日头条、豆瓣影评、携程、小米应用商店、安居客、途家民宿❤️❤️❤️。微信爬虫展示项目:

Every web site provides APIs.

Proxy [Finder | Checker | Server]. HTTP(S) & SOCKS 🎭

scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

DecryptLogin: APIs for loginning some websites by using requests.

链家网和贝壳网房价爬虫，采集北京上海广州深圳等21个中国主要城市的房价数据（小区，二手房，出租房，新房），稳定可靠快速！支持csv,MySQL, MongoDB,Excel, json存储，支持Python2和3，图表展示数据，注释丰富 ，点星支持，仅供学习参考，请勿用于商业用途，后果自负。

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

owllook-小说搜索引擎

Web crawling framework based on asyncio.

新浪微博爬虫，用python爬取新浪微博数据，并下载微博图片和微博视频

简单易用的Python爬虫框架，QQ交流群：597510560

Async Python 3.6+ web scraping micro-framework based on asyncio

The Last Web Recon Tool You'll Need

Dark Web OSINT Tool

vulnx 🕷️ an intelligent Bot, Shell can achieve automatic injection, and help researchers detect security vulnerabilities CMS system. It can perform a quick CMS security detection, information collection (including sub-domain name, ip address, country information, organizational information and time zone, etc.) and vulnerability scanning.

news-please - an integrated web crawler and information extractor for news that just works

Find web directories without bruteforce