webcrawler

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

新闻网页正文通用抽取器 Beta 版.

蓝天采集器是一款免费的数据采集发布爬虫软件，采用php+mysql开发，可部署在云服务器，几乎能采集所有类型的网页，无缝对接各类CMS建站程序，免登录实时发布数据，全自动无需人工干预！是网页大数据采集软件中完全跨平台的云端爬虫系统

A Unix-style personal search engine and web crawler for your digital footprint.

HTTP API for Scrapy spiders

Open-source Enterprise Grade Search Engine Software

An R web crawler and scraper

《Python爬虫开发 从入门到实战》配套源代码。

📲 Bot to help solve HQ trivia

中国大陆大学列表爬虫

O maior livro de receitas culinárias em língua portuguesa

A php crawler that finds emails on the internets

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

A web crawling framework written in Kotlin

使用 Scrapy 写成的 JK 爬虫，图片源自哔哩哔哩、Tumblr、Instagram，以及微博、Twitter

*UNSUPPORTED* Use igcloud to generate Instagram Word Cloud ! 🛫 🛫 ✈ 🔝

Multithreaded Konachan / Yandere (moebooru based site) Image Bulk Downloader | 多线程K站Y站下载器

2019 nCoV realtime track system based Scrapy + influxdb + grafana + NLTK + Stanford CoreNLP

The data and code that used in my book.

一个致力于用Python提高部门工作自动化水平的程序库！（包括数据采集、办公自动化、辅助研究、图网络、复杂系统、3D可视化等）

Yummy Recipe Crawler and Search

Bot para monitoramento de promoções no fórum do Hardmob http://www.hardmob.com.br/promocoes/

This program provides efficient web scraping services for Tor and non-Tor sites. The program has both a CLI and REST API.

Document Search Engine Tool

Simple node worker that crawls sitemaps in order to keep an algolia index up-to-date

A web browser 🌎 hosted as a service, to render your JavaScript web pages as HTML

Social Scraper is a python tool meant for Detection of Child Predators/Cyber Harassers on Social Media

Java web crawling library