Heritrix is the Internet Archive's open-source, extensible, web-scale, archival-quality web crawler project.

从新浪财经、每经网、金融界、中国证券网、证券时报网上，爬取上市公司（个股）的历史新闻文本数据进行文本分析、提取特征集，然后利用SVM、随机森林等分类器进行训练，最后对实施抓取的新闻数据进行分类预测

Open-source Enterprise Grade Search Engine Software

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

Raspagem de dados para iniciante usando Scrapy e outras libs básicas

Web-scraping script that writes the data of all players from FutHead and FutBin to a CSV file or a DB

News extraction and scraping. Article Parsing

초보자들을 위한 웹크롤링 온라인 강의

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: http://ftp.zew.de/pub/zew-docs/dp/dp18033.pdf

Automates the process of repeatedly searching for a website via scraped proxy IP and search keywords

An open source web crawling platform

Web Scraping Framework

ralger makes it easy to scrap a website. Built on the shoulders of titans: rvest, xml2.

Example frontera project

Open Collaborative AI Driven Parser builder for Web Scraping, Data Extraction and Crawling,Knowledge Graph

Jupyter Notebook을 활용한 Time-series data 분석 및 crawling 기술, D3를 이용한 시각화 기술 구현 및 연구

A Web Crawler developed in Python.

This is a quick tutorial on how to use Selenium with Python to create web scraper. The web scraper will pull up a website and record product information and price. You can use my script to run on your own computer. Then improve it to best meet your project needs.

Scrapes attendance and marks related data from AURIS (Ahmedabad University Resource Information System) and notifies the user without him having to check his data repeatedly

A package that helps you to scrap web pages. It shows you a lot of information about the page.

Mixnode Node SDK

a MATLAB script for generating cloud of keywords of the Journal of Physical Oceanography

WebCrawling python script!

🔍 A web crawling app written in java.

A wrapper around popular php http client, GuzzleHttp

This is the Chatbot made with NLTK in python with Term Frequency-Inverse Document Frequencyn(TF-IDF) and Cosine Similarity

A small crawler for getting data from the website: https://sheis.vn