web-crawling

Crawlee—A web scraping and browser automation library for Node.js that helps you build reliable crawlers. Fast.

Machine Learning Model for Sport Predictions (Football, Basketball, Baseball, Hockey, Soccer & Tennis)

A simple but powerful web crawler library for .NET

A simple web scraper to extract Product Data and Pricing from Amazon

Library for Rapid (Web) Crawler and Scraper Development

⚡ Ayakashi.io - The next generation web scraping framework

Scrapy Training companion code

A web crawling framework written in Kotlin

Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO 👉

💵 💰 🇧🇷 Informações sobre taxas oficiais diárias de Inflação, Selic, Poupança, Dólar, Dólar PTAX, Euro e Euro PTAX pelo site do Banco Central do Brasil

Parser and database to index the terpene profile of different strains of Cannabis from online databases

Command Line Tool to download torrents

Simple robots.txt template. Keep unwanted robots out (disallow). White lists (allow) legitimate user-agents. Useful for all websites.

Scraping and Web Crawling Framework For Zhihu Live

Web Scraping Craigslist's Engineering Jobs in NY with Scrapy

Amazon products scraper with using of rotating proxies and headless Chrome from ScrapingAnt

Continuous scalable web crawler built on top of Flink and crawler-commons

Compares price of the product entered by the user from e-commerce sites Amazon and Flipkart 💰 📊

implementing an end-to-end tweets ETL/Analysis pipeline.

Repository for the projects needed to complete the Data Analyst Nanodegree.