Here are
2,002 public repositories
matching this topic...
List of libraries, tools and APIs for web scraping and data processing.
Updated
Feb 5, 2021
Makefile
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
Updated
Feb 3, 2021
Python
PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs
Updated
Dec 10, 2020
Python
A New Version of 30 Days of Python is nearly here. Get started today.
General Assembly's 2015 Data Science course in Washington, DC
Updated
Apr 18, 2016
Jupyter Notebook
A Devtools driver for web automation and scraping
Simple web scraping for R
Snoop — инструмент разведки на основе открытых данных (OSINT world)
Updated
Feb 5, 2021
Python
Collection of scripts corresponding to LucidProgramming YouTube tutorials
Updated
Feb 6, 2021
Python
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Updated
Dec 27, 2020
Ruby
Nextjs server to query websites with GraphQL
Updated
Jan 29, 2021
JavaScript
Faster requests on Python 3
Updated
Jan 21, 2021
Python
Random User-Agent middleware based on fake-useragent
Updated
Sep 17, 2020
Python
A JavaScript library for generating random user agents with data that's updated daily.
Updated
Dec 11, 2020
JavaScript
A framework for creating semi-automatic web content extractors
Updated
Oct 24, 2020
Python
UI.Vision: Open-Source RPA Software (formerly Kantu) - Modern Robotic Process Automation with Selenium IDE++
Updated
Oct 28, 2020
JavaScript
The Python Code Tutorials
Updated
Feb 3, 2021
Jupyter Notebook
Python binding to Modest engine (fast HTML5 parser with CSS selectors).
Updated
Jan 9, 2021
Python
ACHE is a web crawler for domain-specific search.
Updated
Jan 10, 2021
Java
NBA Stats API via Basketball Reference
Updated
Jan 13, 2021
HTML
An unofficial API for Quora.
Updated
Oct 9, 2016
Python
Công cụ quét và phân tích từ khoá các trang báo mạng Việt Nam
Updated
Feb 1, 2021
Python
A command-line utility and Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.
Updated
Apr 26, 2019
Python
Python scripts for building 'Short Jokes' dataset, featured on Kaggle
Updated
Oct 1, 2020
Python
Guide, reference and cheatsheet on web scraping using rvest, httr and Rselenium.
Improve this page
Add a description, image, and links to the
web-scraping
topic page so that developers can more easily learn about it.
Curate this topic
Add this topic to your repo
To associate your repository with the
web-scraping
topic, visit your repo's landing page and select "manage topics."
Learn more
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session.
You signed out in another tab or window. Reload to refresh your session.
Main examples at Apify SDK webpage, Github repo and CLI templates should demonstrate how to manipulate with DOM and retrieve data from it.
Also add one example of scraping with Apify SDK + jQuery to https://sdk.apify.com/docs/examples/basiccrawler
Feedback from: https://medium.com/better-programming/do-i-need-python-scrapy-to-build-a-web-scraper-7cc7cac2081d
I lost an hour trying to make