#

scrape

Here are 313 public repositories matching this topic...

twintproject / twint

Star

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.

python elasticsearch kibana osint twitter tweets scrape pineapple twint tweep scrape-followers scrape-likes scrape-following

Updated Jul 14, 2021
Python

autoscraper

alirezamika / autoscraper

Sponsor Star

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

python crawler machine-learning scraper automation ai scraping artificial-intelligence web-scraping scrape webscraping webautomation

Updated Feb 3, 2021
Python

Anorov / cloudflare-scrape

Star

A Python module to bypass Cloudflare's anti-bot page.

cloudflare anti-bot-page protected-page scrape scraping-websites

Updated Jul 5, 2020
Python

realsirjoe / instagram-scraper

Star

scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

python bot instagram crawler scraper scrape ig igramscraper

Updated Jun 11, 2021
Python

microlinkhq / metascraper

Star

Easily scrape data from websites using Open Graph, HTML metadata & fallbacks.

metadata parse scrape

Updated Jul 27, 2021
HTML

Anonyfox / elixir-scrape

Star

Scrape any website, article or RSS/Atom Feed with ease!

html rss data-science information-retrieval elixir scraping feed readability scrape

Updated Jul 25, 2020
Elixir

instamancer

ScriptSmith / instamancer

Star

Scrape Instagram's API with Puppeteer

instagram data-mining instagram-scraper scrape instagram-api puppeteer

Updated Jul 13, 2021
TypeScript

austinoboyle / scrape-linkedin-selenium

Star

Open

Certifications return empty []

2

anntdiv commented Jun 10, 2021

Hello,
Thanks for new update in personal_info section,
I found out that the attribute 'certifications' return empty list []
Test url: https://www.linkedin.com/in/an-nguyen-9b3248122/
Results:
`{'personal_info': {'name': 'An Nguyen',
'headline': 'Data Scientist/Machine Learning Engineer',
'company': 'PERSOL PROCESS & TECHNOLOGY CO., LTD.',
'school': 'National Chiao Tung University',

Read more

good first issue help wanted

Open

Companyscraper doesn't work and returns error 'NoneType'

3

Open

Scrape linkedin posts

3

Find more good first issues →

andrewstuart / goq

Star

A declarative struct-tag-based HTML unmarshaling or scraping package for Go built on top of the goquery library

html golang decoder selector selectors unmarshall scrape struct goquery unmarshaling unmarshaller html-unmarshaling

Updated Jun 7, 2019
Go

ultralytics / google-images-download

Star

Google/Bing Images Web Downloader

scraper download images scrape google-image-search google-images-downloader bing-images-downloader google-image-downloader

Updated Jan 17, 2021
Python

Altimis / Scweet

Star

A simple and unlimited twitter scraper : scape tweets, likes, retweets, following, followers, user info, images...

python scraper twitter tweets scraping scrape selenium-webdriver followers twitter-scraper scrape-tweets following save-image scrape-images scrape-followers scrape-likes scrape-following dowload-images

Updated Jul 24, 2021
Jupyter Notebook

JaredLGillespie / proxyscrape

Star

Python library for retrieving free proxies (HTTP, HTTPS, SOCKS4, SOCKS5).

python scraper proxy python3 scrape

Updated Mar 17, 2021
Python

luengwaiban / instagram-python-scraper

Star

A instagram scraper wrote in python. Similar to instagram-php-scraper.Usages are in example.py. Enjoy it!

python instagram scraper scrape ig ins clawer

Updated Jun 1, 2021
Python

evyatarmeged / Humanoid

Star

Node.js package to bypass CloudFlare's anti-bot JavaScript challenges

bot scraping anti-bot-page web-scraping scrape anti-bot bypass humanoid bypass-waf

Updated Jul 1, 2021
JavaScript

serp-spider / core

Star

🕷️ The PHP SERP Spider - A search engine scraper

scraping scrape scrap serp

Updated Feb 5, 2021
PHP

badoux / goscraper

Star

Golang pkg to quickly return a preview of a webpage (title/description/images)

url go golang website crawler scraper image webpage engine icon preview scrape description page title scrapper

Updated Jan 18, 2021
Go

rocketlaunchr / google-search

Sponsor Star

scrape google search results

search go api golang google scrape

Updated Jul 1, 2021
Go

brenns10 / tswift

Star

MetroLyrics API for Python

python lyrics taylor-swift scrape

Updated Apr 9, 2021
Python

Swader / diffbot-php-client

Star

[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library

nlp bot php machine-learning scraper ai scraping crawling artificial-intelligence crawl scrape scraped-data diffbot

Updated Jul 4, 2018
PHP

DrKain / scrape-youtube

Star

A lightning fast package to scrape YouTube search results. This was made and optimized for Discord Bots.

nodejs search bot npm playlist youtube video movie discord scrape

Updated May 29, 2021
JavaScript

Jimut123 / jimutmap

Star

API to get enormous amount of high resolution satellite images from apple / google maps quickly through multi-threading! create map your own map dataset. Bringing data to Humans.

Updated Jul 4, 2021
Python

markowanga / stweet

Star

Advanced python library to scrap Twitter (tweets, users) from unofficial API, fully covered by integration tests

python search api scraper twitter tweets twitter-api users unofficial crawl scrape scrap user tweet scrapper twint scrap-tweet searchrunner

Updated Mar 27, 2021
Python

Pringleman83 / SportsBook

Star

A sports data scraping and analysis tool

python json data analysis league prediction soccer comparison menu football-data scrape compare football sport predict

Updated Jul 12, 2019
Python

html2rss

html2rss / html2rss

Star

📰 Build RSS 2.0 feeds from websites (and JSON APIs) with a few CSS selectors.

Updated Jul 13, 2021
Ruby

montoyamoraga / scrapers

Star

scrapers for building your own image databases

scraper scraping selenium scrape

Updated Feb 22, 2019
Python

warifp / Shopee-Scrape

Star

Shopee Scrape is a tool that functions to collect data - the data needed, such as finding data from photos, prices, names, store locations and others.

php scraper marketplace curl php-library indonesia scrape scraped-data scraper-engine curl-functions curl-library scrape-websites curlphp shopee scrape-images shopee-api

Updated Oct 24, 2020
PHP

ScalaWilliam / amazon-wishlist-api

Star

Scrape Amazon wishlist and provide an API. Play 2.5, JSoup, React.

react scala amazon jsoup scrape

Updated Jun 26, 2016
Scala

vietzerg / Scrape-Finance-Data-v2

Star

A standalone package to scrape financial data from listed Vietnamese companies via Vietstock

python docker redis finance data scrape

Updated Jun 4, 2021
Python

fefit / visdom

Star

A fast and easy to use library for html parsing & node selecting & node mutation, suitable for web scraping and html confusion.

jquery rust dom-manipulation scrape confuse html-parse

Updated Apr 23, 2021
Rust

unixfox / pupflare

Star

A webpage proxy that request through puppeteer - can be used to bypass cloudflare anti bot / anti ddos on a non browser application (like curl)

docker koa proxy cloudflare anti-bot-page protected-page scrape scraping-websites puppeteer cloudflare-bypass cloudflare-scrape

Updated Jul 3, 2021
JavaScript

Improve this page

Add a description, image, and links to the scrape topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the scrape topic, visit your repo's landing page and select "manage topics."