All Questions
Tagged with web-scraping node.js
16 questions
2
votes
1
answer
93
views
Automatically extract useful cars from car site
I am using puppeteer to extract data and see when a car that meets my requirements shows up, this is what I did so far. I would like some basic syntax advice, or more advanced tips as well.
I tried to ...
- 79
3
votes
1
answer
135
views
A Language Bot: For creating noun declension or verb conjugation tables
I'm a freshman student.
What does my program solve?
This is actually a bot for reddit, and there are language learning subreddits on that website, sometimes when discussing something with people, we ...
- 97
4
votes
2
answers
393
views
web scraper for emails and links
I created a class to scrape URLS, parse and validate emails and get internal links.
How can I achieve the SOLID principles in this class written in Javascript to make a web scraper?
...
- 61
2
votes
1
answer
107
views
d20pfsrd Discord Bot Scraper Abstraction and Deployment
I wrote a discord bot that listens for links to the d20pfsrd rulebook in discord chat, checks if they are feats or magic, and then scrapes the page, formats it, and spits it back into chat as ...
2
votes
1
answer
1k
views
Text notifications for Facebook Marketplace posting
Application Summary
I received a call from a client asking for a "simple app" that notified him via text message whenever a "Jeep Wrangler" is posted to Facebook Marketplace. It ...
- 171
6
votes
1
answer
216
views
NodeJs crawler for recipes
Here is my try on a crawler made in nodeJs with cheerio, I made it with the idea in mind to use it in a future project I wanna make. Here is the git link: https://github.com/Just4lol/CookCrawler
If ...
- 61
3
votes
0
answers
669
views
Scraping StackOverflow's questions with Node.js
I'm new to node and web scraping in general but with some research I've written this working code.
My goal is to get all questions from X Stack Overflow's pages sorted by most rated and save that ...
- 31
3
votes
1
answer
239
views
Scrape, process and update static site daily with Node
This Node app runs daily to update an overview of interest rates from multiple providers. The overview is displayed on a static website.
The app performs the following tasks:
Scrapes providers' sites ...
- 31
1
vote
2
answers
137
views
Scraping articles on a web site
I'm trying to create a scraping API in Express. The API scrapes the different articles featured on the home page.
Here are the issues I'm trying to resolve:
My code's turning into a ton of jquery ...
- 159
2
votes
1
answer
2k
views
Scrape an infinite-scroll page
My algorithm scrapes an infinite-scroll page but it takes too long. It scrolls three times but I'm wondering if there is a way to do a ScrollBottom() so no need of ...
- 491
2
votes
1
answer
905
views
Basic web scrape project written in NodeJS
Here is a short program web scraping program written in Node.js. I'm just getting to grips with node and this is the first thing I've written with it. I'm liking it so far though I guess I'm kinda ...
- 171
10
votes
1
answer
6k
views
Node.js parallel file download, the ES6 way
I wrote a script that downloads all PDFs found on the web page of a particular government agency. I would have chosen bash for such a task, but I want the script to ...
- 457
2
votes
0
answers
1k
views
Movie torrent-site web scraper with IMDb info and streaming
I'm completely new to Javascript and NodeJs and functional programming in general. The code below scrapes a torrent-website containing movies, gets info about the movie from the OMDb API and lets a ...
- 285
3
votes
1
answer
118
views
Node PSP ISO Scraper
I recently bought a PSP and wanted to know the best ISO files and wrote a scraper to retrieve games ISOs titles that received a high rating and send them to a csv. Any recommendations as to ...
user27606
3
votes
1
answer
1k
views
Node.js web crawler
I've been writing a node.js web crawler, to get it running, I've found myself having to string together really rather a lot of different NPM modules.
I've done my best to keep the code DRY and well ...
- 217