All Questions
Tagged with web-scraping javascript
24 questions
2
votes
1
answer
93
views
Automatically extract useful cars from car site
I am using puppeteer to extract data and see when a car that meets my requirements shows up, this is what I did so far. I would like some basic syntax advice, or more advanced tips as well.
I tried to ...
- 79
3
votes
1
answer
135
views
A Language Bot: For creating noun declension or verb conjugation tables
I'm a freshman student.
What does my program solve?
This is actually a bot for reddit, and there are language learning subreddits on that website, sometimes when discussing something with people, we ...
- 97
4
votes
2
answers
393
views
web scraper for emails and links
I created a class to scrape URLS, parse and validate emails and get internal links.
How can I achieve the SOLID principles in this class written in Javascript to make a web scraper?
...
- 61
2
votes
1
answer
107
views
d20pfsrd Discord Bot Scraper Abstraction and Deployment
I wrote a discord bot that listens for links to the d20pfsrd rulebook in discord chat, checks if they are feats or magic, and then scrapes the page, formats it, and spits it back into chat as ...
2
votes
1
answer
72
views
Refactoring string incrementing and duplicating function
I wrote this script and got what I wanted from it. However I'm looking for input on ways I can improve the function.
My goal was to scrape a bunch of image links from a website I'd like to rebuild. I ...
- 23
2
votes
1
answer
1k
views
Text notifications for Facebook Marketplace posting
Application Summary
I received a call from a client asking for a "simple app" that notified him via text message whenever a "Jeep Wrangler" is posted to Facebook Marketplace. It ...
- 171
6
votes
1
answer
216
views
NodeJs crawler for recipes
Here is my try on a crawler made in nodeJs with cheerio, I made it with the idea in mind to use it in a future project I wanna make. Here is the git link: https://github.com/Just4lol/CookCrawler
If ...
- 61
10
votes
1
answer
287
views
Chart showing the Percentage of Answered CR questions
Every day Duga reports the number of unanswered questions and percentage of answered questions (after she calls the API endpoint /info) in The 2nd Monitor.
I ...
- 29.5k
3
votes
1
answer
239
views
Scrape, process and update static site daily with Node
This Node app runs daily to update an overview of interest rates from multiple providers. The overview is displayed on a static website.
The app performs the following tasks:
Scrapes providers' sites ...
- 31
3
votes
0
answers
627
views
Scraper Class with Regex
I already posted a not (yet) complete version of this class, but that question got closed, because it contained not passing tests.
Here is the full version, all 5 tests pass.
Originally there was ...
- 173
2
votes
1
answer
739
views
JavaScript Website-Content Grabber
I my firm a few people have following problem:
A Content Management System is hosted externally. The treaty doesn't include database-access.
In September the treaty will expire. So they have to get ...
- 5,002
1
vote
2
answers
137
views
Scraping articles on a web site
I'm trying to create a scraping API in Express. The API scrapes the different articles featured on the home page.
Here are the issues I'm trying to resolve:
My code's turning into a ton of jquery ...
- 159
2
votes
1
answer
2k
views
Scrape an infinite-scroll page
My algorithm scrapes an infinite-scroll page but it takes too long. It scrolls three times but I'm wondering if there is a way to do a ScrollBottom() so no need of ...
- 491
2
votes
1
answer
905
views
Basic web scrape project written in NodeJS
Here is a short program web scraping program written in Node.js. I'm just getting to grips with node and this is the first thing I've written with it. I'm liking it so far though I guess I'm kinda ...
- 171
14
votes
2
answers
297
views
OLog Userscript - Logging messages, planets and researches
For the online text-based browser game OGame I am working on an application with as aim to assist the users where possible, for this I have a server-side part and a client-side part, the respective ...
- 10.7k