Tagged Questions

1
vote
1answer
399 views

Error running sample code using Spooky.js

I am new to the whole stack of node.js, phantom.js, casper.js and spooky.js. I have everything installed (in Windows), with PATH updated and followed this example: ...
1
vote
1answer
656 views

Module request how to properly retrieve accented characters? � � �

I'm using Module: Request -- Simplified HTTP request method To scrap a page. But this web page contains accented characters á é ó ú ê ã and others. I tried to use encoding: utf-8, but I'm still ...
0
votes
1answer
75 views

Node.js and phantom issue

I'm playing around with Node.js and the node phantom module that is available. But I seem to be stuck at a basic level, and can't get this basic piece of code to work: var phantom = ...
0
votes
1answer
47 views

Multiline attributes with jsdom

is it possible to get the value of multiline attributes with jsdom (I use it with Node.js+JQuery)? The site to scrape includes this HTML: <li><a data-title="<strong>hello world this ...
0
votes
1answer
220 views

Executing scraped JavaScript with cheerio

I have a web page in which there are some JS APIs that don't alter the dom, but return some numbers. I'd like to write a NodeJS application that downloads such pages and executes those functions in ...
1
vote
0answers
60 views

mikeal request only returns part of the html body

I'm doing a simple get request, but my code is only returning a "chunk" of the webpage html. I thought 'request' is supposed to take care of the buffering and return a single string? var request = ...
1
vote
0answers
402 views

Screen scraping paginated results with zombie / node.js

What is the best method for looping through and scraping a result set of unknown length when using zombie / node.js? Here's my base code: var zombie = require("zombie"); var browser = new ...
0
votes
0answers
24 views

Casper.js to fill out browser username and password?

Some case a website would ask username and password in this form (attached photo below) from the browser. Is it possible to use Casper.js to fill out this? Casper.fill can only do it when it's a HTML ...
0
votes
0answers
28 views

How can I make an http request from node to a local file?

I am working on a node.js script to read from an API. I have a mockup file that will be the end result of the API on my filesystem. without spinning up a web server to serve the mockup on my local ...
0
votes
0answers
71 views

How to push buttons and submit forms while web scraping in node.js

So my question is very simple, I've been able to find some places suggesting different options but nothing that has been clear or reliable. I have different problems. 1 While web scraping, I need ...
0
votes
0answers
150 views

Submitting Forms Node-Phantom

Hey fellows I have been trying for days now to navigate a site with Node's Node-Phantom module for phantom.js. I have successfully been able to inject jQuery into the page and fill a form with the ...
0
votes
0answers
145 views

Passing on input/output between Jobs in Node.io

I have these 2 node.io jobs that I would like to run. The first one retrieves a list of links, the other one scrapes the pages and saves some data from it to a database. I first tried to run these ...
0
votes
0answers
180 views

Domino and Zepto-node use a huge amount of memory

I've written a function that scrapes data from a webpage, but the memory usage is enormous (something upwards of 80M). I'm new to the Domino and Zepto-node modules, but I'm using them in the way ...
0
votes
0answers
128 views

Scraping PDF links with JS (do I need Phantom?)

I need to find a way to download PDF files obfuscated in a ASP.NET webpage. Testing with Firebug, I coded a simple Javascript snippet that scraps every ASP link in the page and extracts the contents ...
-1
votes
0answers
29 views

Writing a multi-threaded scraper that writes to database in Node

I'm building a relatively simple web scraper with NodeJS. Here's the basic functionality flow: Takes a JSON file which has urls, names and addresses to parse. I'm using a foreach to run through each ...

1 2
15 30 50 per page