Tagged Questions

0
votes
0answers
26 views

Scraping a webpage javascript component for use in an Android app?

I have already successfully scraped a plain HTML page using jsoup and used the returned data in an app. Now I'm looking at a webpage which has javascript component. I don't have a lot of experience ...
0
votes
0answers
42 views

simple html dom php script

I have this task to scrape info from a website. I have this code that scrape the title of the company. But the other details are in the javascript that when you click the title the other infos will ...
0
votes
1answer
36 views

Python scraper mechanize/javascript

I have to scrape all info for former US governors from this site. However, to read out the results and then follow the links, I need to access the different results pages, or, preferably, simply set ...
0
votes
2answers
31 views

Getting the same result from Ruby as Javascript for bitwise XOR

In Ruby: -1104507 ^ 3965973030 => -3966969949 In Javascript: -1104507 ^ 3965973030 => 327997347 Someone asked a similar question here but the answer just pointed to a wrapper for Closure. ...
0
votes
1answer
38 views

Scraping a website which employs a CRC32 check

How would one scrape from a website which uses a javascript CRC32 'proof of work' script like this?: function decode_string(in_str) { return decodeURIComponent(in_str); } function test() { ...
0
votes
1answer
36 views

Web scraping with Python, but values are empty

I'd like to grab values from this site: http://cdn.ime-co.ir/ with BeautifulSoup , but values are empty when I try to import tables. I think disabled with javascrip or anything that I don't know. ...
0
votes
1answer
25 views

Read phantom console.log without killing phantom (phantom.exit())

I am reading a webpage where I have to login and then pass my search string. Every time I want to read the results, I have to call phantom.exit() I do get the results but for every query I have to ...
0
votes
1answer
41 views

Building a simple Node.js scraper function

I'm tryign to build a very simple scraper function for nodeJS - just a function that I can pass a URL to, and it returns the scraped data as var data. I'm completely new to Node.js and can't work ...
0
votes
1answer
62 views

Is there a way to pass object to casper.js' evaluate()?

I saw this thread and look like there is no way to pass complex object to evaluate() https://groups.google.com/forum/#!topic/casperjs/x7I8LDFwFJ0 So if I write an object and want to share amongst ...
3
votes
1answer
136 views

PhantomJS hanging when called from CLI or Web

I'm trying to use phantomJS to capture a screenshot of a URL, however when I call phantomJS (from either the command line or web app) it hangs and seesm to never execute the "exit()" call. I can't ...
0
votes
1answer
90 views

How to run Selenium browsers in the background in windows

I'm trying to click on a javascript button in this site(its the search button on the left): http://www.amadeusepower.com/trek/portals/trek/default.aspx?Culture=en-US I couldn't find a way to click on ...
1
vote
0answers
78 views

imacros firefox use csv file to enter keywords into search box and extract results

I'm using the imacros free plugin for firefox to scrape data from a javascript search form on a website. Here is a breakdown of my macro so far. This portion of the macro below automatically enters ...
0
votes
2answers
59 views

how are javascript based websites different?

I am trying to scrape the content of a website which seems to be working on javascript or some other technology. I am using xpath to find the content on the page. I can see the content using firebug ...
0
votes
0answers
179 views

How to scroll down with Phantomjs to load dynamic content

I am trying to scrape links from a page that generates content dynamically as the user scroll down to the bottom (infinite scrolling). I have tried doing different things with Phantomjs but not able ...
1
vote
1answer
87 views

Best way to get word frequency counts for a website? Or part of a website?

Pretty simple, I just looking for a simple means of extracting word frequencies from a given website, or section of a website. I am also interested in calculating average distance between two given ...

1 2 3 4 5 … 12
15 30 50 per page