I am trying to get the links from Google adverts that appear in a webpage in programmatic manner. I have been testing Ruby and Capybara but I have an issue where the DOM as stated by Capybara is not the same as the DOM I see in Firefox. I am using the Selenium driver so I should see the same DOM.
This is the code I am using:
require 'capybara'
session = Capybara::Session.new(:selenium)
session.visit('https://google-developers.appspot.com/custom-search-ads/docs/sample')
puts session.html
The DOM that is reported by Capybara is the actual HTML however if you use DOM Inspector under Firefox you get a DOM containing the executed JavaScript. I also quickly tested the same code using the webkit driver and I get the same result.
Any ideas on how to get access to this DOM and extract Google Ad links? With regards to Google Ad links what I am after is the contents of the iframe that is loaded from JavaScript.
Thanks in advance.
