current community

your communities

more stack exchange communities

company blog
Stack Exchange Inbox Reputation and Badges
tour help
Stack Overflow is a community of 4.7 million programmers, just like you, helping each other.

Join them; it only takes a minute:

Sign up
Join the Stack Overflow community to:
  1. Ask programming questions
  2. Answer and help your peers
  3. Get recognized for your expertise
up vote 0 down vote favorite 
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup

driver = webdriver.PhantomJS()
#driver = webdriver.Firefox()
driver.get('http://global.ahnlab.com/site/securitycenter/securityinsight/securityInsightList.do')
driver.execute_script("getView('2218')")

html_source = driver.page_source
driver.quit()

soup = BeautifulSoup(html_source)

print(soup.h1.string)

When I use Firefox(), the result is [AhnLab Puts in Appearance at RSAConference for 4th Straight Year] that I want. But when I use PhanthomJS(), the result is [Security Insight] that I don't want.

If I use PhantomJS(), I can't get the result that I want? I want to get the first result using a headless browser.

thanks.

share|improve this question
    
Did my answer work for you? – Vikas Neha Ojha Jun 19 '15 at 22:03
up vote 1 down vote accepted

The phantomjs driver is not loading the navigation after the javascript call immediately. Just put a sleep of 5-10 seconds after the javascript call and it should work for you.

import time

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup

driver = webdriver.PhantomJS()
#driver = webdriver.Firefox()
driver.get('http://global.ahnlab.com/site/securitycenter/securityinsight/securityInsightList.do')
driver.execute_script("getView('2218')")

# Introduce a sleep of 5 seconds here
time.sleep(5)

html_source = driver.page_source
driver.quit()

soup = BeautifulSoup(html_source)

print(soup.h1.string)
share|improve this answer
    
Thanks, It works!! – paul Jun 22 '15 at 4:55
    
No problem. I am glad it helped. – Vikas Neha Ojha Jun 22 '15 at 7:14

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.