Tell me more ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.
up vote 21 down vote favorite
6

I'm using the Python bindings to run Selenium's webdriver.

from selenium import webdriver
wd = webdriver.Firefox()

I know I can grab a webelement like so...

elem = wd.find_element_by_css_selector('#my-id')

And I know I can get the full page source with...

wd.page_source

But is there anyway to get the "element source"?

elem.source   # <-- returns the HTML as a string

The selenium webdriver docs for Python are basically non-existent and I don't see anything in the code that seems to enable that functionality.

Any thoughts on the best way to access the HTML of an element (and its children)?

share|improve this question
You also could just parse all the wd.page_source with beautifulsoup – eLRuLL Mar 1 at 13:59

7 Answers

up vote 25 down vote accepted

There is not really a straight-forward way of getting the html source code of a webelement. You will have to use JS. I am not too sure about python bindings but you can easily do like this in Java. I am sure there must be something similar to JavascriptExecutor class in Python.

 WebElement element = driver.findElement(By.id("foo"));
 String contents = (String)((JavascriptExecutor)driver).executeScript("return arguments[0].innerHTML;", element);
share|improve this answer
1  
this is your best bet – prestomanifesto Sep 7 '11 at 6:13
1  
This is essentially what I ended up doing, albeit with the Python equivalent. – Chris W. Sep 7 '11 at 21:15
I think the answer below, using element.getAttribute("innerHTML") is a lot easier to read. I don't understand why people are voting it down. – Ryan Shillington Jul 10 '12 at 2:05
@RyanShillington how are you able to see that ppl are down-voting it? – Kelvin Aug 20 '12 at 19:47
@Kelvin once you have a certain amount of rep (1000?), you can click on the number and see the up/down distribution. – Christian Mann Feb 4 at 19:32
up vote 0 down vote

I hope this could help: http://selenium.googlecode.com/svn/trunk/docs/api/java/org/openqa/selenium/WebElement.html

There is described Java method 

java.lang.String    getText()

But unfortunately in Python its not available. So you can translate the method names to Python from Java :-) and try another logic using present methods without getting the whole page source...

E.g.

 my_id = elem[0].get_attribute('my-id')
share|improve this answer
4  
Python actually does have a "gettext" equivalent (I think its just the "text" attribute?) but that actually just returns the "plaintext" between HTML tags and won't actually return the full HTML source. – Chris W. Sep 7 '11 at 21:17
1  
This returns only the plain text (not the html) in Java too. – Ryan Shillington Jul 10 '12 at 2:06
up vote 21 down vote

On JAVA, you can also do it:

elem.getAttribute("innerHTML");

You should have similar thing on Python..

share|improve this answer
7  
In Python: elem.get_attribute("innerHTML") – erikcw Mar 1 '12 at 22:32
3  
innerHTML is a not DOM attribute. So above answer wouldn't work. innerHTML is a javascript javascript value. Doing above would return null. The answer by nilesh is the proper answer. – bibstha Mar 22 '12 at 13:53
1  
This works great for me, and is much more elegant than the accepted answer. I'm using Selenium 2.24.1. – Ryan Shillington Jul 10 '12 at 2:04
4  
Though innerHTML is not a DOM attribute, it is well supported by all major browsers (quirksmode.org/dom/w3c_html.html). It works also well for me. – CuongHuyTo Jul 23 '12 at 10:57
1  
+1 This appears to work in ruby also. I have a feeling that the getAttribute method (or equivalent in other languages) just calls the js method whose name is the arg. However the documentation doesn't explicitly say this, so nilesh's solution should be a fallback. – Kelvin Aug 20 '12 at 19:45
show 1 more comment
up vote 0 down vote 
WebElement element = driver.findElement(By.id("foo"));
String contents = (String)((JavascriptExecutor)driver).executeScript("return      arguments[0].innerHTML;", element);

This code really works to get JavaScript from source as well!

share|improve this answer
up vote 1 down vote

Sure we can get all HTML source code with this script below in Selenium Python:

elem = driver.find_element_by_xpath("//*")
source_code = elem.get_attribute("outerHTML")

If you you want to save it to file:

f = open('c:/html_source_code.html', 'w')
f.write(source_code.encode('utf-8'))
f.close()

I suggest saving to a file because source code is very very long.

share|improve this answer
up vote 0 down vote

Using the attribute method is, in fact, easier and more straight forward.

Using Ruby with the Selenium and PageObject gems, to get the class associated with a certain element, the line would be element.attribute(Class).

The same concept applies if you wanted to get other attributes tied to the element. For example, if I wanted the String of an element, element.attribute(String).

share|improve this answer
up vote 0 down vote

In Ruby, using selenium-webdriver (2.32.1), there is a 'page_source' method that contains the entire page source.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.