current community

your communities

more stack exchange communities

Stack Exchange
tour help
stack overflow careers
Take the 2-minute tour ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.
up vote 2 down vote favorite
1

so I am doing some web scraping using Selenium with Python and I am having a problem. I am clicking a Next button to move to the next page on a certain website, but I need to stop clicking it when I reach the last page. Now, my idea of doing it would be just to use some_element.click() in a try/except statement and wait until it gives me an error while the button is not clickable anymore. It seems though, .click() doesn't emit any signal of any kind, it doesn't throw an error when the button can't be clicked and it doesn't emit any true or false signal.

a code snippet I tried using:

while True:
    try:
       button = driver.find_element_by_class_name('Next_button')
       button.click()
    except:
       break

Is there any other way? Thanks and cheers.

share|improve this question
    
Is the element displayed when you reach the last page? –  Saifur yesterday
1  
It could be that the link is still clickable, but instead of moving to the next page, it does nothing. Can you share the link to the website you are running the code against? –  alecxe yesterday
    
@Saifur is right. Check what html attributes change for that button when you reac the last page. –  Vikas Neha Ojha yesterday
    
Saifur, the Next page button is still shown, but it is greyed out on the last page. @Vikas Neha Ojha, the class name changes at the last page and so does the xpath to it. How can I detect this change in Python? Or did I understand your advice wrongly. alecxe, yes, that could be true. The loop just goes on forever. –  Demiourgos yesterday
1  
The loop just goes on forever - Please mention whether it is even navigating through the next webpages. You mentioned that class name changes and I can see that you are trying to find the element by class name only, so still your code should work. Could you please post the html of just the button on initial page and the last page? –  Vikas Neha Ojha yesterday

2 Answers 2

up vote 2 down vote accepted

Without knowing more about your example target, the minimal that can be said is that a clickable attribute would have an 'href' attribute.
You can use the get_attribute property of an element:

button = driver.find_element_by_class_name('Next_button')
href_data = button.get_attribute('href')
if href_data is None:
  is_clickable = False

Gets the given attribute or property of the element.

This method will first try to return the value of a property with the given name. If a property with that name doesn’t exist, it returns the value of the attribute with the same name. If there’s no attribute with that name, None is returned.

Values which are considered truthy, that is equals “true” or “false”, are returned as booleans. All other non-None values are returned as strings. For attributes or properties which do not exist, None is returned.

More on Using get_attribute

You could also try is_displayed or is_enabled

share|improve this answer
1  
I am pretty sure this will solve my problem. I do not have the full code right next to me, but I will test it soon. Only one small thing, the class changes its name at the last page. I'll try to test the code soon. Thanks a lot. –  Demiourgos yesterday
1  
If you know for certain that the class name changes on the "last page" you could also perform a check to see if elements with that given class name exist: stackoverflow.com/questions/7991522/… –  Dan O'Boyle yesterday
    
Sorry it took me so long, I had to rewrite the code until that point (since I don't have the source code of my application now). I wanted to try your first suggestion, but I have a new problem. Whenever I use find_element_by_class_name I get an error: "Message: invalid selector: Compound class names not permitted". Basically, because the name of the class has a space in it, "copy-light next". Any quick suggestion? Thanks. –  Demiourgos yesterday
1  
Try just "next" instead of "copy-light next", the partial string should catch. Otherwise try to catch the object a bit differently, see here: stackoverflow.com/questions/17808521/… –  Dan O'Boyle yesterday
    
So, your first suggestion works, I had to use the css_selector (find_element_by_css_selector) to navigate to the button, since I have an error when looking for a class that has a space in its name. Thanks a lot. If you have any suggestion on how to fix that compound class error, I would be even more grateful. –  Demiourgos yesterday
up vote 0 down vote

Use this to get classes element.getAttribute("class") and chcek if list of classes contains characteristic class (for example "disable") from yor html framework which is used to describe unclicable buttons

share|improve this answer
    
Thank you, I will test this too. I did not know getAttribute() could be used in such a way. –  Demiourgos yesterday

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.