current community

your communities

more stack exchange communities

Stack Exchange
tour help
careers 2.0
Take the 2-minute tour ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.
up vote 1 down vote favorite

I have a web scraper developed using C#, windows forms and the HTML Agility Pack.

I had it all working great when the site changed it's code and broke it. I know it happens often with web scrapers but now I am having trouble figuring out how to correct the issue.

At this time my scraper loops thru multiple URL's and scrapes data from each page.

The problem I am running into is that the template of the site it loops thru will randomly show the newer template which does not have the same HTML classes and ID's that I have defined in the program. What I am trying to do is run a simple if that checks if a single node if null and if it is runs a separate set of code for the new template.

The problem I am having is that my program throws a NullReferenceException on my if statement.

Here is the statement I am using to check if it is null:

var varitem = doc.DocumentNode.SelectSingleNode("//h1[@class='producttitle']").InnerText;

 if (varitem == null) MessageBox.Show("no titles");

It throws the exception at the first line defining the varitem and doesn't even make it to the if statement.

Any advise appreciated!

share|improve this question
 
Check if doc.DocumentNode.SelectSingleNode("//h1[@class='producttitle']") is null. When it is null you'll get NullReferenceException (null.InnerText) –  I4V Jun 8 '13 at 17:15
 
Well new I feel like an idiot! Thanks for your help. If you submit your answer, I'll accept it.. –  Reg Jun 8 '13 at 17:22
add comment

2 Answers

up vote 1 down vote accepted

First you should check whether

 doc.DocumentNode.SelectSingleNode("//h1[@class='producttitle']")

returns null.

If it is null you'll get the the NullReferenceException from null.InnerText

share|improve this answer
add comment
up vote 1 down vote

try below

var varitem = doc.DocumentNode.SelectSingleNode("//h1[@class='producttitle']");

SelectSingleNode can return null and also you better check whether InnerText also not null or empty as well

if (varitem == null || string.IsNullOrEmpty(varitem.InnerText))
              MessageBox.Show("no titles");
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.