current community

your communities

more stack exchange communities

Stack Exchange
tour help
careers 2.0
Take the 2-minute tour ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.
up vote 6 down vote favorite
3

According to https://developers.google.com/webmasters/ajax-crawling/docs/html-snapshot, with HtmlUnit (2.13) I am trying to create a snapshot for a webpage using AngularJS (1.2.1).

My Java code is: 

WebClient webClient = new WebClient();

webClient.setAjaxController(new NicelyResynchronizingAjaxController());
webClient.setCssErrorHandler(new SilentCssErrorHandler());

webClient.getOptions().setCssEnabled(true);
webClient.getOptions().setRedirectEnabled(false);
webClient.getOptions().setAppletEnabled(false);
webClient.getOptions().setJavaScriptEnabled(true);
webClient.getOptions().setPopupBlockerEnabled(true);
webClient.getOptions().setTimeout(10000);

webClient.getOptions().setThrowExceptionOnFailingStatusCode(true);
webClient.getOptions().setThrowExceptionOnScriptError(true);
webClient.getOptions().setPrintContentOnFailingStatusCode(true);

HtmlPage page = webClient.getPage(new WebRequest(new URL("..."), HttpMethod.GET));
webClient.waitForBackgroundJavaScript(5000);
String result = page.asXml();

Although webClient.getPage(...) does not throws any exception the result string still contains "unevaluated angular expressions" such as

<div>
    {{name}}
</div>

I am aware of http://htmlunit.10904.n7.nabble.com/htmlunit-to-scrape-angularjs-td29931.html#a30075 but the recomendation given there does not work either.

Of course the same GET-request works without exceptions in all current browsers.

Any ideas/experiences how to get HtmlUnit working with AngularJS?

Addendum:

I created a HTMLUnit bug report.
For the moment, I switched my implementation to PhantomJS. Maybe this code snippet helps others with a similar problem: 

System.setProperty("phantomjs.binary.path", "phantomjs.exe");
DesiredCapabilities caps = new DesiredCapabilities();
caps.setJavascriptEnabled(true);
caps.setCapability("takesScreenshot", false);

PhantomJSDriver driver = new PhantomJSDriver(caps);
driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
driver.get(new URL("..."));
String result = driver.getPageSource();
share|improve this question

5 Answers 5

up vote 4 down vote

I had the same problem but could not use explicit bootstrapping because angular e2e tests don't work with explicit bootstrap.

I solved the problem by using 

<html id="ng-app" class="ng-app: appmodule;">

instead of 

<html ng-app="appmodule">

htmlunit tests work and e2e tests work as well.

Very likely, htmlunit doesn't (fully?) support document.querySelectorAll(). This method is used by angularInit() to find ng-app directives.

The syntactic variant for the ng-app directive works around the document.querySelectorAll() calls in angularInit().

share|improve this answer
    
This should be the accepted answer, as it solved the problem where HTMLUnit was not resolving the angular directives. Thanks! –  chaitanya Apr 16 at 18:47
    
This seems to help, due to HtmlUnit's poor support for AngularJS, but it doesn't resolve all of the issues. Hopefully 2.15 will be better... –  Splaktar Apr 19 at 5:20
up vote 1 down vote

The HtmlUnit issue has now been fixed. AngularJS expressions are now evaluated correctly.

https://sourceforge.net/p/htmlunit/bugs/1559/

share|improve this answer
    
What version is this fixed in? I'm using 2.14 and using BrowserVersion.CHROME or FIREFOX_24 results in unevaluated AngularJS expressions, but using BrowserVersion.INTERNET_EXPLORER_8 does appear to evaluate those expressions. –  Splaktar Apr 18 at 22:30
    
Looks like this is not yet in a release (2.14 was released in February ). This fix was made in March and is not yet part of a release. I tried to build from the Latest SVN, but it failed: sourceforge.net/p/htmlunit/bugs/1596 –  Splaktar Apr 19 at 5:19
    
I've tested this again with the latest SVN branch of 2.15 and I am still required to do the old hack mentioned in the answer by @stephanme . I am using AngularJS 1.0.8. –  Splaktar Apr 19 at 15:10
up vote 0 down vote

A similar code I have works fine when my single page app uses angularjs 1.0.4; the only thing different I have to do was tell htmlunit to use FIREFOX_17 instead of the default IE8 in htmlunit 2.12 (Similar to the link you provided but FIREFOX_17 instead of FIREFOX_10)

final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_17);

I upgraded to angularjs 1.2 and boom my page shows up with all the angular placeholders.

share|improve this answer
up vote 0 down vote

I had same problem with "unevaluated angular expressions" if I use HtmlUnit. The solution is to bootstrap application manually. Reproduction steps:

Minimal example of app working in browser, but not with HtmlUnit:

<!doctype html>
<html ng-app>
<head>
    <script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.2.6/angular.min.js"></script>
</head>
<body>
    <div>
        <label>Name:</label> <input type="text" ng-model="yourName"
            placeholder="Enter a name here">
        <hr>
        <h1>Hello {{yourName}}!</h1>
    </div>
</body>
</html>

Modification steps:

  1. Bootstrap manually
  2. Remove ng-app to not bootstrap app twice

  3. If you use $http or like you should re-sync it with:

    webClient.setAjaxController(new NicelyResynchronizingAjaxController());

And now working example:

<!doctype html>
<html>
<head>
    <script src="https://ajax.googleapis.com/ajax/libs/angularjs/1.2.6/angular.min.js"></script>
    <script>
        angular.element(document).ready(function() {
            angular.module('myApp', []);
            angular.bootstrap(document, ['myApp']);
        });
    </script>
</head>
<body>
    <div>
        <label>Name:</label> <input type="text" ng-model="yourName"
            placeholder="Enter a name here">
        <hr>
        <h1>Hello {{yourName}}!</h1>
    </div>
</body>
</html>

Test:

WebClient webClient = new WebClient();
webClient.setAjaxController(new NicelyResynchronizingAjaxController());
HtmlPage page = webClient.getPage("http://localhost:8080/index.html");

// Initial state
assertEquals("Hello !", page.getElementsByTagName("h1").get(0).asText());

// Set value
((HtmlInput)page.getElementsByTagName("input").get(0)).setValueAttribute("world");

// New state
assertEquals("Hello world!", page.getElementsByTagName("h1").get(0).asText());

It's working solution, but not really pleasure solution. I don't know it is problem of HtmlUnit or Angularjs.

share|improve this answer
up vote 0 down vote

Thanks for reporting, fixed in SVN. Please expect HtmlUnit 2.15 very soon.

The test case now works with Chrome simulation, the reason was querySelectorAll() should be defined in document/element.

Please note, it seems others have already identified the root cause, and giving a minimal test case to HtmlUnit team can have it fixed in a very short time.

Thanks again for your feedback.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.