There is a htmlunit parser that parses the site where the content is loaded by Ajax, example code:

 java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit").setLevel(Level.OFF); WebClient webClient = new WebClient(BrowserVersion.FIREFOX_3); webClient.setThrowExceptionOnScriptError(false); webClient.setAjaxController(new NicelyResynchronizingAjaxController()); HtmlPage page = (HtmlPage) webClient.getPage("https://site.name");

div with the overview loaded via the AJAX; when you call page.asXml() , the necessary div is inside the page ; when you call page.getElementById("overview") a NullPointerException error occurs. How to make the required div be accessible via getElementById() ?

  • What is a URL? - Ahmed Ashour
  • 2
    Please add to the question the text of the exception along with the stack trace, and the site address (if not secret). - zRrr
  • @zRrr in printStackTrace () is simple: java.lang.NullPointerException at parser.Main.main (Main.java:80), a site on localhost - Dmitri88

1 answer 1

Initial data is not enough, but still, if a piece

 page.getElementById("overview")

highlighted in a separate line and the NullPointerException falls on it, it means that your page variable is null . Try after the line

 HtmlPage page = (HtmlPage) webClient.getPage("https://site.name");

do the good old sysout:

 System.out.println((page == null ? "Error: page is null":"normal"));

And if in the console before the exception falls,

 Error: page is null

then the problem is not in the html itself, but in its receipt.