The goal is to get the full code of a certain page. I use WebBrowser. After a full page load in the page code, I found the absence of key elements. Having rummaged in the code I understood that these very elements are formed by a certain script. I did not find the function name in the page code. Tell me where to dig. If you open this page in InternetExplorer and open the source code of the page there - the same thing, the necessary info is hidden somewhere in the depths of the scripts

  • Why don't scripts add necessary elements to WebBrowser? Maybe in the outdated version of IE (by default, something like IE8 is used for compatibility, but in general this is fixable) do the scripts break down? - Surfin Bird
  • And what you got is the full code. Content has the right to be generally generated on the server. No one says that a web page should provide data in a convenient way for scraper. - VladD
  • Use the official site API. If there is no API, the site does not want to be removed from it, and you have no right to do it. - VladD

1 answer 1

"elements are formed by a certain script" - this is now fashionable. But unfortunately this makes manual parsing impossible. Sites that form the necessary data with scripts are almost impossible to parsit in my favorite way - download html and poking around in the text.

The easiest way is to pick this Makar: load in WebBrowser, wait for the DOM to load, then you may need to wait another second or two, and only then start looking for the right items. After all, the scripts can also pump up something from the site.

Also, do not forget about the emulation version, google FEATURE_BROWSER_EMULATION indicating for which language, the value is here: https://msdn.microsoft.com/library/ee330730 (v=vs.85 ).aspx#browser_emulation

If I do not understand with what exactly you have problems - check under the answer.