How to get email from a page with javascript

If you get access to such a block of code, then you can easily extract the contents of the script block, but further BeautifulSoup will not help: it cannot parse JavaScript.

You may notice that the structure of the resulting code is simple: there is a variable link , which is the sum of several other text variables in which parts of the email are written in standard html coding.

Since the code is quite simple, you can parse it with regular expressions.

For example, the expression

 re.findall('var (\w+) = "(.+?)"', script_code) 

will find all variables of the form var name = "string" , and these are all variable parts of the final email.

You can find out the order of their following by parsing the assignment of the value of the link variable:

 re.search('var link = (.+?);', script_code).group(1).split(' + ') 

Here we find the assignment of the variable link , take the assigned value and divide by the plus symbol.

After that, it remains only to collect the contents of the variables in the desired order and decode the text, the standard html library will help with this, more specifically, the html.unescape function.

As a result, I got the following code:

 def extract_email(code): soup = bs4.BeautifulSoup(code, 'lxml') script_code = soup.script.string variables = {name: html.unescape(value) for name, value in re.findall('var (\w+) = "(.+?)"', script_code)} order = re.search('var link = (.+?);', script_code).group(1).split(' + ') return ''.join(variables[name] for name in order)