I am writing a script that should ask for the https site json-information. The site has a registration, which I perform in the browser. Further, almost every request in the browser generates 2 connections: the first is of the CONNECT type - a tunnel is installed with the web server "hello server, give / update my session and ticket", the second one is directly a POST request "give me such and such data". Ok, I take the received user_id, which uses the browser for requests, connect the necessary headers, cookies and substitute it all into the script via requests. Session () => post (..., verify = False) - I get a response 200, but without json -data Change the verify to True - errors:

requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)

..the session created in the script does not contain the data that is in the browser session. In addition, the logs say that the script is knocking on the server via HTTP / 1.0, but it is necessary through 1.1 like this ..

 CONNECT target.com:443 HTTP/1.1 User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:47.0) Gecko/20100101 Firefox/47.0 Connection: keep-alive Connection: keep-alive Host: target.com:443 A SSLv3-compatible ClientHello handshake was found. Fiddler extracted the parameters below. Version: 3.3 (TLS/1.2) Random: 9D C7 9D 23 3F D1 A0 17 90 44 A0 AE 25 55 9B 7E 26 ED 31 1D D5 70 F9 A2 BC 4B 94 6B 32 E6 B1 A1 "Time": 08/12/1988 2.06.53 SessionID: D8 05 00 00 58 26 47 AD D5 43 A7 DE DC A7 4C 15 FE 77 40 05 F4 C5 24 AC 5C 7B 46 76 D1 24 61 E8 Extensions: server_name target.com extended_master_secret empty renegotiation_info 00 elliptic_curves secp256r1 [0x17], secp384r1 [0x18], secp521r1 [0x19] ec_point_formats uncompressed [0x0] SessionTicket empty NextProtocolNego empty ALPN h2, spdy/3.1, http/1.1 status_request OCSP - Implicit Responder signature_algs sha256_rsa, sha384_rsa, sha512_rsa, sha1_rsa, sha256_ecdsa, sha384_ecdsa, sha512_ecdsa, sha1_ecdsa, sha256_dsa, sha1_dsa Ciphers: [C02B] TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 [C02F] TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 [CCA9] TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 [CCA8] TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 [C00A] TLS1_CK_ECDHE_ECDSA_WITH_AES_256_CBC_SHA [C009] TLS1_CK_ECDHE_ECDSA_WITH_AES_128_CBC_SHA [C013] TLS1_CK_ECDHE_RSA_WITH_AES_128_CBC_SHA [C014] TLS1_CK_ECDHE_RSA_WITH_AES_256_CBC_SHA [0033] TLS_DHE_RSA_WITH_AES_128_SHA [0039] TLS_DHE_RSA_WITH_AES_256_SHA [002F] TLS_RSA_AES_128_SHA [0035] TLS_RSA_AES_256_SHA [000A] SSL_RSA_WITH_3DES_EDE_SHA Compression: [00] NO_COMPRESSION

In response to such a request, the server sends the following response to the browser:

 HTTP/1.1 200 Connection Established FiddlerGateway: Direct StartTime: 21.10.02.820 Connection: close Encrypted HTTPS traffic flows through this CONNECT tunnel. HTTPS Decryption is enabled in Fiddler, so decrypted sessions running in this tunnel will be shown in the Web Sessions list. Secure Protocol: Tls Cipher: Aes 256bits Hash Algorithm: Sha1 160bits Key Exchange: ECDHE_RSA (0xae06) 256bits == Server Certificate ========== [Subject] CN=*.target.com, OU=PositiveSSL Wildcard, OU=Domain Control Validated [Issuer] CN=COMODO RSA Domain Validation Secure Server CA, O=COMODO CA Limited, L=Salford, S=Greater Manchester, C=GB [Serial Number] 00CC7F2093507FB207C688867166E5E115 [Not Before] 27/07/2016 2.00.00 [Not After] 26/09/2018 1.59.59 [Thumbprint] EEAD7408E980E2A63242A937EAC13DCFB31696C6 [SubjectAltNames] *.target.com, target.com

..te, by the script, I cannot get the SessionID and certificate from the server to continue the dialogue, although I use all headers and cookies. Maybe it is possible to use the session data received by the browser in a script? Perhaps you need to do and authorization on the site script .. = /

  • you have three options 1) understand what you are doing wrong there and what data is missing 2) scripting firefox on js 3) use something like pyxpcomext.mozdev.org , if you want, you can even compile a headless version (without a graphical muzzle) firefox with This plugin: 3 - strangeqargo
  • Oh, new words, thank you .. =) Here the uncle bitbucket.org/richardpenman/browsercookie seems to be dragging browser cookies to the script, so I thought maybe there is such a thing about Sessions as well .. =) - 0l3
  • There are three things that are not related in any way: 1-proxy, to which you http CONNECT request you make a 2-https request to the final site and the validity of its ssl certificate and 3-session on the site at the application level, for example, backed by cookies (has no relationship to the "open firefox https session," whatever that means (I doubt that you want to conduct MitM on yourself)). To avoid messing with the details, you can try selenium webdriver or Ghost.py. If you want to mess with the details, then start with SSLError - jfs
  • I understood, everything is difficult. - 0l3 pm

0