Make Python Trust Your Corp Proxy Using Sessions

My current employer uses a transparent proxy to filter web traffic. This type of proxy intercepts HTTPS requests and forwards them to the target server, allowing for content inspection and filtering of unwanted material. Essentially, it functions as a man-in-the-middle attack. Since TLS is designed to prevent such scenarios, certificates are used to verify that the server you're contacting is genuine.

Therefore, a transparent proxy must present a certificate that the client trusts. In a corporate setting, these certificates are typically distributed within the organization and imported to clients as trusted root CAs.

If a client doesn't use the operating system's store for trusted certificates, you may encounter a situation where a request is identified as intercepted. In my case, this occurs with:

• NodeJS

• Python

• Docker Containers

Python maintains its own list of trusted root CAs. However, when making a request with the requests module, you can specify an additional certificate to accept. In my situation, the certificate is available from an internal HTTP endpoint. To ensure every subsequent request uses this certificate, you can utilize a session.

Here's an example of creating a session:

def create_session():
    session = requests.Session()
    cert_file_name = os.path.join(tempfile.gettempdir(), "Corp-Root-CA.crt")

    with open(cert_file_name, "wb") as cert_file:
        response = session.get("http://corppki.net/Corp-Root-CA.crt")
        cert_file.write(response.content)
    session.verify = cert_file_name

The session provides HTTP methods directly from the requests module, such as

def authenticate(session):
    response = session.post(
        url=f"{os.getenv("OAUTH_BASE_URL")}//protocol/openid-connect/token",
data=f"grant_type=client_credentials&client_id={os.getenv("OAUTH_CLIENT_ID")}&client_secret={os.getenv("OAUTH_SECRET")}",
        headers={"Content-Type": "application/x-www-form-urlencoded"},
    )

    response_json = response.json()
    session.headers["Authorization"] = f"Bearer {response_json["access_token"]}"

    return response_json["access_token"]

Besides associating a trusted certificate with the session, you can also add headers, such as those for authentication. Each request from this session will include these headers.

In the example above, the access token from an OAuth endpoint is added to an authorization header, which is then sent with each request from this session.