How to capture HTTPS traffic from Python requests or httpx library

This tutorial will show you how to capture HTTPS traffic from Python requests or httpx library for debugging purpose.

Here is what we achieve in this tutorial:

• Setup Proxyman
• ✅ Capture HTTP/HTTPS request response from Python code without changing any code
• Works with requests and httpx library

Here is the Source code for this tutorial

Prerequisites

• Python 3.12.0
• requests or httpx library
• Proxyman macOS (Download from Proxyman.com)

Let's start

1. Download Proxyman from Proxyman.com
2. Open Proxyman and install the certificate to your macOS Devices. You can find the certificate in Proxyman > Certificate Menu -> Install for macOS -> Follow the instruction
3. In Proxyman -> Setup Menu -> Automatic Setup -> Click on the "Open new Terminal" button

TIP #1: You can choose what Terminal should open. Support iTerm, Terminal, Hyper, etc.

TIP #2: If you want to use your own Terminal -> Let's open the Setup Menu -> Manual Setup -> Copy the command -> Open your Terminal and paste the command

4. On this terminal -> Run your python code.

If you don't have any python code, you can use the example code in the Source code

pip install requests httpx

import requests
import httpx

def make_post_request():
    # The URL we're sending the POST request to
    url = 'https://httpbin.org/post'
    
    # Sample data to send in the POST request
    data = {
        'name': 'John Doe',
        'age': 30,
        'message': 'Hello, World!'
    }
    
    try:
        # Make the POST request
        response = requests.post(url, json=data)
        
        # Ensure the request was successful
        response.raise_for_status()
        
        # Print the response
        print("Response Status Code:", response.status_code)
        print("\nResponse Content:")
        print(response.json())
        
    except requests.exceptions.RequestException as e:
        print(f"An error occurred: {e}")

def make_post_request_httpx():
    # The URL we're sending the POST request to
    url = 'https://httpbin.org/post'
    
    # Sample data to send in the POST request
    data = {
        'name': 'John Doe',
        'age': 30,
        'message': 'Hello, World!'
    }
    
    try:
        # Create an httpx client
        with httpx.Client() as client:
            # Make the POST request
            response = client.post(url, json=data)
            
            # Ensure the request was successful
            response.raise_for_status()
            
            # Print the response
            print("Response Status Code:", response.status_code)
            print("\nResponse Content:")
            print(response.json())
            
    except httpx.RequestError as e:
        print(f"An error occurred while making the request: {e}")
    except httpx.HTTPStatusError as e:
        print(f"HTTP Error occurred: {e}")

if __name__ == "__main__":
    make_post_request()
    print("\nNow trying with httpx:")
    make_post_request_httpx()

Run:

python main.py

You will see the output in Proxyman.

4. Proxyman automatically captures the HTTP/HTTPS traffic from your Python code.

5. Done ✅

What's next?

The automatic or Manual Setup can capture the HTTP/HTTPS traffic from any Python, Ruby or NodeJS code. No need to modify any code.