Python Requests in 2025: A Practical Guide with Proxies

Nathan Reynolds

Coding Tutorials

If you write Python and touch the web, you'll meet the requests library sooner rather than later. It's the pragmatic choice for sending HTTP requests from code: fetching pages, calling APIs, submitting data, and reading back structured responses. This guide walks through the parts you'll actually use day to day, then gets specific about routing traffic through proxies for legitimate work like public-data collection, QA, and geo testing.

What the Requests library actually does

Requests is a small, readable HTTP client for Python. Its job is to send an HTTP request to a server and hand you back a clean response object you can inspect. A browser does the same thing under the hood, but it also renders pages, runs JavaScript, and manages a UI. For automation you rarely need any of that overhead, so a plain HTTP client is faster, lighter, and easier to script.

If you want the full spec of the underlying protocol, the HTTP semantics are defined in RFC 9110, and the official library reference lives in the Python documentation.

Installing and importing Requests

Requests isn't part of the standard library, so install it with pip:

Then import it at the top of your script:

import requests

That's the whole setup. You're ready to make requests.

Fetching a web page

Start with the most common task: downloading the raw HTML of a page. Save this as fetch_example.py:

import requests

# Let's try fetching content from httpbin, a useful testing site
url = "https://httpbin.org/html"
response = requests.get(url)

# Check if the request was successful (status code 200)
if response.status_code == 200:
    print(response.text)
else:
    print(f"Request failed with status code: {response.status_code}")

Run it with python fetch_example.py and you'll see the page source printed to your console:

<!DOCTYPE html>
<html>
  <head>
  </head>
  <body>
    <h1>Herman Melville - Moby-Dick</h1>
    ...
  </body>
</html>

The response.text attribute holds the body as a string. Note that Requests doesn't parse HTML for you. To pull out links, headings, or specific text you'll pair it with a parser. Beautiful Soup is the usual companion here, and we cover that pairing in detail in our guide to web scraping with Beautiful Soup.

Working with JSON APIs

HTML isn't the only thing you'll fetch. Requests is just as comfortable talking to APIs, which usually exchange data as JSON. JSON is a compact, readable format designed for machine-to-machine communication. A typical payload might look like this:

{
  "id": 42,
  "username": "dev_guru",
  "email": "guru@example.com",
  "isActive": true,
  "roles": ["admin", "editor"]
}

Let's hit a public test API, DummyJSON, which serves fake datasets for exactly this kind of practice:

import requests

# Fetching sample user data
response = requests.get("https://dummyjson.com/users")

# Requests has a built-in JSON decoder
if response.status_code == 200:
    data = response.json()  # 'data' is now a Python dictionary/list
    # print(data)  # Uncomment to see the full structure
else:
    print(f"API request failed: {response.status_code}")

The response.json() method decodes the body into native Python dictionaries and lists, so you skip manual parsing. Suppose you only want each user's name and email:

import requests

response = requests.get("https://dummyjson.com/users")

if response.status_code == 200:
    data = response.json()
    users = data.get("users", [])  # Use .get for safer access
    print("User List:")
    for user in users:
        username = user.get("username", "N/A")
        email = user.get("email", "N/A")
        print(f"- Username: {username}, Email: {email}")
else:
    print(f"API request failed: {response.status_code}")

The output is a tidy list:

User List:
- Username: atuny0, Email: atuny0@sohu.com
- Username: hbingley1, Email: hbingley1@plala.or.jp
-

Using .get() with defaults keeps your loop from crashing when a field is missing, which is common with real-world APIs. For deeper handling of nested structures, see our walkthrough on working with JSON in Python.

Passing query parameters cleanly

Most APIs let you filter, page, or search using query parameters appended after a ?, like https://example.com/search?query=python&limit=10. Building those URLs by hand with string concatenation gets ugly and error-prone fast. Requests handles it with the params argument:

import requests

api_url = "https://dummyjson.com/users"
query_params = {
    'limit': 5,
    'skip': 10,
    'select': 'firstName,lastName,email'  # Select specific fields
}

response = requests.get(api_url, params=query_params)

# The actual URL requested would be something like:
# https://dummyjson.com/users?limit=5&skip=10&select=firstName%2ClastName%2Cemail
# Requests handles the encoding (like %2C for the comma)

if response.status_code == 200:
    data = response.json()
    print(data)
else:
    print(f"Request failed: {response.status_code}")
    print(f"URL attempted: {response.url}")  # See the final URL

Passing a dictionary keeps the code readable and lets Requests take care of URL encoding, so special characters like the comma become %2C automatically. The response.url attribute shows you the final URL that was actually requested, which is handy for debugging.

Reading HTTP status codes

Every response carries a three-digit status code that tells you how the request went. You've already seen 200 OK. Check the code before you touch the body, because processing an error page as if it were valid data is a classic source of silent bugs:

import requests

# Successful request
response_ok = requests.get("https://dummyjson.com/products/1")
print(f"Product 1 Status: {response_ok.status_code}")  # Expected: 200

# Request for a non-existent resource
response_not_found = requests.get("https://dummyjson.com/products/9999")
print(f"Product 9999 Status: {response_not_found.status_code}")  # Expected: 404

The families you'll run into:

  • 2xx (200 OK, 201 Created): success.

  • 3xx (301 Moved Permanently, 302 Found): redirection.

  • 4xx (404 Not Found, 403 Forbidden, 401 Unauthorized): client errors, meaning something about your request was off.

  • 5xx (500 Internal Server Error, 503 Service Unavailable): server errors, meaning the target had a problem.

The full registry is maintained by the IANA and summarised on Wikipedia. When requests start failing intermittently, a retry-and-backoff strategy usually helps more than hammering the endpoint; we cover that in how to fix failed Python requests.

Other HTTP methods: POST, PUT, PATCH, DELETE

GET retrieves data, but HTTP defines several methods for different intents. Requests gives you a function for each:

  • requests.get(url, ...): retrieve data.

  • requests.post(url, data=..., json=...): submit data, often to create a resource.

  • requests.put(url, data=..., json=...): replace an existing resource in full.

  • requests.patch(url, data=..., json=...): update part of a resource.

  • requests.delete(url, ...): remove a resource.

Here's a POST that sends a new product as JSON:

import requests

new_gadget = {
    'title': 'Awesome New Gadget',
    'description': 'The latest must-have tech item!',
    'price': 199.99,
    'category': 'electronics'
    # DummyJSON might auto-assign ID, brand etc. or ignore extras
}

response = requests.post(
    "https://dummyjson.com/products/add",
    json=new_gadget  # Use 'json' param to send data as JSON
)

print(f"POST Status: {response.status_code}")  # Should be 200 or 201

if response.status_code in [200, 201]:
    print("Response Data:")
    print(response.json())  # See what the API returns (often the created/updated item)
else:
    print("POST request failed.")

Passing your dictionary via the json= argument tells Requests to serialise it and set the Content-Type header to application/json for you. Deleting works the same way:

import requests

# Let's pretend to delete product with ID 5
product_id_to_delete = 5
response = requests.delete(f"https://dummyjson.com/products/{product_id_to_delete}")

print(f"DELETE Status: {response.status_code}")  # Should be 200 if successful

if response.status_code == 200:
    print("Response Data:")
    print(response.json())  # API might return info about the deleted item
else:
    print("DELETE request failed.")

Matching the method to the intent is part of interacting correctly with REST APIs, and many services reject requests that use the wrong verb.

Routing Requests through a proxy

There are plenty of legitimate reasons to send a request from an IP other than your own: checking how a public page renders for visitors in another country, distributing large-scale public-data collection so you stay within a site's rate limits, or QA-testing geo-specific pricing and content. A proxy server sits between you and the target: your request goes to the proxy, the proxy forwards it, and the target sees the proxy's IP.

Evomi provides ethically sourced, Swiss-based proxies across several types: Datacenter from $0.30/GB, Residential from $0.49/GB using real home IPs, Mobile from $2.20/GB on cellular networks, and Static ISP from $1/IP. Which one you pick depends on the target and your budget; residential and mobile look like ordinary consumer connections, while datacenter is fastest and cheapest for tolerant endpoints.

Configuring a proxy in Requests means building a small dictionary keyed by scheme. Replace the placeholders with your own credentials and endpoint:

import requests

# Replace with your actual Evomi credentials and endpoint
# Example using Residential Proxies (HTTP port)
proxy_user = "YOUR_USERNAME"
proxy_pass = "YOUR_PASSWORD"
proxy_host = "rp.evomi.com"  # Evomi residential proxy endpoint
proxy_port = "1000"          # HTTP port for residential

proxy_url_http = f"http://{proxy_user}:{proxy_pass}@{proxy_host}:{proxy_port}"
# Often the same for HTTP-based proxies
proxy_url_https = f"http://{proxy_user}:{proxy_pass}@{proxy_host}:{proxy_port}"

proxies = {
    "http": proxy_url_http,
    "https": proxy_url_https,
}

target_url = "https://geo.evomi.com/"  # Let's check the IP seen by the server

try:
    # Make the request through the proxy, with a timeout
    response = requests.get(target_url, proxies=proxies, timeout=10)
    # Raises exception for bad status codes (4xx or 5xx)
    response.raise_for_status()
    print("Request successful via proxy!")
    print("Response Content (IP Info):")
    # Should show the proxy's IP details
    print(response.text)
except requests.exceptions.RequestException as e:
    print(f"An error occurred: {e}")

This routes both HTTP and HTTPS traffic through your chosen endpoint. Check your Evomi dashboard for the exact host, credentials, and port for each product (for example, dc.evomi.com for Datacenter or mp.evomi.com for Mobile), since ports differ between HTTP, HTTPS, and SOCKS5. To confirm your requests are actually going out through the proxy, you can also verify the resulting IP with the free IP geolocation checker.

A couple of practical notes: always set a timeout so a stalled proxy connection doesn't hang your script forever, and use response.raise_for_status() when you want failures to surface as exceptions rather than silent bad data. Residential and mobile trials are available on the free plan if you want to test throughput before committing.

Handling authentication

Some endpoints require credentials. The simplest scheme is HTTP Basic Authentication, which Requests supports through the auth parameter as a (username, password) tuple:

import requests

# Example using httpbin's basic-auth endpoint
auth_url = 'https://httpbin.org/basic-auth/myuser/mypass'
username = 'myuser'
password = 'mypass'

try:
    response = requests.get(auth_url, auth=(username, password))
    response.raise_for_status()  # Check for errors
    print("Authentication successful!")
    print("Response JSON:")
    print(response.json())
except requests.exceptions.HTTPError as e:
    if e.response.status_code == 401:
        print("Authentication failed: Incorrect credentials.")
    else:
        print(f"An HTTP error occurred: {e}")
except requests.exceptions.RequestException as e:
    print(f"An error occurred: {e}")

A successful call returns a confirmation payload:

{
  "authenticated": true,
  "user": "myuser"
}

For richer schemes like OAuth, Requests pairs with helper packages such as requests-oauthlib, or you can set the Authorization header manually. Whatever you're accessing, use credentials you legitimately own and stay within the platform's terms of service.

Where to go next

You now have the core toolkit: fetching HTML and JSON, sending data with the right method, reading status codes, passing query parameters cleanly, authenticating, and routing traffic through a proxy. Requests handles the awkward parts of HTTP so you can focus on the logic that matters.

To turn this into real projects, pair Requests with a parser and build out a full pipeline. Our Python web scraping guide is a natural next step, and if you're weighing tooling, the roundup of Python scraping tools compares the popular options. Otherwise, pick a public API that interests you and start pulling data.

If you write Python and touch the web, you'll meet the requests library sooner rather than later. It's the pragmatic choice for sending HTTP requests from code: fetching pages, calling APIs, submitting data, and reading back structured responses. This guide walks through the parts you'll actually use day to day, then gets specific about routing traffic through proxies for legitimate work like public-data collection, QA, and geo testing.

What the Requests library actually does

Requests is a small, readable HTTP client for Python. Its job is to send an HTTP request to a server and hand you back a clean response object you can inspect. A browser does the same thing under the hood, but it also renders pages, runs JavaScript, and manages a UI. For automation you rarely need any of that overhead, so a plain HTTP client is faster, lighter, and easier to script.

If you want the full spec of the underlying protocol, the HTTP semantics are defined in RFC 9110, and the official library reference lives in the Python documentation.

Installing and importing Requests

Requests isn't part of the standard library, so install it with pip:

Then import it at the top of your script:

import requests

That's the whole setup. You're ready to make requests.

Fetching a web page

Start with the most common task: downloading the raw HTML of a page. Save this as fetch_example.py:

import requests

# Let's try fetching content from httpbin, a useful testing site
url = "https://httpbin.org/html"
response = requests.get(url)

# Check if the request was successful (status code 200)
if response.status_code == 200:
    print(response.text)
else:
    print(f"Request failed with status code: {response.status_code}")

Run it with python fetch_example.py and you'll see the page source printed to your console:

<!DOCTYPE html>
<html>
  <head>
  </head>
  <body>
    <h1>Herman Melville - Moby-Dick</h1>
    ...
  </body>
</html>

The response.text attribute holds the body as a string. Note that Requests doesn't parse HTML for you. To pull out links, headings, or specific text you'll pair it with a parser. Beautiful Soup is the usual companion here, and we cover that pairing in detail in our guide to web scraping with Beautiful Soup.

Working with JSON APIs

HTML isn't the only thing you'll fetch. Requests is just as comfortable talking to APIs, which usually exchange data as JSON. JSON is a compact, readable format designed for machine-to-machine communication. A typical payload might look like this:

{
  "id": 42,
  "username": "dev_guru",
  "email": "guru@example.com",
  "isActive": true,
  "roles": ["admin", "editor"]
}

Let's hit a public test API, DummyJSON, which serves fake datasets for exactly this kind of practice:

import requests

# Fetching sample user data
response = requests.get("https://dummyjson.com/users")

# Requests has a built-in JSON decoder
if response.status_code == 200:
    data = response.json()  # 'data' is now a Python dictionary/list
    # print(data)  # Uncomment to see the full structure
else:
    print(f"API request failed: {response.status_code}")

The response.json() method decodes the body into native Python dictionaries and lists, so you skip manual parsing. Suppose you only want each user's name and email:

import requests

response = requests.get("https://dummyjson.com/users")

if response.status_code == 200:
    data = response.json()
    users = data.get("users", [])  # Use .get for safer access
    print("User List:")
    for user in users:
        username = user.get("username", "N/A")
        email = user.get("email", "N/A")
        print(f"- Username: {username}, Email: {email}")
else:
    print(f"API request failed: {response.status_code}")

The output is a tidy list:

User List:
- Username: atuny0, Email: atuny0@sohu.com
- Username: hbingley1, Email: hbingley1@plala.or.jp
-

Using .get() with defaults keeps your loop from crashing when a field is missing, which is common with real-world APIs. For deeper handling of nested structures, see our walkthrough on working with JSON in Python.

Passing query parameters cleanly

Most APIs let you filter, page, or search using query parameters appended after a ?, like https://example.com/search?query=python&limit=10. Building those URLs by hand with string concatenation gets ugly and error-prone fast. Requests handles it with the params argument:

import requests

api_url = "https://dummyjson.com/users"
query_params = {
    'limit': 5,
    'skip': 10,
    'select': 'firstName,lastName,email'  # Select specific fields
}

response = requests.get(api_url, params=query_params)

# The actual URL requested would be something like:
# https://dummyjson.com/users?limit=5&skip=10&select=firstName%2ClastName%2Cemail
# Requests handles the encoding (like %2C for the comma)

if response.status_code == 200:
    data = response.json()
    print(data)
else:
    print(f"Request failed: {response.status_code}")
    print(f"URL attempted: {response.url}")  # See the final URL

Passing a dictionary keeps the code readable and lets Requests take care of URL encoding, so special characters like the comma become %2C automatically. The response.url attribute shows you the final URL that was actually requested, which is handy for debugging.

Reading HTTP status codes

Every response carries a three-digit status code that tells you how the request went. You've already seen 200 OK. Check the code before you touch the body, because processing an error page as if it were valid data is a classic source of silent bugs:

import requests

# Successful request
response_ok = requests.get("https://dummyjson.com/products/1")
print(f"Product 1 Status: {response_ok.status_code}")  # Expected: 200

# Request for a non-existent resource
response_not_found = requests.get("https://dummyjson.com/products/9999")
print(f"Product 9999 Status: {response_not_found.status_code}")  # Expected: 404

The families you'll run into:

  • 2xx (200 OK, 201 Created): success.

  • 3xx (301 Moved Permanently, 302 Found): redirection.

  • 4xx (404 Not Found, 403 Forbidden, 401 Unauthorized): client errors, meaning something about your request was off.

  • 5xx (500 Internal Server Error, 503 Service Unavailable): server errors, meaning the target had a problem.

The full registry is maintained by the IANA and summarised on Wikipedia. When requests start failing intermittently, a retry-and-backoff strategy usually helps more than hammering the endpoint; we cover that in how to fix failed Python requests.

Other HTTP methods: POST, PUT, PATCH, DELETE

GET retrieves data, but HTTP defines several methods for different intents. Requests gives you a function for each:

  • requests.get(url, ...): retrieve data.

  • requests.post(url, data=..., json=...): submit data, often to create a resource.

  • requests.put(url, data=..., json=...): replace an existing resource in full.

  • requests.patch(url, data=..., json=...): update part of a resource.

  • requests.delete(url, ...): remove a resource.

Here's a POST that sends a new product as JSON:

import requests

new_gadget = {
    'title': 'Awesome New Gadget',
    'description': 'The latest must-have tech item!',
    'price': 199.99,
    'category': 'electronics'
    # DummyJSON might auto-assign ID, brand etc. or ignore extras
}

response = requests.post(
    "https://dummyjson.com/products/add",
    json=new_gadget  # Use 'json' param to send data as JSON
)

print(f"POST Status: {response.status_code}")  # Should be 200 or 201

if response.status_code in [200, 201]:
    print("Response Data:")
    print(response.json())  # See what the API returns (often the created/updated item)
else:
    print("POST request failed.")

Passing your dictionary via the json= argument tells Requests to serialise it and set the Content-Type header to application/json for you. Deleting works the same way:

import requests

# Let's pretend to delete product with ID 5
product_id_to_delete = 5
response = requests.delete(f"https://dummyjson.com/products/{product_id_to_delete}")

print(f"DELETE Status: {response.status_code}")  # Should be 200 if successful

if response.status_code == 200:
    print("Response Data:")
    print(response.json())  # API might return info about the deleted item
else:
    print("DELETE request failed.")

Matching the method to the intent is part of interacting correctly with REST APIs, and many services reject requests that use the wrong verb.

Routing Requests through a proxy

There are plenty of legitimate reasons to send a request from an IP other than your own: checking how a public page renders for visitors in another country, distributing large-scale public-data collection so you stay within a site's rate limits, or QA-testing geo-specific pricing and content. A proxy server sits between you and the target: your request goes to the proxy, the proxy forwards it, and the target sees the proxy's IP.

Evomi provides ethically sourced, Swiss-based proxies across several types: Datacenter from $0.30/GB, Residential from $0.49/GB using real home IPs, Mobile from $2.20/GB on cellular networks, and Static ISP from $1/IP. Which one you pick depends on the target and your budget; residential and mobile look like ordinary consumer connections, while datacenter is fastest and cheapest for tolerant endpoints.

Configuring a proxy in Requests means building a small dictionary keyed by scheme. Replace the placeholders with your own credentials and endpoint:

import requests

# Replace with your actual Evomi credentials and endpoint
# Example using Residential Proxies (HTTP port)
proxy_user = "YOUR_USERNAME"
proxy_pass = "YOUR_PASSWORD"
proxy_host = "rp.evomi.com"  # Evomi residential proxy endpoint
proxy_port = "1000"          # HTTP port for residential

proxy_url_http = f"http://{proxy_user}:{proxy_pass}@{proxy_host}:{proxy_port}"
# Often the same for HTTP-based proxies
proxy_url_https = f"http://{proxy_user}:{proxy_pass}@{proxy_host}:{proxy_port}"

proxies = {
    "http": proxy_url_http,
    "https": proxy_url_https,
}

target_url = "https://geo.evomi.com/"  # Let's check the IP seen by the server

try:
    # Make the request through the proxy, with a timeout
    response = requests.get(target_url, proxies=proxies, timeout=10)
    # Raises exception for bad status codes (4xx or 5xx)
    response.raise_for_status()
    print("Request successful via proxy!")
    print("Response Content (IP Info):")
    # Should show the proxy's IP details
    print(response.text)
except requests.exceptions.RequestException as e:
    print(f"An error occurred: {e}")

This routes both HTTP and HTTPS traffic through your chosen endpoint. Check your Evomi dashboard for the exact host, credentials, and port for each product (for example, dc.evomi.com for Datacenter or mp.evomi.com for Mobile), since ports differ between HTTP, HTTPS, and SOCKS5. To confirm your requests are actually going out through the proxy, you can also verify the resulting IP with the free IP geolocation checker.

A couple of practical notes: always set a timeout so a stalled proxy connection doesn't hang your script forever, and use response.raise_for_status() when you want failures to surface as exceptions rather than silent bad data. Residential and mobile trials are available on the free plan if you want to test throughput before committing.

Handling authentication

Some endpoints require credentials. The simplest scheme is HTTP Basic Authentication, which Requests supports through the auth parameter as a (username, password) tuple:

import requests

# Example using httpbin's basic-auth endpoint
auth_url = 'https://httpbin.org/basic-auth/myuser/mypass'
username = 'myuser'
password = 'mypass'

try:
    response = requests.get(auth_url, auth=(username, password))
    response.raise_for_status()  # Check for errors
    print("Authentication successful!")
    print("Response JSON:")
    print(response.json())
except requests.exceptions.HTTPError as e:
    if e.response.status_code == 401:
        print("Authentication failed: Incorrect credentials.")
    else:
        print(f"An HTTP error occurred: {e}")
except requests.exceptions.RequestException as e:
    print(f"An error occurred: {e}")

A successful call returns a confirmation payload:

{
  "authenticated": true,
  "user": "myuser"
}

For richer schemes like OAuth, Requests pairs with helper packages such as requests-oauthlib, or you can set the Authorization header manually. Whatever you're accessing, use credentials you legitimately own and stay within the platform's terms of service.

Where to go next

You now have the core toolkit: fetching HTML and JSON, sending data with the right method, reading status codes, passing query parameters cleanly, authenticating, and routing traffic through a proxy. Requests handles the awkward parts of HTTP so you can focus on the logic that matters.

To turn this into real projects, pair Requests with a parser and build out a full pipeline. Our Python web scraping guide is a natural next step, and if you're weighing tooling, the roundup of Python scraping tools compares the popular options. Otherwise, pick a public API that interests you and start pulling data.

Author

Nathan Reynolds

Web Scraping & Automation Specialist

About Author

Nathan specializes in web scraping techniques, automation tools, and data-driven decision-making. He helps businesses extract valuable insights from the web using ethical and efficient scraping methods powered by advanced proxies. His expertise covers overcoming anti-bot mechanisms, optimizing proxy rotation, and ensuring compliance with data privacy regulations.

Like this article? Share it.
You asked, we answer - Users questions:
Is the Requests library part of Python's standard library?+
What is the difference between response.text and response.json()?+
How do I send POST data as JSON versus form data?+
Why should I use a proxy with Requests?+
How do I set a timeout so my script doesn't hang?+
Which Evomi proxy type should I use with Requests?+

In This Article