This tutorial will show you how to make HTTP API calls using Python. There are many ways to skin a cat and there are multiple methods for making API calls in Python, but today we'll be demonstrating the requests library, making API calls to the hugely popular OpenAI ChatGPT API.
We'll give you a demo of the more pragmatic approach and experiment with their dedicated Software Development Kit (SDK) so you can easily integrate AI into your project. We'll also explain how to make API requests to our Web Scraping API which will give you the power to pull data from any website into your project.
💡 Interested in web scraping with Python? Check out our guide to the best Python web scraping libraries for 2025 and Python Web Scraping Tutorial
HTTP API Call vs Plain HTTP Call
An HTTP call refers to any request made over the Hypertext Transfer Protocol (HTTP). This is a flexible but unstructured way to communicate between a client and a server. Besides semantics carried by the HTTP method (GET, POST, PUT, DELETE, etc.), HTTP doesn't impose any restrictions on the data format or structure.
Essentially, integration with every HTTP-based API could look totally different, as each API might have its own way of structuring requests and responses.
This is precisely why REST (Representational State Transfer) was born. REST APIs are designed to provide a structured way to interact with a service or application, and they typically return data in a structured format like JSON. However, other formats like XML are also possible.
When we talk about making an HTTP API call, we usually imply making an HTTP request to a REST API, which is nowadays one of the most common integration patterns.
Prerequisites
If you want to follow the examples, ensure you have the following:
- Python installed on your machine
- ChatGPT API key (you can get one by signing up at OpenAI )
If you don't want to run the examples, you can just keep on reading :)
Virtual Environment
Tip: in order to avoid juggling various python versions and packages, you might want to consider using a virtual environment. You can create one using the following commands:
python -m venv venv
And activate it:
source venv/bin/activate
Only then install the required packages.
Authenticating with an API
Most APIs will require you to authenticate before you can make requests. This is used not just for security reasons but also to track usage and enforce rate limits.
Token-based authentication is one of the most common solutions.
When you sign up, API providers issue a unique identifier that you need to include in your request headers to authenticate yourself. In most cases, the header name is Authorization, and the value is Bearer YOUR_API_KEY.
Generally, it's a good practice to avoid hardcoding your API key in your code. Instead, you can inject it via environment variables, which is as easy as:
import os
openapi_token = os.getenv("OPEN_AI_API_KEY")
Alternatively, you can use python-dotenv (pip install python-dotenv). To do this, create a .env file, define your environment variable there:
OPEN_AI_API_KEY=YOUR_API_KEY_HERE
And now, you can use the library to load the environment variables from the .env file:
from dotenv import load_dotenv
load_dotenv()
openai_token = os.getenv("OPEN_AI_API_KEY")
Issuing Our First API Call
Let's do our first API call ever. We'll use the ChatGPT API from OpenAI. The API allows you to interact with a chatbot model and generate responses based on the input you provide.
import requests
url = "https://api.openai.com/v1/chat/completions"
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {OPENAI_API_KEY}"
}
data = {
"model": "gpt-4o-mini",
"messages": [
{
"role": "user",
"content": "I'm writing a technical article about performing API calls with Python and using ChatGPT as an example. What operations would you showcase?"
}],
"temperature": 0.7
}
response = requests.post(url, headers=headers, json=data)
And that's it! We've made our first API call using Python and received a response, which contains the status code, headers, and the response body.
Handling the Response
The fact that we received a response doesn't mean it was successful. We should always check the status code to ensure everything went well.
HTTP status codes:
- the 2xx range indicates success
- the 4xx range indicates client errors
- the 5xx range indicates server errors
However, beware, not all APIs return errors in the 4xx or 5xx range. Some APIs might return a 200 status code even if the request failed. In such cases, you should check the response body for error messages. API documentation should be the main source of truth here.
This is a bad practice that goes against REST and HTTP specifications, but it happens... and you should be prepared for it.
Let's now add some error handling to our code:
# ...
if response.ok:
print("Response content:", response.json())
elif 400 <= response.status_code < 500:
print("Client error:", response.status_code, response.json())
else:
print("Error:", response.status_code, response.text)
response.ok is a convenient way to check if the status code is in the 2xx range.
For example, if we run our script too often, we might hit the rate limit. In this case, the response will contain a 429 status code, and the response body will contain an error message:
Client error: 429
{
'error': {
'message': 'You exceeded your current quota, please check your plan and billing details. For more information on this error, read the docs: https://platform.openai.com/docs/guides/error-codes/api-errors.',
'type': 'insufficient_quota',
'param': None,
'code': 'insufficient_quota'
}
}
If we run the script with an invalid API key, we'll get a 401 status code:
Client error: 401
{
'error': {
'message': 'Incorrect API key provided: invalids***************************************************************************************************************************************************************SYQA. You can find your API key at https://platform.openai.com/account/api-keys.',
'type': 'invalid_request_error',
'param': None,
'code': 'invalid_api_key'
}
}
Fortunately, I haven't managed to get a 5xx status code from the OpenAI API :)
On a successful run, we receive a response with the generated text:
{'choices': [{'finish_reason': 'stop',
'index': 0,
'logprobs': None,
'message': {'content': 'In your article, you could showcase how '
'to perform GET and POST requests to the '
'ChatGPT API, demonstrating how to '
'retrieve generated text based on user '
'input. Additionally, illustrate error '
'handling by capturing and managing '
'exceptions that may arise during the API '
'call. Finally, consider highlighting '
'rate limiting and basic authentication '
'techniques to ensure robust and secure '
'API interactions.',
'refusal': None,
'role': 'assistant'}}],
'created': 1737393972,
'id': 'chatcmpl-ArppMkITcOBMdIr2nsFZzych7rHNJ',
'model': 'gpt-4o-mini-2024-07-18',
'object': 'chat.completion',
'service_tier': 'default',
'system_fingerprint': 'fp_72ed7ab54c',
'usage': {'completion_tokens': 68,
'completion_tokens_details': {'accepted_prediction_tokens': 0,
'audio_tokens': 0,
'reasoning_tokens': 0,
'rejected_prediction_tokens': 0},
'prompt_tokens': 41,
'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0},
'total_tokens': 109}}
Often, responses will contain more information that you might need. In this case, we're interested in the message.content field, which contains the generated text.
Luckily, ChatGPT's idea for the article aligns with what we're doing here!
Leveraging OpenAPI Specification to generate our own (Software Development Kit) SDK
Some API providers decide to go an extra mile and provide an OpenAPI Specification (formerly known as Swagger). This is a standard way to describe REST APIs, and it provides a machine-readable format that can be used to generate client libraries and Software Development Kits SDKs, which makes working with the API much easier.
You can use the openapi-python-client library to generate a client library/SDK for the API based on the OpenAPI Specification. This library will provide you with a Python client that you can use to interact with the API without having to write the HTTP requests manually. The generated SDK simplifies API interactions by providing high-level client methods, making API consumption easier for developers.
Here's an example OpenAPI specification for a fictitious API:
openapi: 3.0.3
info:
title: Task API
version: 1.0.0
description: A simple API for managing tasks.
paths:
/tasks:
get:
summary: Get all tasks
responses:
200:
description: A list of tasks
content:
application/json:
schema:
type: array
items:
$ref: '#/components/schemas/Task'
post:
summary: Create a new task
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/Task'
responses:
201:
description: Task created
/tasks/{taskId}:
get:
summary: Get a task by ID
parameters:
- name: taskId
in: path
required: true
schema:
type: string
responses:
200:
description: The requested task
content:
application/json:
schema:
$ref: '#/components/schemas/Task'
404:
description: Task not found
delete:
summary: Delete a task by ID
parameters:
- name: taskId
in: path
required: true
schema:
type: string
responses:
204:
description: Task deleted
components:
schemas:
Task:
type: object
properties:
id:
type: string
title:
type: string
completed:
type: boolean
required:
- id
- title
- completed
You can generate a client library for this specification using openapi-generator:
openapi-generator generate -i openapi.yaml -g python -o ./python-client
And then you can use the generated client library to interact with the API:
import openapi_client
from openapi_client.models.task import Task
from openapi_client.rest import ApiException
from pprint import pprint
configuration = openapi_client.Configuration(host = "http://localhost")
with openapi_client.ApiClient(configuration) as api_client:
api_instance = openapi_client.DefaultApi(api_client)
try:
api_response = api_instance.tasks_get()
print("The response of DefaultApi->tasks_get:\n")
pprint(api_response)
except Exception as e:
print("Exception when calling DefaultApi->tasks_get: %s\n" % e)
As you can see, instead of manually composing HTTP requests, you can interact with a high-level client.
Naturally, this API does not exist, so this code will fail when trying to connect, but you get the idea.
Leveraging an SDK
Many API providers offer SDKs (Software Development Kits) that ultimately simplify the process of interacting with APIs, so they’re worth searching for when you start working with an API.
SDKs are essentially libraries that wrap the API calls and provide a more user-friendly interface. They often include methods that correspond to the API endpoints, and they handle the authentication and request/response serialization for you.
OpenAI API Python SDK Example
For example, OpenAI provides a Python SDK as well - you can install it using pip install openai.
Let's see a simple example:
from openai import OpenAI
client = OpenAI()
chat = client.chat.completions.create(
messages=[
{
"role": "user",
"content": "What's the best way to showcase ChatGPT Python SDK? Answer in no more than 3 sentences.",
}
],
model="gpt-4o-mini",
)
reply = chat.choices[0].message.content
print(f"ChatGPT: {reply}")
Notice that we didn't worry about choosing the right endpoint, composing the request, or handling the response. The SDK took care of all that for us.
Additionally, OpenAI automatically reads the API key from the environment variable OPENAI_API_KEY, so we don't have to pass it explicitly.
ScrapingBee Python SDK Example
Here’s an example of using our Python SDK to make calls to our web scraping API to scrape headings from one of our blogs. Sign up here for 1,000 free credits and run this example:
pip install scrapingbee
from scrapingbee import ScrapingBeeClient
import json
# Initialize the ScrapingBee Client
sb_client = ScrapingBeeClient(api_key='YOUR_SCRAPINGBEE_API_KEY')
URL = 'https://www.scrapingbee.com/blog/best-python-web-scraping-libraries/'
# Extract Structured Data Using CSS selectors.
response = sb_client.get(
URL,
params={
'extract_rules': {
'title': 'h1',
'sections': {
'selector': 'h2',
'type': 'list',
'output': 'text'
}
}
}
)
data = json.loads(response.content)
print(json.dumps(data, indent=2))
'''
OUTPUT:
{
"title": "7 Best Python Web Scraping Libraries for 2025",
"sections": [
"1. ScrapingBee",
"2. Selenium",
"3. Playwright",
"4. Requests-HTML",
"5. Scrapy",
"6. BeautifulSoup",
"7. MechanicalSoup",
"Conclusion",
"Tired of getting blocked while scraping the web?"
]
}
'''
And if you don’t want to set up your own Python environment, here’s a link to an (easy to use Google Colab)[https://colab.research.google.com/drive/1kLBMeKXcHQ-IesfrbnvbKVB0j9F2-Ht5?usp=sharing] that gives you an example of our Python SDK making an API call to scrape the headlines from the Guardian’s homepage, give it a spin.
Conclusion
We've shown you how to make API calls in Python using the requests library, covering topics such as handling responses, and authenticating with an API key. We've also shown you how to use dedicated SDKs which should be your first port of call when working with APIs as they make working with an API super easy.
