Calling an LLM from Python: it's just an API request

What actually happens when your Python code "talks to" a language model?

Short answer: it sends an HTTP request. You POST some JSON with your prompt to the provider's URL, include your API key in a header, and get a JSON reply back with the model's answer a few levels deep. If you have called any web API from Python, you already know most of it.

This post walks through that request piece by piece: the URL, the headers, the body, the reply, and what to do when it fails. It follows The Python you need to build an AI feature, which covers why these skills matter.

A note on the code: the examples are here to read and understand, not to copy and run. Every provider has its own URL, field names and model names, and they change. Check your provider's documentation for the exact details. The Python patterns are the same everywhere, and those are what this post is about.

The whole thing in one function

Here is a complete call, using the popular requests library. The provider URL and model name are placeholders:

import os
import requests
 
API_URL = "https://api.example-llm.com/v1/chat"  # placeholder
 
def ask_model(prompt):
    response = requests.post(
        API_URL,
        headers={"Authorization": f"Bearer {os.environ['LLM_API_KEY']}"},
        json={
            "model": "example-model",  # placeholder
            "messages": [{"role": "user", "content": prompt}],
        },
        timeout=30,
    )
    response.raise_for_status()
    data = response.json()
    return data["choices"][0]["message"]["content"]

That is about a dozen lines, and only the URL and the body's field names are specific to AI. The rest of this post takes it apart.

1. It's a POST request

You are sending data, your prompt, so it is a POST, the same method you would use to submit a form or create a record. requests.post(url, ...) sends it. How REST APIs work covers HTTP methods, and Sending Data With Post (opens in a new tab) practices POST requests in Python.

2. The API key goes in a header

The provider needs to know who is calling, so you send a secret key with every request. It goes in a header, commonly Authorization: Bearer <key>, though some providers use a different header name.

headers={"Authorization": f"Bearer {os.environ['LLM_API_KEY']}"}

Two rules about that key:

• Never write it in your code. Read it from an environment variable, like os.environ["LLM_API_KEY"]. Code gets shared and committed to Git. A key in the code is a key anyone can use.
• Never send it to the browser. If front-end JavaScript calls the provider directly, anyone can open developer tools and read your key. Calls belong on your server, which part 3 of this series covers.

3. The prompt goes in a JSON body

The json= argument turns a Python dictionary into a JSON body and sets the right content type for you:

json={
    "model": "example-model",
    "messages": [{"role": "user", "content": prompt}],
}

Many providers use a list of messages like this, where each message has a role, such as user, and some content. The exact field names vary, which is why this is the part to check against your provider's docs. In Python terms it is just a dictionary containing a list of dictionaries.

4. Reading the nested reply

The reply is JSON, which response.json() turns into Python dictionaries and lists. A common shape looks something like this:

{
    "id": "resp_123",
    "choices": [
        {
            "message": {
                "role": "assistant",
                "content": "A for loop runs code once per item in a list."
            }
        }
    ],
    "usage": {"input_tokens": 12, "output_tokens": 14}
}

The answer is four steps in:

data["choices"][0]["message"]["content"]

Read it left to right: the choices key holds a list, [0] takes the first item, that item's message is a dictionary, and its content is the text. That is exactly the skill in Reading Nested JSON (opens in a new tab). Other providers nest it differently, but you unpack it the same way.

5. Handling failure

A model call can fail in several ways, and each needs a different response:

Two lines in the function above do the groundwork. timeout=30 stops the request from waiting forever. Without it, a stuck connection can hang your program indefinitely. response.raise_for_status() turns any 4xx or 5xx status into an exception, so an error page doesn't get treated as an answer.

Then wrap the call in try/except:

try:
    answer = ask_model("Explain a for loop in one sentence.")
except requests.Timeout:
    answer = None
    print("The model took too long. Try again.")
except requests.HTTPError as error:
    answer = None
    if error.response.status_code == 429:
        print("Too many requests. Wait a moment and retry.")
    else:
        print(f"The request failed: {error}")
except (KeyError, IndexError):
    answer = None
    print("The reply wasn't in the expected shape.")

Each except matches one kind of failure from the table. That is the same try/except you would use anywhere in Python. Python try/except: handle errors without crashing covers the basics, and When A Request Fails (opens in a new tab) practices failed requests.

6. Don't trust the answer blindly

Even a successful call can return something you didn't ask for. If you need a specific format, like one word or valid JSON, check it before you use it, and treat a bad answer like any other failure. Part 1 of this series shows how to validate a reply.

Common mistakes

• Hardcoding the API key. Read it from an environment variable, every time.
• No timeout. requests waits indefinitely by default. Always pass timeout=.
• Skipping raise_for_status(). Without it, an error reply can slip through and crash later with a confusing KeyError.
• Retrying a 401 forever. A bad key won't fix itself. Only retry errors that can succeed later, like timeouts and 429.
• Calling the provider from the browser. It exposes your key. Call it from your server.
• Copying field names from an old tutorial. Providers change their APIs. Check the current docs.

Quick recap

• Calling a model is a POST request with a JSON body.
• The API key goes in a header, and comes from an environment variable.
• The answer is nested in the JSON reply. Unpack it one key at a time.
• Set a timeout, call raise_for_status(), and catch each kind of failure.
• Check the answer before you use it.

Frequently asked questions

How do I call an AI model from Python?

Send an HTTP POST request to the provider's API with your prompt in a JSON body and your API key in a header, then read the answer from the JSON reply. Many providers also offer an official Python library that wraps this same request.

Should I use the provider's Python library or requests?

Either works. The official library saves some typing and keeps up with the provider's changes. Writing the request yourself, at least once, shows you what the library is doing underneath, which helps when something breaks.

Where should I store my API key?

In an environment variable, read with os.environ. Never in your code, and never in front-end JavaScript that runs in the browser.

How do I handle rate limits?

A rate limit usually comes back as status 429. Catch it, wait a little, and retry, and wait a bit longer each time it happens again. Don't retry errors that won't change, like a bad key.

Do your courses teach how to use LLMs?

No. Intermediate Python teaches the parts underneath this post: requests, nested JSON and error handling. Those are what make an LLM call work reliably.

Practice

The request skills in this post are in the course, and the exercises run in your browser. Start with How An API Call Works (opens in a new tab), send data in Sending Data With Post (opens in a new tab), then handle errors in When A Request Fails (opens in a new tab). There is nothing to install.

Keep going

Intermediate Python (opens in a new tab) covers requests, JSON and error handling. Part 3 of this series wraps this call in your own API, which is what Building APIs with FastAPI (opens in a new tab) teaches you to build.

Check out our related courses

Intermediate

Intermediate Python

Write interactive Python programs that ask questions, clean up text, handle errors, process data files, call real APIs, and model games with classes.

87 lessons

Instructor

Brad Traversy

4.8

21 enrolled

Intermediate

Building APIs with FastAPI

Build JSON APIs in Python with FastAPI. Learn how HTTP works by calling real APIs, then write routes, read parameters and request bodies, validate data, and send honest status codes.

57 lessons

Instructor

Brad Traversy

4.8

31 enrolled