How to use the ChatGPT API

What a request is made of

A request is an array of messages. Each one carries a role, and the model reads them in order. Roles are how it tells an instruction apart from a line of conversation.

system

The instruction that governs the whole conversation: tone, output format, limits. It goes first and stays the same between turns.

user

The person's message. What the model is answering right now.

assistant

The model's own earlier replies. They are what let it remember what it already said.

curl https://api.llm-gate.tech/v1/chat/completions \
  -H "Authorization: Bearer $CHATGPT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-6.1-sol",
    "messages": [
      { "role": "system", "content": "Отвечай кратко, по-русски." },
      { "role": "user", "content": "Что такое токен?" }
    ]
  }'

Running a conversation

The API is stateless. The model remembers nothing from the previous call, so you resend the conversation every time: the system instruction, every earlier turn, and the new question. You append the model's reply to the array yourself, otherwise it won't see it on the next step.

messages = [
    {"role": "system", "content": "Отвечай кратко, по-русски."},
    {"role": "user", "content": "Что такое токен?"},
    # ответ модели возвращаем обратно в массив
    {"role": "assistant", "content": "Токен это кусок текста примерно в 4 символа."},
    {"role": "user", "content": "А сколько их в слове «программирование»?"},
]

response = client.chat.completions.create(
    model="gpt-6.1-sol",
    messages=messages,
    max_completion_tokens=300,
)

History grows and the bill grows with it: you pay for every input token on every request. On long conversations, trim old turns or fold them into a short summary.

Streaming: output as it is generated

By default the answer arrives in one piece once the model has finished. On long answers that reads as a freeze lasting tens of seconds. With streaming the text arrives in chunks right away, the way the ChatGPT web interface behaves.

stream = client.chat.completions.create(
    model="gpt-6.1-sol",
    messages=messages,
    stream=True,
)

for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)

• Chat interfaces where someone is watching the screen and waiting
• Long outputs: prose, code, detailed breakdowns
• Not needed for background work: classification, field extraction, scheduled jobs

The parameters that actually matter

There are more, but in practice you will touch these four. The rest matter only in narrow cases.

Errors and what they mean

The status code tells you whose problem it is and whether retrying helps. Half of these clear on a retry; half never will.

Retries with growing delay

Retrying immediately is pointless: on a rate limit you get the same error and burn quota. Grow the pause with each attempt and cap the number of tries. The official SDKs do this out of the box; if you call the API directly, the shape is this.

import time

for attempt in range(5):
    try:
        response = client.chat.completions.create(
            model="gpt-6.1-sol",
            messages=messages,
            max_completion_tokens=300,
        )
        break
    except Exception:
        if attempt == 4:
            raise
        time.sleep(2 ** attempt)  # 1, 2, 4, 8 секунд

Keeping the bill down

The bill tracks tokens, not requests. Four things account for most of the savings.

FAQ