Handling OpenAI API Cost Retrieval Changes

If you’ve been using OpenAI’s API, you might have encountered a recent change in their system. Specifically, OpenAI removed an undocumented method for pulling API costs, resulting in the following error:

{
  "error": {
    "message": "Your request to GET /v1/dashboard/billing/usage must be made with a session key (that is, it can only be made from the browser). You made it with the following key type: secret.",
    "type": "server_error",
    "param": null,
    "code": null
  }
}

But don’t worry! There’s an alternative way to retrieve your monthly costs from OpenAI. In this blog post, we’re going to demonstrate how to get your API costs for the current month and store this information in a JSON file.

First, we’ll start with setting up the necessary parameters for the API request:

# billing api parameters
import os
import requests
import json
import datetime
import calendar
import time

url = "https://api.openai.com/v1/usage"
api_key = os.getenv("OPENAI_API_KEY")
headers = {"Authorization": f"Bearer {api_key}"}

Next, we define the cost per token for each model. This data is used to calculate the total cost based on the usage data returned by the API:

model_costs = {
    "gpt-3.5-turbo-0301": {"context": 0.0015, "generated": 0.002},
    "gpt-3.5-turbo-0613": {"context": 0.0015, "generated": 0.002},
    "gpt-3.5-turbo-16k": {"context": 0.003, "generated": 0.004},
    "gpt-3.5-turbo-16k-0613": {"context": 0.003, "generated": 0.004},
    "gpt-4-0314": {"context": 0.03, "generated": 0.06},
    "gpt-4-0613": {"context": 0.03, "generated": 0.06},
    "gpt-4-32k": {"context": 0.06, "generated": 0.12},
    "gpt-4-32k-0314": {"context": 0.06, "generated": 0.12},
    "gpt-4-32k-0613": {"context": 0.06, "generated": 0.12},
    "whisper-1": {
        "context": 0.006 / 60,
        "generated": 0,
    },  # Cost is per second, so convert to minutes
}

Now we’re going to define a helper function get_daily_cost(date), which sends a GET request to the /v1/usage endpoint for a specific date. If the request is successful, it calculates the daily cost based on the response data:

def get_daily_cost(date):
    params = {"date": date}

    while True:
        try:
            response = requests.get(url, headers=headers, params=params)
            response.raise_for_status()
            usage_data = response.json()["data"]
            whisper_data = response.json()["whisper_api_data"]

            daily_cost = 0
            for data in usage_data + whisper_data:
                model = data.get("model_id") or data.get("snapshot_id")
                if model in model_costs:
                    if "num_seconds" in data:
                        cost = data["num_seconds"] * model_costs[model]["context"]
                    else:
                        context_tokens = data["n_context_tokens_total"]
                        generated_tokens = data["n_generated_tokens_total"]
                        cost = (context_tokens / 1000 * model_costs[model]["context"]) + (
                            generated_tokens / 1000 * model_costs[model]["generated"]
                        )
                    daily_cost += cost
                else:
                    print(
                        "[red]model not defined, please add model and associated costs to model_costs object[/red]"
                    )
                    return None

            return daily_cost

        except requests.HTTPError as err:
            if err.response.status_code == 429:
                print("[red]Rate limit exceeded. Sleeping for 69 seconds...[/red]")
                time.sleep(69)
                continue
            print(f"Request failed: {err}")
            return None
        except KeyError as err:
            print(f"Missing key in API response: {err}")
            return None

Then we define the main function get_costs(start_date, end_date) that iterates over each date in the given range, retrieves the daily cost for that date, and adds it to the total cost. If the date is already in the stored data and it’s not today, it skips the API request and adds the stored cost to the total cost. If the date is today and it’s not in the stored data or it was queried more than 60 seconds ago, it retrieves the cost from the API. After retrieving the cost, it updates the stored cost data with the newly retrieved cost and the current query time, and writes this updated data back into the JSON file:

def get_costs(start_date, end_date):
    try:
        with open("j_costs.json", "r") as file:
            stored_costs = json.load(file)
    except FileNotFoundError:
        stored_costs = []

    stored_costs_dict = {d["date"]: d for d in stored_costs}
    total_cost = 0

    for date in range(start_date.toordinal(), end_date.toordinal() + 1):
        date_obj = datetime.date.fromordinal(date)
        date_str = date_obj.isoformat()

        if (
            date_str in stored_costs_dict
            and date_str != datetime.date.today().isoformat()
        ):
            daily_cost = stored_costs_dict[date_str]["cost"]
        else:
            daily_cost = get_daily_cost(date_str)

            if daily_cost is None:
                return None

            query_time = datetime.datetime.now(datetime.timezone.utc)

            stored_costs_dict[date_str] = {
                "cost": daily_cost,
                "query_time": query_time.isoformat(),
            }

        total_cost += daily_cost

    with open("j_costs.json", "w") as file:
        json.dump(list(stored_costs_dict.values()), file)

    return total_cost

To use these functions, you would simply call get_costs(start_date, end_date), where start_date and end_date are datetime.date