text-embedding-3-small Dimensions: A 256, 512 and 1536 Retrieval Test

Choose embedding dimensions by measuring retrieval on your own labelled queries and budgeting vector storage separately. A smaller vector is not automatically cheaper to generate, and a larger vector is not automatically better for every corpus.

Crazyrouter is the gateway used to test text-embedding-3-small at 256, 512 and 1536 dimensions; all three settings ranked the expected document first for three synthetic queries.

By Brian J · Tested November 3, 2026

Which dimensions should I use for text-embedding-3-small?#

Start with a representative retrieval evaluation and compare a few dimension settings using the same documents and queries. This small test tied at 3/3 top-one matches for 256, 512 and 1536 dimensions. That tie demonstrates a working comparison method; it does not establish 256 as the best choice for a production corpus.

The experiment keeps the retrieval task fixed#

Four documents cover password reset, invoice download, leaked-key rotation, and profile language. Three queries have manually specified relevant document IDs. For each dimension setting, one embedding request includes the same four documents and three queries. The script validates response indices, vector lengths, and finite numeric elements, then ranks documents by cosine similarity.

The ranking uses exhaustive comparison over four documents, not an approximate nearest-neighbor index. Scores can differ between dimensions even when the first result is the same. Do not compare a single cosine threshold across dimension settings without calibrating it.

November 3 test results#

The tests ran on Windows with Python 3.10.2 and synthetic inputs. The API host was https://api.crazyrouter.com; the complete request path, model and parameters appear in the runnable code below. These are dated observations, not a guarantee of future behavior.

Observed response or request IDs (including failed-output responses):

1793713093877298
1793713095026198
1793713097279796

Embedding body response IDs were absent; the listed values are observed x-request-id headers.

Separate vector storage from API usage#

A float32 vector uses four bytes per component. The tested dimensions therefore require 1,024, 2,048 and 6,144 raw vector bytes respectively. One million 1536-dimensional float32 vectors use 6.144 GB in decimal units before IDs, metadata, index overhead or replication. The JSON response has a different wire size and is not a float32 storage file.

All three requests reported 78 prompt tokens for the same seven texts. This observation does not show a lower generation bill at smaller dimensions. The OpenAI embeddings documentation describes the dimensions parameter; gateway billing must be checked separately against the actual applicable schedule and usage ledger.

Plan a dimension change as an index migration#

Query and document vectors must belong to the same model, normalization scheme and dimension. Write a new namespace or index, re-embed the corpus, evaluate it, then switch reads deliberately. Do not truncate an existing stored vector or mix old and new lengths merely because an API call succeeded.

For deployment, expand beyond these easy examples: ambiguous questions, near-duplicate documents, domain language, no-answer cases and access-filtered retrieval. Measure recall at the relevant k, latency over repeated runs, and real index memory. Three requests cannot provide a p95 latency estimate.

Reproduce the test#

Install requests, set CRAZYROUTER_API_KEY in the process environment, save the complete script below, and run it with Python. It sends the synthetic requests shown in the code using your own authorized credential. It does not print authorization headers. Output includes the observed fixture outcomes; completing the script does not mean every model answer passed.

import json
import math
import os
import time
import requests

ENDPOINT="https://api.crazyrouter.com/v1/embeddings"
def embed(texts,dimensions,receipts):
    client=requests.Session();client.trust_env=False
    payload={"model":"text-embedding-3-small","dimensions":dimensions,"encoding_format":"float","input":texts}
    start=time.perf_counter()
    try:
        r=client.post(ENDPOINT,headers={"Authorization":"Bearer "+os.environ["CRAZYROUTER_API_KEY"]},json=payload,timeout=(10,45),allow_redirects=False)
        r.raise_for_status();data=r.json();rows=sorted(data["data"],key=lambda x:x["index"])
        assert [x["index"] for x in rows]==list(range(len(texts)))
        vectors=[x["embedding"] for x in rows]
        assert all(len(v)==dimensions and all(type(n) in (int,float) and math.isfinite(n) for n in v) for v in vectors)
        receipts.append({"http":r.status_code,"request":payload,"response_id":data.get("id"),"request_id":r.headers.get("x-request-id"),"usage":data.get("usage",{}),"seconds":round(time.perf_counter()-start,3),"vectors":len(vectors),"dimensions":dimensions,"validated":True})
        return vectors
    finally:client.close()

def cosine(a,b):
    return sum(x*y for x,y in zip(a,b))/(math.sqrt(sum(x*x for x in a))*math.sqrt(sum(y*y for y in b)))
def main():
    docs=["Reset a forgotten password using the reset link sent to your email.","Request an invoice from the billing dashboard after payment.","Rotate a leaked API key by creating a replacement and revoking the old key.","Change the display language in profile preferences."]
    queries=[("I forgot my password. How do I sign in?",0),("Where can I download a payment invoice?",1),("My secret API key was exposed. What should I do?",2)]
    receipts=[];results=[]
    for dimensions in [256,512,1536]:
        vectors=embed(docs+[q for q,_ in queries],dimensions,receipts)
        ranked=[]
        for j,(query,expected) in enumerate(queries):
            scores=[cosine(vectors[4+j],v) for v in vectors[:4]]
            found=max(range(4),key=lambda i:scores[i])
            ranked.append({"query":query,"expected_doc":expected,"top_doc":found,"top1_correct":found==expected,"scores":scores})
        results.append({"dimensions":dimensions,"float32_bytes_per_vector":dimensions*4,"top1_correct":sum(x["top1_correct"] for x in ranked),"queries":ranked})
    print(json.dumps({"endpoint":ENDPOINT,"requests":receipts,"results":results,"corpus":docs,"storage_excludes":"JSON, IDs, metadata and index overhead"}))
if __name__=="__main__":main()

Sources and related implementation guides#

Official references checked November 3, 2026: OpenAI embeddings guide. Their documented behavior is distinct from the gateway results measured here.

• Incremental Embeddings in SQLite: Test Deduplication and Crash Recovery
• AI API Retries in Python: Retry-After, Timeouts, and Safe Replay
• AI API Response Caching in Python: TTL, Isolation, and a Live Test
• AI SaaS Unit Economics: Calculate Cost per Usable Output
• Lower AI API Token Use: A Paired Prompt-Compression Test in Python

FAQ#

Did 1536 dimensions outperform 256 here?#

No. Both, and 512, scored 3/3 on this tiny top-one fixture. The dataset is too small to establish equivalence.

Is the storage calculation a database size estimate?#

Only the raw float32 component is calculated. Index and application overhead are excluded.

Will smaller dimensions lower API charges?#

This test did not query charges. Each setting reported the same 78 input tokens for these texts.

Can I mix dimensions in one similarity comparison?#

No. Keep the model and dimension consistent for the query and document vectors.

Why are response IDs sometimes absent?#

These embedding responses did not supply a body response ID. The observed x-request-id values are recorded instead; no IDs were invented.

How should I choose a setting after this tutorial?#

Run the included comparison on representative labelled examples and measure your actual storage and retrieval requirements.

Last actually tested: November 3, 2026. This API-dependent article is due for review December 3, 2026.