Image Search Solutions on OCI

Image search looks like one problem: put an embedding next to every image, and find the nearest ones. It is actually three. The system you build for a million images is not the system you build for a hundred million, and the thing that kills the small one (you hand-rolled a vector index) is rarely the thing that kills the big one (recall collapse under a shifting catalog). This guide walks the same workload three times — at three scales — so the architecture grows only when the previous one breaks, and every component earns its place.

Before the scenarios, three properties every image search system must have, no matter the scale. They are the difference between a demo and a product.

The three things that make it production-grade

An invariant, not a feature: the model version and the index are bound. Every embedding is produced by one specific model. Change the model and the new vectors are not comparable to the old ones — you have rebuilt the index from zero, or you are silently returning garbage. Stamp model_version onto every record and treat a model change as a full re-index, not an upgrade.

An evaluation harness, run like CI. Search quality drifts as the catalog grows and the data distribution shifts. Without a golden set of query → relevant-images pairs that you run on every change, you cannot safely move the system at all. This is covered in detail later — but treat it as foundational, not optional.

One log line that serves three purposes. Every query should emit one record: query_hash · latency_ms · top_k_ids · index_version · model_version. That single line is simultaneously your production debugging trail, your usage/billing meter, and the raw material you mine for new evaluation cases. Write it everywhere; it pays for itself.

Scenario 1 — Small scale: serverless, on-demand, nothing to manage

A catalog in the low millions of images. Traffic is bursty and unpredictable. The defining constraint is operational: you do not want to run a database, a GPU farm, or a vector index. The whole point is to ship the feature and stop thinking about infrastructure.

This is where OCI Generative AI's fully-managed stack fits exactly. It provides the embedding model and the vector store as a single on-demand service, and you call them over an API. Nothing to load, nothing to scale, nothing to patch.

The flow is deliberately thin. New images land in Object Storage; an ObjectCreate event invokes an OCI Function, which calls the Generative AI embedding endpoint, and upserts the vector into the managed vector store. At query time another Function call encodes the query and runs the ANN search against the same store. Both the encoder and the store are someone else's problem.

Where it breaks, and motivates the next scenario. The managed vector store is simple, but it is not a relational database. The moment you need to filter search results by structured attributes — images from this user, in this date range, with this status — you are either maintaining a parallel database and joining in application code, or wishing you had one store that did both. You also outgrow the serverless tier's throughput ceiling when indexing becomes a steady stream rather than a burst. That is the moment for Scenario 2.

Scenario 2 — Small-to-medium: when you need a real database

The catalog is still in the millions to low tens of millions, but the requirements have changed: the search results must be filtered and joined with relational data, and you want the search and the business data to live in one consistent store rather than two stores you reconcile by hand.

This is where Autonomous Database (Oracle Database 23ai) is the right answer. Its native AI Vector Search gives you a VECTOR type with HNSW indexing inside a production relational database — so a single SQL query does the vector search and the structured filtering in one transactionally-consistent operation. You keep using OCI Generative AI's managed embedding service (no reason to run your own model yet), but you host the vectors where your data already lives.

Both the compute tier and Autonomous Database scale independently — you can size the query fleet to traffic and let ADB auto-scale storage and compute as the catalog and load grow. Because the vectors and the relational rows share a store, "search this user's private images, sorted by relevance, excluding archived" becomes one query, not a two-system dance.

The database is doing two jobs here — vector search and relational storage. That is a strength at this scale, and a liability past it. Watch the HNSW index build time and query latency as the vector count climbs; when either degrades, the relational store has become the wrong place for a hundred-million-vector index. That is the trigger for Scenario 3.

Scenario 3 — Large scale: dedicated, sharded, image-grade

Hundreds of millions to billions of images. The defining constraints have inverted: the catalog is now image-first (not relational-first), the vector index is now the system's heart, and throughput is high enough that a managed embedding endpoint may become the bottleneck or the line item. You now deliberately run infrastructure, because the scale earns it.

Two changes define this scenario. First, the vector store becomes a dedicated, sharded OpenSearch cluster (via OCI Search with OpenSearch) — its k-NN plugin and ability to shard across nodes are built for this cardinality. Second, you move embedding to self-hosted models on GPU for the indexing path (where you control batching and cost via preemptible instances), while deciding per-case whether the query encoder stays on managed Generative AI or also moves in-house.

At this scale the indexing path becomes asynchronous and buffered: image creates flow through OCI Streaming (Kafka-compatible) into ingestion workers, which feed the GPU encoder in batches and write vectors plus metadata into OpenSearch. The query path gets a Redis hot-query cache in front of the ANN search, and thumbnails are served through a CDN partner — because at hundreds of millions of images, the perceived speed of the system is dominated by thumbnail delivery, not by the vector math.

A note on OCI and the edge. OCI does not offer a first-party CDN. At this scale you front thumbnail delivery with a partner (Akamai, Cloudflare). And use a Service Gateway so private subnets reach Object Storage over Oracle's backbone — without paying NAT egress, which matters a great deal when you are bulk-fetching images.

The evaluation harness — your search-quality CI

Every scenario above will degrade if you cannot measure recall. Treat search quality like a test suite: a golden dataset, an eval job, and a gate.

• Golden set lives in Object Storage as JSONL. Seed it with hand-labeled examples, then keep mining it from real query logs — the logging you build for operations is the same logging that grows your test set.
• Run it on every change. A model swap, a re-shard, a new chunking strategy — all are regression risks. No eval pass, no production deploy.
• Read three metrics against explicit red lines: recall@k (did the right thing make the cut), mAP/nDCG (is it ranked well), p99 latency (is the experience still fast). They trade off against each other; decide the acceptable trade-off in writing, not in the incident channel.

The highest-leverage finding from running this consistently: recall often collapses slowly as the catalog grows and the data distribution shifts. Without the harness, you find out from users. With it, you find out before a release.

Logging, metrics, and the one log line

OCI gives you a fully-native observability stack; the job is wiring it up and deciding what to log.

The one non-negotiable: every query emits query_hash · latency_ms · top_k_ids · index_version · model_version. That line is your debugging trail when a user reports bad results, the meter for your usage statistics, and the source you sample to grow the golden set. Three needs, one record — instrument it from day one in every scenario.

Usage statistics and cost estimation

Track usage by aggregating that query log in Logging Analytics. Track cost with OCI Cost Analysis and Budgets, and tag every resource (env, tier, service) so you can chargeback by dimension. Order-of-magnitude monthly estimates — verify everything in the OCI Pricing Calculator, as region and discounting swing these widely:

The cost lever that matters most is on the indexing path, not the query path: use preemptible GPU for batch embedding (failures are safe to retry), and keep the query encoder warm and resident — its cold start is your P99 latency. Recall that the single most cost-effective investment is the evaluation harness: every point of recall you gain is a page the user does not have to load.

Choosing the scenario

You do not pick the scenario you want; you pick the one your constraints allow, and you design so you can graduate between them.

• Scenario 1 when operational simplicity is the product and you have no relational filtering needs — managed embedding + managed vector store, nothing to run.
• Scenario 2 when search and relational data must live together — Autonomous Database 23ai does vectors and SQL in one consistent store.
• Scenario 3 when the catalog is image-first, the cardinality is high, and the vector index is now the heart of the system — dedicated OpenSearch and self-hosted embedding, with the full async indexing pipeline.

Build the scenario you are in, but from day one carry the three properties that travel across all of them: the model-version invariant, the evaluation harness, and the one query log line. Those three are what let you scale gradually — moving from one scenario to the next because the current one genuinely broke, not because the architecture was wrong from the start.