Search for "running shoes" in a normal database, and it looks for rows that contain those exact words. A listing titled "lightweight jogging sneakers" never shows up, even though it is exactly what you wanted. Traditional databases are excellent at exact matches and terrible at "things like this".
A vector database fixes that. It stores data as embeddings, which are numerical fingerprints of meaning, and answers one question extremely well: "which stored items are closest in meaning to this one?" That single ability sits underneath semantic search, product recommendations and most RAG chatbots you see today.
When I explain vector databases to students, I start with a library. A normal library shelves books by title or accession number. Now imagine a library that shelves books by what they are about, so every book on cricket strategy sits together no matter what its title says. That is the heart of a vector database. By the end of this page you will know how one works, how it differs from SQL, which popular names to know, and when you can skip one entirely.
What is a vector database in simple terms?
In simple terms, a vector database is a storage system that keeps data as lists of numbers representing meaning, and finds the items most similar to whatever you search for.
Go back to the library-by-meaning idea. In this library:
- Every book has a location based on its topic, not its title. Books on Indian history cluster together, and books on machine learning sit in another corner.
- When you walk in with a question, the librarian works out where your question would sit on the shelves.
- The librarian then hands you the books nearest to that spot, even if none of them use the exact words in your question.
In a vector database, the "locations" are embeddings, lists of hundreds or thousands of numbers produced by an embedding model. The "librarian" is a search algorithm that finds the closest vectors very quickly, even among millions.
Along with each vector, the database usually stores the original content (a paragraph, a product title, an image link) and metadata such as the date, category or price. That lets you search by meaning and filter by facts at the same time.
What is a vector database in AI and why is it used?
In AI, a vector database is the memory layer that lets applications store embeddings and retrieve relevant information by meaning, which is why it shows up in so many LLM apps.
AI systems increasingly turn everything into embeddings: documents, questions, images, user profiles and products. Once you have millions of those vectors, you need somewhere to keep them and a fast way to search them. A spreadsheet or a regular table can store the numbers, but it was not designed to answer "find the 5 closest vectors out of 10 million" in a few milliseconds.
The main reasons AI teams use vector databases:
- Semantic search. Find documents by meaning, so "salary credited late" matches a help article titled "Payroll delays".
- RAG chatbots. Give a large language model the right passages from your documents before it answers.
- Recommendations. Suggest videos, courses or products similar to what a user liked.
- Agent memory. Let an AI agent store notes from earlier conversations and recall the relevant ones later.
- Duplicate and fraud detection. Spot near-identical complaints, listings or images that are worded differently.
- Image and multimodal search. Search a photo collection by typing a description.
The common thread is simple: whenever an app needs "find similar" at scale, a vector database is a strong candidate.
How does a vector database work?
A vector database works by turning data into embeddings, organising those embeddings in a special index, and then searching that index for the vectors nearest to a query.
Here is the full flow, from raw data to search result:
- Prepare the data. Split long content into chunks, such as product descriptions or a few paragraphs of a PDF.
- Create embeddings. Run each chunk through an embedding model to get a vector. Some vector databases can call the model for you; others expect you to send ready-made vectors.
- Insert. Store each vector with an ID, the original text and metadata like category, date or language.
- Index. The database builds an index, a smart data structure that groups or links vectors so it does not have to compare every single one during a search.
- Query. A user's search text is embedded with the same model used for the data.
- Search. The database uses the index to find the vectors closest to the query vector, usually measured with cosine similarity or distance.
- Filter and return. It applies any metadata filters (for example, "only documents from 2026") and returns the top results with their text and scores.
Steps 1 to 4 happen when data is added or updated. Steps 5 to 7 happen on every search, often in a fraction of a second.
The index in step 4 is the clever part, and it deserves its own explanation.
How similarity search works: nearest neighbours, explained
Similarity search works by finding the nearest neighbours of a query vector, meaning the stored vectors that sit closest to it in the space of meaning.
Exact search: check everyone
The simplest method compares the query with every stored vector and keeps the closest ones. People call this exact or brute-force k-nearest-neighbour search, where k is the number of results you want. It always finds the true closest matches. It is also slow when you have millions of vectors, because the work grows with the size of the data.
Approximate search: check the likely neighbourhood
Most vector databases use approximate nearest neighbour (ANN) search instead. The idea is to skip most of the data and only check vectors in the most promising area. You give up a tiny bit of accuracy and gain a large jump in speed.
Two popular ANN approaches, at a high level:
- Graph-based (HNSW). Hierarchical Navigable Small World graphs link each vector to its neighbours in several layers. A search starts on a sparse top layer, like taking a national highway to the right state, then moves down to denser layers, like district roads and then local lanes, until it reaches the closest points. HNSW was described in a paper by Yury Malkov and Dmitry Yashunin and is widely used.
- Cluster-based (IVF). Vectors are grouped into clusters ahead of time, a bit like sorting addresses by PIN code. At search time, the database checks only the few clusters nearest to the query instead of the whole country.
Most databases let you tune this trade-off. Settings that check more of the neighbourhood give better recall (finding more of the true best matches) but take longer.
Vector database example: an online store's product search
A clear vector database example is an online clothing store that lets shoppers search by description and get relevant products even when the words do not match.
Imagine a small ethnic-wear brand in Jaipur with 20,000 products. Each product has a title, a description, a price and a category. The team does the following:
- Combines title and description for each product and creates an embedding.
- Stores each embedding in a vector database with metadata: price, category, size range and stock status.
- Embeds every shopper's search text with the same model at search time.
A shopper types: "light cotton kurta for summer office wear under 1000 rupees".
- The vector search finds products whose meaning is close: breathable cotton kurtas, short kurtis described as "ideal for hot weather", and simple straight-cut designs tagged as workwear.
- A metadata filter removes anything above 1,000 rupees and anything out of stock.
- The shopper sees relevant results, including items that never used the word "office" or "summer" in their titles.
A plain keyword search would have matched "cotton" and "kurta" and missed the intent behind "summer office wear". That gap, between the words people type and the meaning they intend, is exactly what vector databases close.
The same pattern shows up in job portals matching resumes to job descriptions, edtech apps suggesting the next lesson, and news apps grouping similar stories together.
Vector database in RAG: where it fits
In RAG, the vector database is the retrieval engine: it stores your document chunks as embeddings and returns the most relevant ones for each question.
A RAG system answers questions using your own documents. Its pipeline looks like this, and the vector database handles the store and retrieve steps:
- Chunk your documents into small passages.
- Embed each passage.
- Store the embeddings and passages in a vector database. (Vector database)
- Retrieve the passages closest to the user's question. (Vector database)
- Augment the prompt with those passages.
- Generate the answer with an LLM.
Take a college rulebook chatbot. A student asks, "What happens if I miss an internal exam due to illness?" The vector database finds the rulebook passages about medical absence and re-tests, even though the rulebook might say "make-up assessment" rather than "re-exam". The LLM then writes an answer from those passages.
Metadata filters matter a lot in RAG. If the rulebook has a 2025 and a 2026 version, filtering by year stops the bot quoting the old rule. If an HR bot serves several offices, filtering by location stops a Pune employee getting Hyderabad's holiday list.
If you are deciding between adding knowledge through RAG or retraining a model, our RAG vs fine-tuning guide covers that choice.
Vector database vs SQL database: what is the difference?
A SQL database finds rows that exactly match conditions you write, while a vector database finds items whose meaning is closest to a query.
| Traditional SQL database | Vector database | |
|---|---|---|
| Stores | Rows and columns (names, prices, dates) | Vectors (embeddings) plus content and metadata |
| Query style | Exact conditions, like price below 1000 and city equals Pune | Similarity, like "find items most like this text" |
| Result | Every row that matches, or none | The top k closest items, ranked by similarity score |
| Handles synonyms and paraphrases? | No, unless you add special search features | Yes, through embeddings |
| Best at | Transactions, accounts, inventory, reports | Semantic search, recommendations, RAG |
| Typical examples | MySQL, PostgreSQL, Oracle | Pinecone, Milvus, Qdrant, Weaviate, Chroma |
The two are partners, not rivals. A typical app keeps orders, users and payments in a SQL database, because you never want an "approximately correct" bank balance. It then uses vector search for the parts that need meaning, like search and recommendations.
The line is also blurring. PostgreSQL can do vector search through the open-source pgvector extension, and several general-purpose databases and search engines have added vector search features. Check current documentation for exact capabilities, since this area changes quickly.
Popular vector database names to know
The vector database names you will hear most often are Pinecone, Weaviate, Milvus, Qdrant, Chroma and pgvector for PostgreSQL.
| Name | What it is | Often chosen when |
|---|---|---|
| Pinecone | A fully managed cloud vector database service | You want a hosted service and do not want to run servers |
| Weaviate | An open-source vector database, also offered as a managed cloud | You want open source with a hosted option |
| Milvus | An open-source vector database designed for large-scale deployments | You expect very large collections and can run your own infrastructure |
| Qdrant | An open-source vector database written in Rust, with a cloud option | You want open source with filtering features and a hosted option |
| Chroma | An open-source embedding database popular with Python developers | You are prototyping or learning and want a quick local setup |
| pgvector | An open-source extension that adds vector search to PostgreSQL | You already use Postgres and want vectors next to your normal data |
You will also come across FAISS, a library from Meta's AI research team for fast similarity search. FAISS is a library you build into your own code, not a full database with storage, users and an API, but many tools use it under the hood.
I am deliberately not ranking these. Features, pricing and limits change often, and the right choice depends on your data size, budget, team skills and where your app runs. Read each project's current documentation and try two of them on your own data before deciding.
When do you not need a vector database?
You do not need a vector database when your data is small enough to search directly in memory, or when your problem is really about exact matches.
Here is my contrarian take: many beginner projects add a vector database because tutorials do, not because the project needs one. If you have a few hundred or a few thousand chunks, a simple list of embeddings in memory works fine. You compare the query vector with every stored vector, sort by similarity and take the top five. On a modern laptop that is fast enough for a student project or an internal tool used by a small team.
You can usually skip a dedicated vector database when:
- Your data is small. A single syllabus, one company handbook or a few hundred FAQs.
- Your data fits in the prompt. If the whole document fits comfortably in the model's context window, you may not need retrieval at all.
- You need exact lookups. Roll numbers, order IDs and PIN codes belong in a normal database or a keyword index.
- You already run Postgres. Adding pgvector can be simpler than running another system.
You probably do need one when you have hundreds of thousands of vectors or more, frequent updates, many users searching at once, or heavy metadata filtering.
How to start learning vector databases as a beginner
The best way to learn vector databases is to build one tiny semantic search project, first without a database and then with one.
A path that works well:
- Understand embeddings first. Make sure you are comfortable with what embeddings are and how cosine similarity works.
- Build it in memory. Embed 100 short texts, such as FAQs from your college website, store the vectors in a Python list and write your own "find top 5" function. This shows you exactly what a vector database automates.
- Move to a local vector database. Load the same data into a beginner-friendly option like Chroma, or into Postgres with pgvector, and compare the code.
- Add metadata filters. Tag each FAQ with a department and search within one department only.
- Plug it into an LLM. Pass the top results to a model and you have built a basic RAG system.
I think step 2 is the one most people skip, and it is the one that makes everything else click. Once you have written nearest-neighbour search by hand, every vector database feels like a faster version of something you already understand.
Vector databases come up regularly in interviews for AI engineering and data roles, usually alongside RAG and embeddings. Our AI interview prep section has practice questions on all three.
Key takeaways
- A vector database stores embeddings and finds the items most similar in meaning to a query.
- It works like a library shelved by topic: your question is placed on the shelf, and the nearest books come back.
- Most vector databases use approximate nearest neighbour search (like HNSW) to trade a little accuracy for a lot of speed.
- SQL databases handle exact matches and transactions; vector databases handle similarity, and many apps use both.
- Popular names include Pinecone, Weaviate, Milvus, Qdrant, Chroma and pgvector for PostgreSQL.
- For small datasets, a simple in-memory list of embeddings is often enough.