Vector Search Without the Cloud: FAISS as a Solution for RAG
Cloud vector databases are expensive, slow, and insecure for internal documents. Our team regularly encounters requests for local RAG where every millisecond of latency matters. FAISS by Meta is not a database but a high-performance vector search engine that operates in memory or on disk without network interaction. We use it for embedding into applications, offline scenarios, and situations where an external service is unacceptable. Five years of experience in NLP and 15+ implemented RAG projects allow us to guarantee pipeline stability. According to tests, local FAISS on GPU saves up to 70% compared to cloud services like Pinecone or Weaviate. Switching to local FAISS saves significant amounts monthly for 10 million vectors. Get a consultation on implementing FAISS in your project.
Problems Solved by FAISS
High embedding cost and latency. With batch requests to the OpenAI API, delays grow linearly. FAISS on a local GPU gives p99 latency <5 ms for 100K vectors — 50 times faster than cloud solutions. Data confidentiality. Financial reports, medical records, trade secrets — we do not send embeddings to external services. FAISS stores everything locally. Offline mode. Field devices, closed loops — FAISS works without internet. Order an audit of your current pipeline to identify bottlenecks.
Why Is FAISS Faster Than Traditional Databases?
FAISS vector search does not use SQL, B-trees, or metadata filtering — instead, it applies approximate search algorithms: IVF (Inverted File), HNSW (Hierarchical Navigable Small World), and Product Quantization. Compare:
| Aspect | FAISS (HNSW) | PostgreSQL + pgvector | Pinecone (cloud) |
|---|---|---|---|
| Latency p99 (100K vectors) | 1–5 ms | 10–50 ms | 20–100 ms |
| Throughput (batch 100) | >10K QPS | ~1K QPS | ~500 QPS |
| Dependencies | No network | Network to DB | Internet required |
| Cost (10M vectors/month) | Only hardware | ~$50–200 | $700–2000+ |
How We Build RAG with FAISS: Stack and Case Study
Typical stack: Python 3.10+, FAISS 1.7+, OpenAI text-embedding-3-small (1536 dim), GPT-4o-mini for generation. For one client with a corpus of 50,000 technical articles, we chose IndexHNSWFlat (M=16, efConstruction=200). This gave 98% recall and 2 ms latency per query.
Example FAISS indexing code
import faiss import numpy as np import pickle from openai import OpenAI openai_client = OpenAI() def build_faiss_index(texts: list[str], dimension: int = 1536) -> tuple: """Creates a FAISS index and corresponding list of texts""" # Получаем embeddings батчами embeddings = [] batch_size = 100 for i in range(0, len(texts), batch_size): batch = texts[i:i + batch_size] response = openai_client.embeddings.create( model="text-embedding-3-small", input=batch, ) batch_embeddings = [e.embedding for e in response.data] embeddings.extend(batch_embeddings) # Конвертируем в numpy float32 vectors = np.array(embeddings, dtype=np.float32) # Нормализуем для cosine similarity (через inner product) faiss.normalize_L2(vectors) # Создаём HNSW индекс index = faiss.IndexHNSWFlat(dimension, 16) # M=16 index.hnsw.efConstruction = 200 index.add(vectors) return index, texts # Сохранение на диск def save_index(index, texts, path_prefix: str): faiss.write_index(index, f"{path_prefix}.index") with open(f"{path_prefix}_texts.pkl", "wb") as f: pickle.dump(texts, f) # Загрузка def load_index(path_prefix: str) -> tuple: index = faiss.read_index(f"{path_prefix}.index") with open(f"{path_prefix}_texts.pkl", "rb") as f: texts = pickle.load(f) return index, texts Search and RAG answer:
def faiss_rag_answer( question: str, index: faiss.Index, texts: list[str], top_k: int = 5 ) -> str: # Embedding вопроса query_embedding = openai_client.embeddings.create( model="text-embedding-3-small", input=question, ).data[0].embedding query_vector = np.array([query_embedding], dtype=np.float32) faiss.normalize_L2(query_vector) # Поиск distances, indices = index.search(query_vector, top_k) # Извлечение текстов context_texts = [texts[i] for i in indices[0] if i >= 0] context = "\n\n---\n\n".join(context_texts) # Генерация ответа response = openai_client.chat.completions.create( model="gpt-4o-mini", messages=[ {"role": "system", "content": "Отвечай строго на основе предоставленного контекста."}, {"role": "user", "content": f"Контекст:\n{context}\n\nВопрос: {question}"} ], temperature=0, ) return response.choices[0].message.content How to Accelerate FAISS on GPU?
Transferring the index to GPU gives up to 100× performance boost. We use faiss.StandardGpuResources:
res = faiss.StandardGpuResources() gpu_index = faiss.index_cpu_to_gpu(res, 0, index) # GPU 0 distances, indices = gpu_index.search(query_vectors, top_k) RAG Pipeline Development Process
- Analysis: Examine the corpus, requirements for latency, accuracy (recall), update volume. Choose the index type.
- Design: Define the pipeline — embedder (OpenAI / local), index, LLM (GPT-4o-mini / Claude). Work out the context format.
- Implementation: Deploy indexing and search code, integrate with existing application (API / embedded).
- Testing: Measure latency, recall, answer quality. Optimize parameters (M, efSearch, batch size).
- Deployment: Containerization (Docker), CI/CD, monitoring metrics (p99 latency, QPS, GPU utilization).
What's Included in the Work?
- Pipeline architecture (documentation + diagram)
- Indexing and search code (Python, versioned Git)
- Index configuration tailored to your corpus (recommendations for HNSW/IVF)
- Integration with LLM (OpenAI, local models via vLLM)
- Load testing (report with metrics)
- Post-deployment support (2 weeks warranty maintenance)
Typical Mistakes When Implementing FAISS
- Normalization not performed. Without L2 normalization, inner product is not equivalent to cosine similarity — results degrade by 10–20%.
- Choosing the wrong index. IndexFlatL2 on 10M vectors consumes >60 GB RAM — use IVFPQ.
- efSearch not tuned. For HNSW, efSearch < 100 reduces recall to 80% — raise to 200–400.
- Ignoring GPU memory. For large indexes, the index does not fit on GPU — switch to IVF with quantization.
Comparison of FAISS Index Types
| Index Type | Recall | Speed (latency) | Memory | Recommended Corpus Size |
|---|---|---|---|---|
| IndexFlatL2 | 100% | ~50 ms for 100K | ~600 MB (1536 dim) | Up to 100K |
| IndexIVFFlat | ~95% (100 centroids) | ~5 ms | ~600 MB + centroids | 100K – 10M |
| IndexHNSWFlat | ~98% (efSearch=200) | ~2 ms | ~1.2 GB (with graphs) | 100K – 10M |
| IndexIVFPQ | ~90% (8x quantization) | ~1 ms | ~75 MB (compression 8x) | >10M |
Timeline and How to Order
Estimated timeline: for a corpus up to 500K vectors — 1–2 weeks turnkey. For larger volumes (>10M) — 3–4 weeks with index optimization. Contact us for an assessment of your scenario — we will select a configuration and calculate the cost individually. Get a consultation on implementing FAISS in your project. Order an audit of your current pipeline to identify bottlenecks.