Haystack Integration for NLP Pipelines

We often encounter a situation: a company has already collected a document corpus, but search works via grep or plain BM25. Results are irrelevant, answers to customer queries must be found manually. Or the team tried LangChain, but the prototype turned out too fragile for production. Haystack (deepset) solves both problems: a production-ready framework with a declarative pipeline model, where components are connected into a graph with typed data. This simplifies testing, versioning, and replacing components. Our experience includes over 5 years in NLP and 20+ implemented RAG systems. Order an audit of your document corpus – we will select the optimal architecture. Saving up to 40% of time on information retrieval is a real result of implementation.

Why Haystack Over LangChain for RAG?

Haystack wins in scenarios where stability and testability are needed. For document-centric tasks – when the main work involves searching and processing a document corpus. For production-grade RAG – a reliable system is required, not a prototype. Haystack for RAG is 3–5 times more reliable than LangChain on large document volumes. Our team prefers explicit configuration: YAML pipelines are easier to audit than LangChain's Python code. Haystack also has built-in components for multi-hop question answering. We use Haystack for projects where stability is important, and leave LangChain for rapid prototyping and agent scenarios.

How to Build a RAG Pipeline on Haystack?

In Haystack 2.x, the architecture became stricter: typed @component.input and @component.output, unified Document object, DocumentStore abstraction. Here is a minimal example:

from haystack import Pipeline, Document from haystack.components.retrievers import InMemoryBM25Retriever from haystack.components.generators import OpenAIGenerator from haystack.components.builders import RAGPromptBuilder pipeline = Pipeline() pipeline.add_component("retriever", InMemoryBM25Retriever(document_store=store)) pipeline.add_component("prompt_builder", RAGPromptBuilder()) pipeline.add_component("generator", OpenAIGenerator(model="gpt-4o-mini")) pipeline.connect("retriever.documents", "prompt_builder.documents") pipeline.connect("prompt_builder.prompt", "generator.prompt") 

Which DocumentStore to Choose?

The choice depends on scale and infrastructure. For fast development – InMemoryDocumentStore (up to 10K documents). For production – Elasticsearch (BM25 + semantic) or Qdrant (high performance, >1M vectors). If you already use PostgreSQL – pgvector. Qdrant configuration example:

from haystack_integrations.document_stores.qdrant import QdrantDocumentStore document_store = QdrantDocumentStore( url="http://localhost:6333", index="documents", embedding_dim=1536, recreate_index=False, ) 

Document Indexing: Step-by-Step Recipe

Indexing pipeline is a separate stage. We use the following components:

1. Conversion: PyPDFToDocument for PDF, TextFileToDocument for TXT.
2. Cleaning: DocumentCleaner removes garbage.
3. Splitting: DocumentSplitter splits into sentences (split_length=5, split_overlap=2).
4. Embedding: OpenAIDocumentEmbedder with model text-embedding-3-small.
5. Writing: DocumentWriter saves to DocumentStore.

from haystack.components.converters import PyPDFToDocument from haystack.components.preprocessors import DocumentCleaner, DocumentSplitter from haystack.components.embedders import OpenAIDocumentEmbedder from haystack.components.writers import DocumentWriter indexing = Pipeline() indexing.add_component("converter", PyPDFToDocument()) indexing.add_component("cleaner", DocumentCleaner()) indexing.add_component("splitter", DocumentSplitter( split_by="sentence", split_length=5, split_overlap=2 )) indexing.add_component("embedder", OpenAIDocumentEmbedder( model="text-embedding-3-small" )) indexing.add_component("writer", DocumentWriter(document_store=document_store)) 

Hybrid Search: Combining BM25 and Semantics

Haystack supports hybrid search via DocumentJoiner with reciprocal_rank_fusion (RRF) mode. This yields 30–40% better relevance than each method alone. Saves time on manual result filtering. Example:

from haystack.components.retrievers import InMemoryBM25Retriever, InMemoryEmbeddingRetriever from haystack.components.joiners import DocumentJoiner pipeline.add_component("bm25", InMemoryBM25Retriever(document_store=store, top_k=10)) pipeline.add_component("semantic", InMemoryEmbeddingRetriever(document_store=store, top_k=10)) pipeline.add_component("joiner", DocumentJoiner(join_mode="reciprocal_rank_fusion")) 

How to Speed Up a RAG Pipeline?

Performance is critical. We use:

• async mode via pipeline.run_async() for concurrent request processing;
• batching for embedder components – up to 10x speedup during indexing;
• CachingChecker + Redis to cache search results;
• Prometheus metrics via Hayhooks middleware. Typical RAG pipeline latency with gpt-4o-mini and Qdrant is 1–3 seconds per request.

Serialization and Deployment: Step-by-Step Process

1. Serialize the pipeline to YAML. Haystack supports export via pipeline.dump().
2. Save YAML to Git – enables code review of configuration.
3. Set up CI/CD: on push to main, run tests (evaluation metrics) and deploy via Hayhooks.
4. Haystack Hayhooks provides REST API for serving pipelines, including Prometheus metrics.

version: "2.0" components: - name: retriever type: InMemoryBM25Retriever params: document_store: store - name: prompt_builder type: RAGPromptBuilder - name: generator type: OpenAIGenerator params: model: gpt-4o-mini connections: - retriever.documents -> prompt_builder.documents - prompt_builder.prompt -> generator.prompt