Deploying your own GPU infrastructure for inference of modern LLMs is an expensive endeavor. A100/H100 clusters require budgets starting from millions of rubles and a team of MLOps engineers. Together AI removes this barrier: we take over the integration of your services with the platform. We connect an OpenAI-compatible API, configure routing between models, optimize p99 latency — and you get a ready-made endpoint for your tasks. With over 5 years of experience, we have completed more than 30 LLM integration projects in production — we will evaluate your project within a day.
Case study: for a fintech startup, we deployed a RAG system on Together AI in 3 days. Previously, they paid for dedicated GPU instances on AWS — Together reduced costs by 5x, p99 latency did not exceed 200 ms, and answer accuracy reached 94%. This is not an isolated example: according to the platform documentation, its use can reduce inference costs by up to 80%. Compared to renting GPU instances on AWS, Together AI provides up to 80% savings with comparable performance.
What problems does Together AI solve?
High GPU cost. Renting dedicated A100/H100 instances in the cloud results in tens of thousands of dollars monthly. Together AI uses a shared GPU pool, reducing inference cost by 5–10x compared to cloud instances. Deployment complexity: selecting CUDA version, optimizing via vLLM or TensorRT-LLM — engineering hours. The platform handles all low-level optimizations. Lack of flexibility: balancing quality and speed — we implement a router that automatically switches the model depending on the context.
Basic integration
from openai import OpenAI, AsyncOpenAI # Together uses OpenAI SDK client = OpenAI( api_key="TOGETHER_API_KEY", base_url="https://api.together.xyz/v1", ) # Model selection MODELS = { "quality": "meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo", "balanced": "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo", "fast": "meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo", "code": "Qwen/Qwen2.5-Coder-32B-Instruct", "reasoning": "deepseek-ai/DeepSeek-R1-Distill-Llama-70B", } response = client.chat.completions.create( model=MODELS["balanced"], messages=[{"role": "user", "content": "Task"}], temperature=0.1, max_tokens=2048, ) print(response.choices[0].message.content) How does model routing work?
The router analyzes the request: if the task requires deep reasoning — it routes to DeepSeek-R1, if speed is needed — to Llama 3.1 8B. This reduces average token cost by 30–40% without quality loss. Under the hood, we use dynamic thresholds based on latency and confidence.
More about routing configuration
We set threshold values through A/B testing on a representative sample of queries. The router then automatically adapts to load, switching models based on current latency and required quality.
How does fine-tuning work?
Fine-tuning open models on your own data improves accuracy and reduces hallucinations. Together AI provides infrastructure for LoRA and full fine-tuning. We help prepare the dataset, select hyperparameters, and conduct A/B testing. Example launch:
# Together allows fine-tuning open models on your own data import together together.api_key = "TOGETHER_API_KEY" # Upload dataset (JSONL format: {"prompt": "...", "completion": "..."}) file_response = together.Files.upload(file="training_data.jsonl") file_id = file_response["id"] # Start fine-tuning ft_response = together.Finetune.create( training_file=file_id, model="meta-llama/Meta-Llama-3.1-8B-Instruct-Reference", n_epochs=3, batch_size=16, learning_rate=1e-5, suffix="my-custom-model", ) ft_job_id = ft_response["id"] # Check status status = together.Finetune.retrieve(ft_job_id) print(status["status"]) # "running" | "completed" | "failed" Advantages of fine-tuning on Together AI
The platform manages all compute resources — no need to think about GPUs. Pre-trained LoRA weights are available, which can be customized to your data. We help with hyperparameter selection and metric evaluation on a validation dataset. Fine-tuning pays off by increasing accuracy and reducing the number of expensive tokens.
Embeddings and RAG
For Retrieval-Augmented Generation, we use embedding models such as BAAI/bge-large-en-v1.5. Together AI supports asynchronous embedding generation, which is important under high load:
response = client.embeddings.create( model="BAAI/bge-large-en-v1.5", # One of the best for search input=["First text", "Second text"], ) embeddings = [item.embedding for item in response.data] Model comparison on Together AI
| Model | Quality | Speed (tokens/s) | Resource intensity |
|---|---|---|---|
| Llama 3.1 405B | Excellent | ~50 | Requires H100 |
| Llama 3.1 70B | Very Good | ~150 | Medium |
| Llama 3.1 8B | Good | ~400 | Low |
| Qwen2.5-Coder 32B | Code-specific | ~120 | Medium |
| DeepSeek-R1 70B | Reasoning | ~100 | Medium |
How to integrate Together AI in 3 steps?
- Get an API key from the Together AI console.
- Replace the base URL in your OpenAI client with
https://api.together.xyz/v1. - Configure routing — we select models for your scenarios. Get your endpoint in half a day.
Contact us for a consultation on Together AI integration.
How long does integration take?
Basic API integration takes half a day: replacing the base URL and key. A full project with fine-tuning and RAG takes from 5 working days. The cost is calculated individually based on traffic volume and model complexity. Request integration — get a ready endpoint in 1 day.
Process overview
| Stage | Duration |
|---|---|
| Scenario analysis | 1–2 days |
| Model selection and testing | 2–3 days |
| API integration and routing | 1–2 days |
| Fine-tuning and A/B test | 3–5 days |
| Deployment and optimization | 2–3 days |
| Team training | 1 day |
What's included
- Connect Together AI to your project (0.5 days)
- Fine-tuning pipeline with A/B testing (3–5 days)
- Embedding integration for RAG (1–2 days)
- Cost optimization (routing, caching, batch processing)
- Documentation and team training
- Post-launch support (monitoring, tuning)
Contact us — we will evaluate your project and propose the optimal solution. We guarantee a 2–5x reduction in inference costs compared to renting GPUs. Additionally, read about LLMs — the foundation for all modern AI solutions.