Integrating ChatDev for Multi-Agent Software Development

How ChatDev accelerates prototyping?

Imagine compressing a sprint into one day: a product manager describes an idea, and a developer spends half a day cutting a prototype. A week later the hypothesis is validated, but the prototype goes into the bin — 80% of the time went into routine. ChatDev is a framework from Tsinghua University where AI agents with roles of CEO, CTO, Programmer, Reviewer, and Tester mimic a waterfall process, generating working code from a natural language description. We use it in our project workflow to obtain a PoC in 30–90 minutes instead of 1–2 days.

How it works?

ChatDev builds a chain of dialogues: CEO sets the task, CTO chooses architecture, Programmer writes code, Reviewer and Tester find errors. Each phase is a separate dialogue with clear input/output. It is not a chatbot but a managed process where the result of one phase feeds into the next.

Why chat agents are more effective than a single LLM prompt?

A single run through GPT-4 with a prompt "create a scraper" often produces non-working code — mixes libraries, misses edge cases. ChatDev splits the task: CTO specifies architecture, Programmer writes to spec, Reviewer checks compliance. Each agent sees only its part of the context, reducing the load on the model's attention window. Practice shows: the percentage of immediately deployable code is ~15% for simple utilities, but time to first working commit decreases by 60–70%.

Examples of ChatDev integration

Basic launch and Python API

# Installation
git clone https://github.com/OpenBMB/ChatDev.git
cd ChatDev
pip install -r requirements.txt

# Launch development
python run.py \
    --task "Develop a web scraper to extract prices from e-commerce sites. Python + BeautifulSoup + requests. Save to CSV." \
    --name "PriceScraperApp" \
    --model GPT_4_TURBO

from chatdev.chat_chain import ChatChain
import os

os.environ["OPENAI_API_KEY"] = "sk-..."

chat_chain = ChatChain(
    config_path="CompanyConfig/Default",
    config_phase_path="PhaseConfig/Default",
    config_role_path="RoleConfig/Default",
    task_prompt="Create a utility for converting data formats (CSV, JSON, XML, YAML)",
    project_name="DataConverter",
    org_name="TechTeam",
    model_type="GPT_4_TURBO",
)

chat_chain.pre_processing()
chat_chain.make_recruitment()
chat_chain.execute_chain()
chat_chain.post_processing()

Custom phases

{ "chain": [
    {"phase": "DemandAnalysis", "phaseType": "SimplePhase"},
    {"phase": "LanguageChoose", "phaseType": "SimplePhase"},
    {"phase": "Coding", "phaseType": "SimplePhase"},
    {"phase": "SecurityAudit", "phaseType": "SimplePhase"},
    {"phase": "CodeCompleteAll", "phaseType": "ComposedPhase", "cycleNum": 3, "Composition": [
      {"phase": "CodeReviewComment", "phaseType": "SimplePhase"},
      {"phase": "CodeReviewModification", "phaseType": "SimplePhase"},
      {"phase": "TestErrorSummary", "phaseType": "SimplePhase"},
      {"phase": "TestModification", "phaseType": "SimplePhase"}
    ]},
    {"phase": "Manual", "phaseType": "SimplePhase"}
  ]
}

from chatdev.experience_pool import ExperiencePool
experience_pool = ExperiencePool.load("./experience_pool")
chat_chain = ChatChain(
    task_prompt="...",
    experience_pool=experience_pool,
    use_experience=True,
)
experience_pool.update(chat_chain.get_experience())
experience_pool.save("./experience_pool")

Practical results and limitations

Case: rapid prototyping

Problem: a client team generated 2–3 prototypes per week for hypothesis validation. Each prototype required 1–2 developer days. We introduced ChatDev as a draft generator.

Usage: prototypes of simple utilities (converters, scrapers, report generators), CLI tools for internal use, basic CRUD APIs for PoCs.

Results from practice:

• Prototype creation time: 1–2 days → 30–90 minutes
• Review and refinement: 2–4 hours (vs. full write from scratch)
• Production readiness without refinement: ~15% (only the simplest utilities)
• Main value: fast concept validation — the team tested 12 hypotheses in a quarter instead of 6
• Time savings: ~15 person-days per month

Limitations: maximum efficiency on projects up to 500 lines of code. Complex multi-file projects with dependencies require significant post-processing. No native support for existing codebases.

Comparison with MetaGPT

What the integration includes

When ordering ChatDev integration, you get: agent configuration for your typical tasks, a set of custom phases (SecurityAudit, PerformanceCheck, StyleLint), CI/CD integration (trigger generation from a button or from Jira), a basic ExperiencePool assembly with patterns from your projects, documentation (architecture, launch examples, troubleshooting) and a team training session (4 hours, remote).

Working process:

1. Analytics: we analyse typical team tasks, determine the suitable volume for ChatDev (up to 500 lines).
2. Design: configure roles, phases, ExperiencePool for your tech stack (Python, Go, TypeScript — any).
3. Implementation: write configs, integrate API, add CI triggers.
4. Testing: run through 10–15 real tickets, record generation success rate.
5. Deployment: deploy on a dev stand or in the cloud, hand over refined templates.

Timelines: basic launch and setup — from 1 day; custom phases aligned with team processes — 3–5 days; full integration with CI and training — from 1 week. Cost is calculated individually — contact us to discuss details. Get a consultation: we will send a sample config for your project and estimate timelines.

Typical mistakes and ChatDev's role

• Overly long task descriptions (>200 words): agents lose focus — split the task into subtasks.
• Using for legacy refactoring: ChatDev does not see the code, generates from scratch. An agent-analyser is needed.
• Skipping the SecurityAudit phase: generated code may contain vulnerabilities (SQL injections, hardcoded tokens).

ChatDev will not replace developers but will accelerate them: it takes over routine (writing boilerplate, tests, documentation). 85% of results require refinement, but that refinement takes hours, not days. If you generate 20 prototypes per month, ChatDev saves ~15 person-days. That is enough to clear the backlog or launch a parallel experiment. Contact us — we will show how to embed ChatDev into your pipeline.