Build Your Knowledge Agent with AG2

Imagine an assistant that reads your product docs, returns cited answers when mentioned in chat, and streams context back to the UI in real time. That’s what the AG2 Knowledge Agent delivers.

What You’ll Build

An AG2 (AutoGen 2) agent that behaves like a documentation expert.
Triggered only when invoked (e.g., @knowledge).
Retrieves & cites snippets from an on-disk knowledge store.
Integrated into CometChat conversations with streaming responses.

Prerequisites

Python 3.10+ and pip.
OPENAI_API_KEY in a .env file (used for embeddings + chat completions).
Optional overrides:
- KNOWLEDGE_AGENT_MODEL (default gpt-4o-mini)
- KNOWLEDGE_AGENT_EMBEDDING_MODEL (default text-embedding-3-small)
- KNOWLEDGE_AGENT_TOP_K, KNOWLEDGE_AGENT_TEMPERATURE, etc.
CometChat app credentials (App ID, Region, Auth Key).

Quick links

Repo (examples): ai-agent-ag2-examples/ag2-knowledge-agent
Agent source: agent.py
FastAPI server: server.py
Sample web embed: web/index.html

How it works

This project packages a retrieval-augmented AG2 agent that:

Ingests sources into namespaces using the KnowledgeStore. POST payloads to /tools/ingest with URLs, local files, or raw text. Files (.pdf, .md, .txt) are parsed and chunked before embeddings are stored under knowledge/<namespace>/index.json.
Searches namespaces via /tools/search, ranking chunks with cosine similarity on OpenAI embeddings (defaults to top 4 results).
Streams answers from /agent. The server wraps AG2’s ConversableAgent, emits SSE events for tool calls (tool_call_start, tool_call_result) and text chunks, and always terminates with [DONE].
Cites sources. The agent compiles the retrieved snippets, instructs the LLM to cite them with bracketed numbers, and appends a Sources: line.
Hardens operations: namespaces are sanitized, ingestion is thread-safe, and errors are surfaced as SSE error events so the client can render fallbacks.

Key modules:

agent.py — KnowledgeAgent, store helpers, embedding + retrieval logic.
server.py — FastAPI app hosting /agent, /tools/ingest, /tools/search, /health.
knowledge/ — default storage root (persisted chunks + index).
web/index.html — CometChat Chat Embed sample that points to your deployed agent.

Setup

Clone & install

git clone https://github.com/cometchat/ai-agent-ag2-examples.git then cd ai-agent-ag2-examples/ag2-knowledge-agent. Create a virtualenv and run pip install -r requirements.txt.

Configure environment

Create a .env file, set OPENAI_API_KEY, and override other knobs as needed (model, temperature, namespace).

Prime knowledge

Run python server.py once (or uvicorn server:app) to initialize folders. Use the ingest endpoint or the provided knowledge samples as a starting point.

Run locally

Start FastAPI: uvicorn server:app —reload —host 0.0.0.0 —port 8000. Verify GET /health returns {"status": "healthy"}.

Create a tunnel

Expose /agent publicly with ngrok, Cloudflare Tunnel, or Render so CometChat can reach it.

Project structure

Dependencies: requirements.txt
Agent logic: agent.py
Knowledge store: knowledge/
API server: server.py
Web sample: web/index.html

Core configuration

Environment variables

OPENAI_API_KEY=sk-your-key
KNOWLEDGE_AGENT_MODEL=gpt-4o
KNOWLEDGE_AGENT_TEMPERATURE=0.2
KNOWLEDGE_AGENT_TOP_K=4
KNOWLEDGE_AGENT_EMBEDDING_MODEL=text-embedding-3-small
KNOWLEDGE_AGENT_DEFAULT_NAMESPACE=docs
KNOWLEDGE_AGENT_STORAGE_PATH=knowledge

Knowledge ingestion schema

POST /tools/ingest
{
  "namespace": "docs",
  "sources": [
    {
      "type": "url",
      "value": "https://example.com/handbook",
      "title": "Handbook"
    },
    {
      "type": "file",
      "path": "/absolute/path/to/policies.pdf"
    },
    {
      "type": "text",
      "title": "Inline FAQ",
      "value": "Q: How do refunds work?\nA: ..."
    }
  ]
}

Streaming response shape

data: {"type":"tool_call_start","tool_call_id":"call_123","tool_name":"search_docs","args":{"namespace":"docs","query":"@agent refund policy"}}

data: {"type":"tool_call_result","tool_call_id":"call_123","result":{"matches":[{"chunk_id":"...","score":0.82,"source":"knowledge/docs/...","title":"Refund Policy","preview":"..."}]},"is_frontend_tool":false}

data: {"type":"text_message","delta":"Refunds are processed within 5 business days "}
data: {"type":"text_message","delta":"once the request is approved.[1] Sources: [1] Refund Policy"}
data: [DONE]

Step 1 - Ingest documentation

Pick a namespace

Stick with docs or choose a sanitized string (alphanumeric, -, _).

POST to /tools/ingest

Supply a mix of URLs, file paths, and inline text. The server fetches, parses (PDF/Markdown/Text), chunks, embeds, and persists everything.

Inspect storage

Confirm knowledge/<namespace> contains generated .md copies and an updated index.json.

Step 2 - Ask the knowledge agent

Make a request

curl -N -X POST http://localhost:8000/agent \
  -H "Content-Type: application/json" \
  -d '{
    "thread_id": "demo-thread",
    "run_id": "demo-run",
    "messages": [
      { "role": "user", "content": "@knowledge What does our refund policy cover?" }
    ],
    "tool_params": { "namespace": "docs" }
  }'

Handle SSE

Streamed events include tool telemetry and text deltas. Buffer them until you receive [DONE].

Observe citations

Replies contain bracketed numbers matching the matches payload returned by the retrieval tool.

Step 3 - Deploy the API

Host the FastAPI app (Render, Fly.io, ECS, etc.) or wrap it in a serverless function.
Expose /agent, /tools/ingest, and /tools/search over HTTPS.
Keep KNOWLEDGE_AGENT_STORAGE_PATH on a persistent volume so ingested docs survive restarts.
Harden CORS, rate limiting, and authentication before opening the endpoints to the public internet.

Step 4 - Configure CometChat

Open Dashboard

Go to app.cometchat.com.

Add agent

Navigation: AI Agents → Add Agent.

Provider & IDs

Set Provider=AG2 (AutoGen), Agent ID=knowledge, Deployment URL=public /agent endpoint.

Optional metadata

Add greeting, intro, or suggested prompts such as “@knowledge Summarize the onboarding guide.”

Enable

Save then confirm the toggle shows Enabled.

For more on CometChat AI Agents, see: Overview · Instructions · Custom agents

Step 5 - Customize in UI Kit Builder

Open variant

From the Dashboard variant card, click Customize and Deploy.

Update UX

Themes, layouts, attachment rules, and action bindings—all reflect in exports.

Preview

Run conversations and confirm the knowledge agent responds only when invoked.

Step 6 - Integrate

Once configured, embed the agent in your product using the exported configuration.

Widget Builder

React UI Kit

Pre Built UI Components

The AG2 Knowledge agent you connected is bundled automatically. End-users will see it immediately after deployment.

Step 7 - Test end-to-end

API generates response

POST to /agent returns cited answers.

Search utility works

POST to /tools/search returns ranked snippets.

Dashboard check

In CometChat, ensure the agent appears as a selectable bot.

Widget/UI verification

Preview in UI Kit Builder or your app and confirm streaming + citations behave as expected.

Security & production checklist

Protect ingestion routes with auth (API key/JWT) and restrict who can write to each namespace.
Validate URLs and file paths before fetching to avoid SSRF or untrusted uploads.
Rate limit the /agent and /tools/* endpoints to prevent abuse.
Keep OpenAI keys server-side; never ship them in browser code or UI Kit Builder exports.
Log tool calls (namespace, duration) for debugging and monitoring.

Troubleshooting

Agent replies without citations: confirm the retrieval step returns matches; check embeddings or namespace.
Empty or slow results: ensure documents were ingested successfully and the top-k limit isn’t set to 0.
503 Agent not initialized: verify server logs—KnowledgeAgent requires OPENAI_API_KEY during startup.
SSE errors in widget: make sure your reverse proxy supports streaming and disables response buffering.

Next steps

Add new tools (e.g., summarize, link_to_source) by extending the AG2 agent before responding.
Swap in enterprise vector stores by replacing KnowledgeStore with Pinecone, Qdrant, or Postgres.
Schedule re-ingestion jobs when docs change, or watch folders for automatic refreshes.
Layer analytics: log tool_call_result.matches for insight into content coverage.

Guides

Tutorials

Build Your Knowledge Agent with AG2

What You’ll Build

Prerequisites

Quick links

How it works

Setup

Project structure

Core configuration

Step 1 - Ingest documentation

Step 2 - Ask the knowledge agent

Step 3 - Deploy the API

Step 4 - Configure CometChat

Step 5 - Customize in UI Kit Builder

Step 6 - Integrate

Widget Builder

React UI Kit

Step 7 - Test end-to-end

Security & production checklist

Troubleshooting

Next steps

Guides

Tutorials

​What You’ll Build

​Prerequisites

​Quick links

​How it works

​Setup

​Project structure

​Core configuration

​Step 1 - Ingest documentation

​Step 2 - Ask the knowledge agent

​Step 3 - Deploy the API

​Step 4 - Configure CometChat

​Step 5 - Customize in UI Kit Builder

​Step 6 - Integrate

Widget Builder

React UI Kit

​Step 7 - Test end-to-end

​Security & production checklist

​Troubleshooting

​Next steps

What You’ll Build

Prerequisites

Quick links

How it works

Setup

Project structure

Core configuration

Step 1 - Ingest documentation

Step 2 - Ask the knowledge agent

Step 3 - Deploy the API

Step 4 - Configure CometChat

Step 5 - Customize in UI Kit Builder

Step 6 - Integrate

Step 7 - Test end-to-end

Security & production checklist

Troubleshooting

Next steps