Segmenting AI Workflow Server

Illustration of a technique to split AI workflows into descrete steps

Description

THIS IS A WORK IN PROGRESS.

This project is a companion to an article that I haven't finished writing yet. The idea behind it is that, just as an enterprise application can be architectured as a collection of agents communicating over A2A, an individual agent can be architectured as a collection of discrete AI tools communicating over MCP.

For example, imagine an agent that is responsible for processing scanned documents - it takes the document image, extracts the text, and, say, pulls keywords from the text to use for indexing. Then the image, text and keywords are passed to another agent for storage. In this example it would be simple to use a multimodal model to process the image, extract the text and pull the keywords all in one session, and in many cases it would make sense to do it that way.

What this example does is splits that workflow into three different LLM sessions, each using a different model - an agentic model (qwen3) for the main process, a model tuned for OCR (glm-ocr) to handle image-to-text, and a small model (phi3:mini) for keyword extraction. The OCR and keyword extraction models are implimented as tools on an MCP server.

There are a number of benefits to splitting an application this way:

• Testing each segment becomes easier - since most segments inputs and outputs are narrowly defined, automating testing is more straight-forward.
• Tuning each segment becomes easier - you can tweak prompts within a segment without worrying about the impact on other parts of the workflow and since each is it's own session, you can set temperature and context length per segment.
• Load balancing becomes a per-segment issue instead of a per-agent issue.
• You can reduce token usage (and cost - tokens are never going to get any cheaper) by moving parts of your AI workflow to cheaper or in-house LLMs.
• You have better control of information security - by handling some information with in-house tools, you can make sure not to expose anything sensitive to third-party LLMs

This is not an architecture you'd use on every project, just another option when designing an application.

Installation

License

MIT License

Credits