In this post, we examine the components of Agent Maker, what it takes to help AI practitioners to build enterprise ready agents.

The LLM-Harness Divide

Initially called agent scaffolding, industry has converged on agent harness as the name for everything else of agent except LLM. It makes sense:

  • Consider the analogy of dog sled:
    • Dogs are just dogs without the harness.
    • With the harness (sled included), dogs become transportation.
    • With other harness (such as those in a circus), dogs may become entertainers.

With the LLM/Harness divide comes Harness Engineering:

Anatomy of an Agent

From an agentic app all the way down to a tool, an agent typically consists of a hierarchically organized bill of materials (BOM). Looking at a high level, between each pairs of component levels there exhibits a many-to-many relationship. One aspect of agent governance is to manage the component configuration of an agent. We call this system the Agent Configuration Manager.

Above diagram leaves out the most important component of an agent: LLM, which deserves it’s own treatment:

  • The many-to-many relationship between agent and LLMs needs a purposely built router
    • To achieve maximum tokenomics
    • To go beyond simple rule-based routing policy

Agent Development Life Cycle (ADLC)

More power comes with more scrutiny. In addition to the typical development, packaging and deployment lifecycle for normal apps, the non-deterministic nature of agents calls for additional authorizations at both configuration and runtime:

  • At Configuration time
    • Usage of critical skills/tools must be controlled with a subscription/approval process.
  • At Runtime
    • Inbound access to the agent and outbound access to resources must be controlled by the identities of users and agent, or the combination of the two.

Registry vs Repo

To satisfy the complex configuration needs of agents, it’s beneficial to differentiate between registry and repository:

  • Registry contains meta-data and access control lists (ACLs) of artifacts
    • Registry data can be used to generate statistics for reporting purpose
  • Repo is the storage layer where the artifacts physically live
    • Artifactory is a repo for binary artifacts
    • GitHub is a repo for (largely) textual artifacts

Below diagram shows a big picture of registry, repo, artifacts, meta-data and the tools in between:

SIS and SPS

Mimicking the recursive naming convention of PIP (Pip Installs Packages):

  • Sis Installs Skills (SIS) is an agent skills installer:
    • It behaves exactly as PIP
    • Whatever PIP does for Python packages, SIS does for agent skills.
    • In addition, when used in a container packaging step, it checks agent configuration authorization before installing a skill
  • Sps Publishes Skills (SPS) is an agent skills publisher
  • It run the to-be-published skills through various quality/security gates
  • There should be a dev section in the skills repo where,
  • Local development of agents and skills have the complete freedom to install/publish skills for experimentation purpose

Frameworks and Runtimes

Reputable agent development frameworks have emerged:

And K8s native as well as proprietary agent runtimes are standardizing:

References

  1. Google’s Agent Substrate & Sandbox, as well as their relationship to Agent Runtime.
  2. Learn about competing technologies that power agent runtime: AWS MicroVM | Google gVisor
  3. OpenAI Codex: Codex behind-the-scenes | How it works | How it’s built
Agent Maker