Microsoft is taking significant strides in its AI application development strategy with the advancement of Semantic Kernel, an open-source set of tools for managing and orchestrating AI prompts. Since its launch as a way to simplify building retrieval-augmented generation (RAG) applications, Semantic Kernel has grown into a framework for building and managing agentic AI. At Ignite 2024, Microsoft announced several new features for Semantic Kernel, positioning it as its preferred tool for building large-scale agentic AI applications.

One of the key new features in Semantic Kernel is the Agent Framework, which will soon move out of preview into general availability. This will ensure a stable, supported set of tools ready to deliver production-grade enterprise AI applications. The Agent Framework is designed to help build applications around agent-like patterns, offering a way to add autonomy to applications and deliver what Microsoft calls "goal-oriented applications." This framework enables the construction of workflows based on user requests, allowing multiple agents to collaborate, share data, and manage long transactions that work across many different application APIs and endpoints.

The Agent Framework is available as an extension to the base Semantic Kernel and is delivered as a set of .NET libraries. These libraries help manage human/agent interactions and provide access to OpenAI's Assistant API. The framework is intended to be controlled via conversation, although it's easy enough to build and run agents that respond to system events rather than direct human actions, and to add human approval steps as part of a dynamic workflow. This lets developers focus on using agents to manage tasks.

Semantic Kernel's agent features are designed to extend the concepts and tools used to build RAG-powered AI workflows. As always, Semantic Kernel manages the overall orchestration and individual agents, handling context and state as well as calls to AI endpoints via Azure AI Foundry and similar services. Building a Semantic Kernel agent requires an Agent class before using an Agent Chat to support interactions between the agent workflow and the AI and API endpoints used to complete the current task. If multiple agents need to be called, developers can use an Agent Group Chat to manage these internal prompts by using Semantic Kernel to interact and pass results between each other.

In addition to Semantic Kernel, Microsoft is also integrating AutoGen, a no-code agent development tool, into its AI application development strategy. AutoGen builds on Semantic Kernel's Process Framework, which is designed to manage long-running business processes and works with distributed application frameworks such as Dapr and Orleans. Workflows are event-driven, with steps built around Semantic Kernel Functions. A process isn't an agent, as it's a defined workflow, and there is no self-orchestration. However, a step can contain an agent if it has well-defined inputs and outputs.

Using AutoGen as part of its agent tooling will help deliver better support for multi-agent operations in Semantic Kernel. AutoGen simplifies the process of building agents, with a no-code GUI and support for a variety of different large language models (LLMs) such as OpenAI, Azure OpenAI, Ollama, and Gemini. Getting started with AutoGen requires the core AutoGen application and a model client. Once installed, developers can build a simple agent with a handful of lines of code. Things get interesting when building a multi-agent application or, as AutoGen calls it, a team.

AutoGen Studio is perhaps the most interesting part of the integration, providing a place to construct teams of agents and extensions without needing to write any additional code. It installs as a local web application and allows developers to drag components onto the Studio canvas, adding termination conditions to construct a multi-agent application. Agents can be further configured by adding models and extensions, for example, using an extension to deliver a RAG query against enterprise data. Multiple model support helps developers choose a suitable AI model for an agent, perhaps one that's been fine-tuned or that offers multi-model actions so they can work with images and audio as well as text prompts.

Under the hood, AutoGen is a declarative agent development environment, with a JSON description of the various elements that go into making an agent. Developers can switch to a JSON view to make changes or even convert AutoGen AgentChat Python to JSON and edit it in Studio. To simplify building new applications, AutoGen offers a gallery where agents and other components can be shared with other users. Once developers have built an agent, they can evaluate it inside Studio's playground before building it into a larger process.

The integration of Semantic Kernel and AutoGen marks a significant step forward in Microsoft's AI application development strategy. As the company moves beyond the experimental phase of enterprise AI, it's clear that the focus is shifting towards building workflow tools more suited to the multi-channel, multi-event processes that form the backbone of businesses. With Semantic Kernel and AutoGen, Microsoft is poised to take advantage of the skills and knowledge that exist across organizations, beyond IT and development teams.

As the AI landscape continues to evolve, it will be interesting to watch how Semantic Kernel and AutoGen are adopted and used in real-world applications. With its advanced features and no-code development capabilities, Microsoft's agentic AI development kit is well-positioned to revolutionize the way businesses approach workflow automation and AI-driven decision-making.