Trustrorthy AI
Knowledge Base

Extensible AI

Though it barely registered as a concept in early 2024, by late 2024 it has become apparent that Extensible AI may be where the bulk of an organization’s AI workloads are categorized in the coming years.

Admittedly, there is a lot of grey area here, and in any case, the spectrum of incremental, extensible, and differential workloads is really meant more as a conceptual framework than any hard technical boundary. That said, extensible AI occupies the broad middle range where more incremental workloads are extended to suit an organization’s specific scenarios.

Figure 22: Incremental, Extensible, and Differential AI workloads provide a conceptual framework—a spectrum—for evaluating an organization’s portfolio of investment in AI workloads. Health portfolios are well balanced, though different organizations and industries will have varying risk tolerances.

For example, consider the scenario above where an organization has consolidated a store of its proprietary data and desires that AI reason over this data to produce generative responses via a chat-based user interface, including directly through Copilot for Microsoft 365. This would have required engineering a bespoke RAG-based workload even as recently as early 2024, complexity that would have landed this scenario squarely in the realm of Differential AI.

Figure 23: Simplified models of typical Extensible AI scenarios.

Developers can extend many Copilots using their organization’s data to further enhance the functionality of these Copilots, making them even more relevant and effective for specific business needs. Copilot Studio is increasingly purpose-built for just these types of scenarios, so if you’ve not read more of the AI Developer Tools dimension, this is a good segue to that earlier topic. Let’s now discuss several other key extensibility tools:

• Graph connectors allow data from external systems to be indexed and made searchable within the Microsoft ecosystem. By integrating these connectors, developers can extend the capabilities of Microsoft Copilots to access and utilize data from a variety of sources. Graph Connectors use APIs to crawl data from external sources such as file systems, databases, and SaaS applications. This data is then indexed and made available within Microsoft Search experiences. Consider a company that uses both SharePoint and an external CRM system. By creating a Graph Connector for the CRM, employees can search for customer data directly from their Microsoft 365 environment, enabling seamless access to critical information;

• Teams message extensions provide a way to extend the functionality of Microsoft Teams by allowing users to interact with your services and data directly within Teams messages. They can be categorized into Search Message Extensions and Action Message Extensions. These extensions allow users to search for information from external systems and insert the results into a Teams conversation:

o A Search Message Extension could be developed to allow employees to search for knowledge base articles and insert relevant excerpts into a Teams chat, facilitating quick access to helpful information during discussions;

o Action Message Extensions enable users to initiate workflows or perform actions based on message content, for example, allowing users to create a new task in a project management tool directly from a Teams message, streamlining task management and reducing context switching;

• API plugins (in preview as of fall 2024) enable developers to integrate third-party APIs with Microsoft Copilots, allowing these intelligent assistants to interact with external services and retrieve data in real-time. API plugins involve writing custom code that calls external APIs and processes the returned data. This data can then be used by the Copilot to provide informed responses or perform specific actions. A Copilot could be extended with an API plugin that retrieves weather data from an external service. Users could ask the Copilot for the current weather conditions or a forecast, and the Copilot would provide accurate, up-to-date information;

• Copilot Studio Agents allow for the definition of Copilot behaviors through configuration rather than code. This approach can simplify the process of extending Copilots and make it accessible to non-developers. Copilot Studio Agents use configuration files or low-code tools to define how they should interact with users and data sources. These configurations can specify triggers, responses, and integrations with other services. A Copilot Studio Agent could be configured to monitor a SharePoint list for new entries and send a notification to a Teams channel whenever a new item is added. This setup requires minimal coding and can be quickly adapted to changing business needs;

• The Teams AI Library provides tools and frameworks for building intelligent bots and Copilots within Microsoft Teams. By leveraging these resources, developers can create sophisticated, AI-powered assistants that enhance collaboration and productivity. The Teams AI library includes pre-built models for natural language understanding, tools for building conversational interfaces, and integration capabilities with Microsoft Graph and other services. Using the Teams AI library, for example, a developer could create a Copilot that helps employees schedule meetings. The Copilot could understand natural language requests, check participants' availability, and suggest suitable meeting times, all within a Teams conversation.

The bottom line of extensible AI, though, is that architects need to quickly transition from making black and white assessments of whether a pre-built AI tool such as Copilot for Microsoft 365 can do the job, or if an entirely bespoke AI development is needed to increasingly embrace extensibility scenarios that fall in the broad middle of the spectrum. In other words, seek first to extend, then consider entirely custom development.