Trustrorthy AI
Knowledge Base

AI Development Tools

AI Development Tools account for the specific tooling used by developers to build AI capabilities, and whether the organization has deployed, configured, and made them generally available to their developer population. We can think of these tools as belonging to one of three classes:

• Large language models (LLMs) and other AI models themselves;

• AI tools like GitHub Copilot or Power Apps Copilot that increase developer productivity across many technologies, AI or not, by reducing time spent writing code;

• Developer tools like Azure AI Studio or Copilot Studio that enable or accelerate development of AI workloads specifically.

Let’s take each type in turn.

As of late 2024 most organizations were not training their own large language models (LLMs), but rather meta-prompting existing LLMs through various AI studios, leveraging pre-existing retrieval augmented generation (RAG) and other patterns, fine-tuning models, and extending the models to which they already had access. This is the same as the operating system paradigm. Every company, or even every sector, does not have a custom operating system that they build. Instead, they rely heavily on custom apps for their industry and their specific business.

Given that existing AI models are already quite diverse, and the diversity of capability is likely to increase as startups, the open-source community and other innovators push the bounds of what is possible. It seems unlikely that this status quo will change significantly in the near term. Technology firms and open-source communities will continue to build and mature their LLMs and other models, but investments from Microsoft and its competitors portend a world in which typical firms across “non tech” sectors are able to leverage pre-existing models in increasingly sophisticated, and easily accessible, ways. In other words, in almost all cases it is both unnecessary and unlikely that a firm in a non-technology sector will achieve results - at least not with any reasonable level of investment—that surpass those of the specialist firms in the near future.

Given that existing AI models are already quite diverse, and the diversity of capability is likely to increase as startups, the open-source community and other innovators push the bounds of what is possible. It seems unlikely that this status quo will change significantly in the near term. Technology firms and open-source communities will continue to build and mature their LLMs and other models, but investments from Microsoft and its competitors portend a world in which typical firms across “non tech” sectors are able to leverage pre-existing models in increasingly sophisticated, and easily accessible, ways. In other words, in almost all cases it is both unnecessary and unlikely that a firm in a non-technology sector will achieve results - at least not with any reasonable level of investment—that surpass those of the specialist firms in the near future.

We’ve worked with some organizations eager to employ multiple LLMs or to train their own but were struck by how immature all of these organizations were across the 25 AI Strategy Framework dimensions. We’ll sum it up by saying, “Better to learn how to play the piano before you attempt to compose a symphony”.

Just as most organizations are not training their own LLMs, so should nearly every organization assume that they’ll leverage the LLMs provided by their preferred technology vendor - be it Microsoft, Google, AWS, etc. - and avoid oversubscribing to too many until they’ve reached a sophisticated point of maturity across their AI portfolio (a consistent score of “4” or “Proactive” in the AI Maturity Model that we’ll discuss later).

We’ll not spend much time on the second class of AI development tools, those like GitHub Copilot or Power Apps Copilot - those that increase developer productivity across many technologies, AI or not, by reducing time spent developing software. These tools can be as valuable to a software developer as tools like ChatGPT or Copilot for Microsoft 365 are to colleagues in other disciplines, but a certain sense of stating the obvious abounds: Organizations with colleagues developing technologies using tools that offer an AI-enabled assistant should go ahead and turn the assistant on. There are few more straightforward ways of wringing value from artificial intelligence than to put purpose-built AI into the hands of technically sophisticated users.

Which brings us to our third class of AI development tools, those like Azure AI Studio or Copilot Studio that enable or accelerate development of AI workloads specifically. Given the current rapid advancement in this area, it can be challenging to discern which tools are worthy of investment both in budget and in the time required to deploy them. We’ll consider two candidates below.

Copilot Studio is an AI development tool used to create agents (formerly “custom Copilots”), as well as agents that extend existing Copilots.

“Low-code” to a large extent, Copilot Studio, offers tools that allow developers to efficiently design, train, and deploy agents to reason over data housed in a range of data storage technologies

Figure 20: Simple architecture depicting how Azure provides AI infrastructure, foundation models, and AI orchestration both to Microsoft Copilots and agents created by developers using a combination of Copilot Studio, Azure AI Studio, and development tools you’re likely already familiar with (Visual Studio and GitHub).

Copilot Studio is amongst the most accessible and easiest to implement AI development tools so is a good place to begin. Though it is absolutely accessible to less technical “citizen developers” in personal or small team productivity scenarios, experienced engineers will be able to develop more enterprise grade workloads thanks to familiarity with programming languages such as Python or R, ability to preprocess and analyze data, and knowledge of machine learning.

Azure AI Studio, by contrast, is firmly the domain of experienced engineers. If Copilot Studio is the first port-of-call for the development of extensible AI workloads (see the later dimension Extensible AI), then Azure AI Studio is best employed for more advanced extensible AI as well as Differential AI workloads that exceed the former’s capabilities. It provides a robust environment for developing, training, and deploying AI models, making it a popular choice among developers. Azure AI Studio offers additional benefits including scalability for larger or more complex AI development projects and integration with technical services across all of Azure.

Developers here will benefit from their proficiency in other cloud technologies where an understanding of cloud infrastructure and the data platform is essential for effectively leveraging Azure AI Studio - proficiency in languages such at Python, experience with machine learning libraries, and data science skills such as preprocessing, model training and evaluation. What’s mandatory is proficiency in building end-to-end apps since AI apps are still...apps.

The “better to learn how to play the piano before you attempt to compose a symphony” wisdom rings true with these AI development tools, as well. Though there are others available and, in some cases, required to get the job done, organizations ought not invest significantly in pre-deploying them. For those choosing to rely on Microsoft technology, the duo of Copilot Studio and Azure AI Studio is a solid combination, at least in the early days.