Section 01 · Unit introduction

The Azure AI service family

Generative models are not the whole of Azure AI. Alongside Azure OpenAI sits a family of task-specific services — language, vision, speech, and document intelligence — that solve well-defined problems reliably and cheaply. A capable solution often combines them: document intelligence to extract structured data from a scanned form, a language service to classify it, and a generative model to summarise the result. Knowing what each service does well keeps you from reaching for a large language model when a purpose-built service would be faster, cheaper, and more accurate.

This unit surveys those services, then goes deep on the pattern most production solutions need: retrieval-augmented generation (RAG) with Azure AI Search, which grounds a generative model in your own data.

By the end of this unit
  • Match a problem to the right Azure AI service rather than defaulting to a generative model.
  • Describe the RAG pattern and the role Azure AI Search plays in grounding responses.
  • Identify the steps that turn a RAG prototype into a production-grade solution.

Generative versus task-specific services

  • Open-ended generation: drafting, summarising, answering in natural language
  • Reasoning over context you supply at request time
  • Tasks where the output format and content are flexible
  • A well-defined, repeatable task with a known output shape
  • Extracting fields from a form, transcribing speech, detecting objects
  • When accuracy, cost, and latency on that narrow task all matter
  • Document intelligence extracts → language service classifies → model summarises
  • Speech-to-text transcribes → model answers questions about the transcript
  • The task-specific service does the structured work; the model does the open-ended work
Reach for the smallest tool that solves the problem. A document intelligence model that extracts an invoice total is more accurate, cheaper, and faster than asking a language model to read the scan.
Working principle · Solution design
Section 02

Language, vision, speech, documents

The Azure AI services group purpose-built capabilities behind consistent APIs and the same enterprise controls as Azure OpenAI. Here are the four you will reach for most often.

Azure AI Language

Text analytics: entity recognition, key-phrase extraction, sentiment, language detection, summarisation, and custom classification. Use it to structure unstructured text — routing support tickets, tagging documents, or extracting named entities — without invoking a generative model for every record.

Azure AI Vision

Image analysis: object and tag detection, optical character recognition (OCR) for text in images, and image captioning. Pair it with generative models for multimodal scenarios, or use it standalone where the task is a defined classification or extraction.

Azure AI Speech

Speech-to-text (transcription), text-to-speech (natural voices), and speech translation. Common in contact-centre and accessibility scenarios. A frequent pattern: transcribe a call with Speech, then use a generative model to summarise it and extract action items.

Azure AI Document Intelligence

Extracts structured data from documents — invoices, receipts, forms, contracts — including key-value pairs, tables, and layout. Prebuilt models cover common document types; custom models can be trained for your own forms. Far more reliable than asking a language model to parse a scanned PDF.

Design note

The deciding question is whether the task is open-ended or well-defined. Open-ended generation and reasoning are a generative model's strength. A repeatable extraction or classification with a known output shape is usually better served by a task-specific service — and the two compose cleanly in a pipeline.

Section 03

RAG with Azure AI Search

A generative model knows only what it was trained on and what you give it at request time. Retrieval-augmented generation (RAG) is the pattern for grounding a model in your own current data: retrieve the most relevant content, supply it to the model as context, and have the model answer using that content. Azure AI Search is the retrieval engine in the Microsoft stack.

How the pattern works

Index your content
Your documents are chunked into passages, each passage is converted to a vector embedding (using an embedding model), and the chunks plus their vectors are stored in an Azure AI Search index. Chunking strategy matters: chunks too large dilute relevance and waste tokens; chunks too small lose context. Store useful metadata alongside each chunk so you can filter and cite.
Retrieve at query time
When a user asks a question, you embed the question and search the index. Hybrid retrieval — combining keyword (BM25) and vector search — generally outperforms either alone, and adding a semantic re-ranker further improves which chunks rise to the top. The goal is to return a small set of highly relevant passages, not a large set of loosely relevant ones.
Generate the grounded answer
The retrieved passages are placed into the model's prompt as context, with an instruction to answer using only that context and to say when the answer is not present. This grounding is what reduces hallucination and lets the model answer about your private, current data. Pass through citations so the user can see the source of each claim.
Better retrieval beats more retrieval
The single biggest quality lever in RAG is retrieval quality. Fewer, more relevant chunks improve answer quality and reduce token cost at the same time. Invest effort in chunking, hybrid search, re-ranking, and filtering before reaching for a larger model — a small model with excellent retrieval often beats a large model with poor retrieval.
Reflect

Think of a question your users currently cannot get answered from your systems. What corpus would need to be indexed to answer it, and how current does that index need to be? RAG quality starts with the content you index and how fresh you keep it — the model is the last step, not the first.

Section 04

From prototype to production-grade

A RAG prototype can be stood up in an afternoon. A production-grade one accounts for evaluation, freshness, safety, and the operational realities the prototype ignored. Walk these four before you ship.

1. Evaluate, do not eyeball

Build an evaluation set of representative questions with known good answers, and measure groundedness, relevance, and accuracy systematically. Azure AI Foundry's evaluation tooling supports this. Eyeballing a few demo questions hides the failure cases that real users will find on day one.

2. Keep the index fresh

Decide how the index stays current as source content changes — scheduled re-indexing, incremental updates, or change-driven pipelines. A RAG system answering from stale content is confidently wrong, which is worse than admitting it does not know.

3. Apply content safety and grounding guards

Use Azure AI Content Safety to filter harmful inputs and outputs, and instruct the model to refuse when the answer is not in the retrieved context rather than inventing one. Pass citations through so answers are verifiable. Safety and groundedness are production requirements, not polish.

4. Handle scale and failure

Apply the reliability and capacity practices from earlier units — retries, fallbacks, quota planning, cost attribution — to both the search service and the model. A RAG solution has two dependencies that can throttle or fail, and both need a planned response.

Continue on Microsoft Learn

Go deeper on building and grounding Azure AI solutions:

Implement Retrieval Augmented Generation with Azure AI ↗
Build AI apps and agents with Azure AI Foundry ↗
Develop AI agents with Azure AI Foundry (Applied Skill) ↗

End of Unit 4

You should now be able to:

  • Choose between a generative model and a task-specific Azure AI service.
  • Explain the RAG pattern and Azure AI Search's role in it.
  • List what turns a RAG prototype into a production-grade solution.
Section 05

Unit review

Question 1 of 4

You need to extract the line items and totals from thousands of scanned invoices. Which service fits best?

Question 2 of 4

What is the core purpose of the RAG pattern?

Question 3 of 4

In a RAG solution, what generally gives the biggest improvement in answer quality?

Question 4 of 4

Which step is essential to move a RAG prototype to production?

End of module

You have completed Course 04: Building With Azure AI. Next: Azure AI in Production — monitoring, resilience, safe deployment, and cost governance at scale.

Craig Stanley Studio · Deploy — Architecture, Integration & Operations · Building With Azure AI · Access by direct link only.