|
| 1 | +--- |
| 2 | +title: Intelligent applications |
| 3 | +description: "Use AI options such as OpenAI and vectors to build intelligent applications with Azure SQL Database." |
| 4 | +author: damauri |
| 5 | +ms.author: damauri |
| 6 | +ms.reviewer: damauri, randolphwest |
| 7 | +ms.date: 10/30/2023 |
| 8 | +ms.service: sql-database |
| 9 | +ms.topic: conceptual |
| 10 | +--- |
| 11 | + |
| 12 | +# Intelligent applications with Azure SQL Database |
| 13 | + |
| 14 | +[!INCLUDE [asdb](../includes/appliesto-sqldb.md)] |
| 15 | + |
| 16 | +This article provides an overview of using artificial intelligence (AI) options, such as OpenAI and vectors, to build intelligent applications with Azure SQL Database. |
| 17 | + |
| 18 | +## Overview |
| 19 | + |
| 20 | +Large language models (LLMs) enable developers to create AI-powered applications with a familiar user experience. |
| 21 | + |
| 22 | +Using LLMs in applications brings greater value and an improved user experience when the models can access the right data, at the right time, from your application's database. This process is known as Retrieval Augmented Generation (RAG) and Azure SQL Database has many features that support this new pattern, making it a great database to build intelligent applications. |
| 23 | + |
| 24 | +The following links provide sample code of various Azure SQL Database options to build intelligent applications: |
| 25 | + |
| 26 | +| AI Option | Description | |
| 27 | +| --- | --- | |
| 28 | +| **[Azure OpenAI](#azure-openai)** | Generate embeddings for RAG and integrate with any model supported by Azure OpenAI. | |
| 29 | +| **[Vectors](#vectors)** | Learn how to store and query vectors in Azure SQL Database. | |
| 30 | +| **[Azure Cognitive Search](#azure-cognitive-search)** | Use Azure SQL Database together with Azure Cognitive Search to train LLM on your data. | |
| 31 | +| **[Intelligent applications](#intelligent-applications)** | Learn how to create an end-to-end solution using a common pattern that can be replicated in any scenario. | |
| 32 | + |
| 33 | +## Key concepts |
| 34 | + |
| 35 | +This section includes key concepts that are critical for implementing RAG with Azure SQL Database and Azure OpenAI. |
| 36 | + |
| 37 | +### <a id="retrieval-augmented-generation"></a> Retrieval Augmented Generation (RAG) |
| 38 | + |
| 39 | +RAG is a technique that enhances the LLM's ability to produce relevant and informative responses by retrieving additional data from external sources. For example, RAG can query articles or documents that contain domain-specific knowledge related to the user's question or prompt. The LLM can then use this retrieved data as a reference when generating its response. For example, a simple RAG pattern using Azure SQL Database could be: |
| 40 | + |
| 41 | +1. Insert data into an Azure SQL Database table. |
| 42 | +1. Link Azure SQL Database to Azure Cognitive Search. |
| 43 | +1. Create an Azure OpenAI GPT4 model and connect it to Azure Cognitive Search. |
| 44 | +1. Chat and ask questions about your data using the trained Azure OpenAI model from your application and from Azure SQL Database. |
| 45 | + |
| 46 | +The RAG pattern, with prompt engineering, serves the purpose of enhancing response quality by offering more contextual information to the model. RAG enables the model to apply a broader knowledgebase by incorporating relevant external sources into the generation process, resulting in more comprehensive and informed responses. For more information on *grounding* LLMs, see [Grounding LLMs - Microsoft Community Hub](https://techcommunity.microsoft.com/t5/fasttrack-for-azure/grounding-llms/ba-p/3843857). |
| 47 | + |
| 48 | +### Prompts and prompt engineering |
| 49 | + |
| 50 | +A prompt refers to specific text or information that serves as an instruction to an LLM, or as contextual data that the LLM can build upon. A prompt can take various forms, such as a question, a statement, or even a code snippet. |
| 51 | + |
| 52 | +The following is a list of prompts that can be used to generate a response from an LLM: |
| 53 | + |
| 54 | +- **Instructions**: provide directives to the LLM |
| 55 | +- **Primary content**: gives information to the LLM for processing |
| 56 | +- **Examples**: help condition the model to a particular task or process |
| 57 | +- **Cues**: direct the LLM's output in the right direction |
| 58 | +- **Supporting content**: represents supplemental information the LLM can use to generate output |
| 59 | + |
| 60 | +The process of creating good prompts for a scenario is called *prompt engineering*. For more information about prompts and best practices for prompt engineering, see [Azure OpenAI Service](/azure/ai-services/openai/concepts/prompt-engineering). |
| 61 | + |
| 62 | +### Tokens |
| 63 | + |
| 64 | +Tokens are small chunks of text generated by splitting the input text into smaller segments. These segments can either be words or groups of characters, varying in length from a single character to an entire word. For instance, the word `hamburger` would be divided into tokens such as `ham`, `bur`, and `ger` while a short and common word like `pear` would be considered a single token. |
| 65 | + |
| 66 | +In Azure OpenAI, input text provided to the API is turned into tokens (tokenized). The number of tokens processed in each API request depends on factors such as the length of the input, output, and request parameters. The quantity of tokens being processed also impacts the response time and throughput of the models. There are limits to the number of tokens each model can take in a single request/response from Azure OpenAI. To learn more, see [Azure OpenAI Service quotas and limits](/azure/ai-services/openai/quotas-limits). |
| 67 | + |
| 68 | +### Vectors |
| 69 | + |
| 70 | +Vectors are ordered arrays of numbers (typically floats) that can represent information about some data. For example, an image can be represented as a vector of pixel values, or a string of text can be represented as a vector or ASCII values. The process to turn data into a vector is called *vectorization*. |
| 71 | + |
| 72 | +### Embeddings |
| 73 | + |
| 74 | +Embeddings are vectors that represent important features of data. Embeddings are often learned by using a deep learning model, and machine learning and AI models utilize them as features. Embeddings can also capture semantic similarity between similar concepts. For example, in generating an embedding for the words `person` and `human`, we would expect their embeddings (vector representation) to be similar in value since the words are also semantically similar. |
| 75 | + |
| 76 | +Azure OpenAI features models to create embeddings from text data. The service breaks text out into tokens and generates embeddings using models pretrained by OpenAI. To learn more, see [Creating embeddings with Azure OpenAI](/azure/ai-services/openai/concepts/understand-embeddings). |
| 77 | + |
| 78 | +### Vector search |
| 79 | + |
| 80 | +Vector search refers to the process of finding all vectors in a dataset that are semantically similar to a specific query vector. Therefore, a query vector for the word `human` searches the entire dictionary for semantically similar words, and should find the word `person` as a close match. This closeness, or distance, is measured using a similarity metric such as cosine similarity. The closer vectors are in similarity, the smaller is the distance between them. |
| 81 | + |
| 82 | +Consider a scenario where you run a query over millions of document to find the most similar documents in your data. You can create embeddings for your data and query documents using Azure OpenAI. Then, you can perform a vector search to find the most similar documents from your dataset. However, performing a vector search across a few examples is trivial. Performing this same search across thousands, or millions, of data points becomes challenging. There are also trade-offs between exhaustive search and approximate nearest neighbor (ANN) search methods including latency, throughput, accuracy, and cost, all of which depends on the requirements of your application. |
| 83 | + |
| 84 | +Since Azure SQL Database embeddings can be efficiently stored and queried using to columnstore index support, allowing exact nearest neighbor search with great performance, you don't have to decide between accuracy and speed: you can have both. Storing vector embeddings alongside the data in an integrated solution minimizes the need to manage data synchronization and accelerates your time-to-market for AI application development. |
| 85 | + |
| 86 | +## Azure OpenAI |
| 87 | + |
| 88 | +Embedding is the process of representing the real world as data. Text, images, or sounds can be converted into embeddings. Azure OpenAI models are able to transform real-world information into embeddings. The models are available as REST endpoints and thus can be easily consumed from Azure SQL Database using the [`sp_invoke_external_rest_endpoint`](/sql/relational-databases/system-stored-procedures/sp-invoke-external-rest-endpoint-transact-sql) system stored procedure: |
| 89 | + |
| 90 | +```sql |
| 91 | +DECLARE @retval INT, @response NVARCHAR(MAX); |
| 92 | +DECLARE @payload NVARCHAR(MAX); |
| 93 | + |
| 94 | +SET @payload = JSON_OBJECT('input': @text); |
| 95 | + |
| 96 | +EXEC @retval = sp_invoke_external_rest_endpoint @url = 'https://<openai-url>/openai/deployments/<model-name>/embeddings?api-version=2023-03-15-preview', |
| 97 | + @method = 'POST', |
| 98 | + @credential = [https://<openai-url>/openai/deployments/<model-name>], |
| 99 | + @payload = @payload, |
| 100 | + @response = @response OUTPUT; |
| 101 | + |
| 102 | +SELECT CAST([key] AS INT) AS [vector_value_id], |
| 103 | + CAST([value] AS FLOAT) AS [vector_value] |
| 104 | +FROM OPENJSON(JSON_QUERY(@response, '$.result.data[0].embedding')); |
| 105 | +``` |
| 106 | + |
| 107 | +Using a call to a REST service to get embeddings is just one of the integration options you have when working with SQL Database and OpenAI. You can let any of the [available models](/azure/ai-services/openai/concepts/models) access data stored in Azure SQL Database to create solutions where your users can interact with the data, such as the following example. |
| 108 | + |
| 109 | +:::image type="content" source="media/ai-artificial-intelligence-intelligent-applications/data-chatbot.png" alt-text="Screenshot of an AI bot answering the question using data stored in Azure SQL Database."::: |
| 110 | + |
| 111 | +For additional examples on using SQL Database and OpenAI, see the following articles: |
| 112 | + |
| 113 | +- [Generate images with Azure OpenAI Service (DALL-E) and Azure SQL Database](https://devblogs.microsoft.com/azure-sql/generate-images-with-openai-and-azure-sql/) |
| 114 | +- [Using OpenAI REST Endpoints with Azure SQL Database](https://devblogs.microsoft.com/azure-sql/using-openai-rest-endpoints-with-azure-sql-database/) |
| 115 | + |
| 116 | +## Vectors |
| 117 | + |
| 118 | +Although Azure SQL Database doesn't have a native **vector** type, a vector is nothing more than an ordered tuple, and relational databases are great at managing tuples. You can think of a tuple as the formal term for a row in a table. |
| 119 | + |
| 120 | +Azure SQL Database also supports columnstore indexes and [batch mode execution](/sql/relational-databases/query-processing-architecture-guide#batch-mode-execution). A vector-based approach is used for batch mode processing, which means that each column in a batch has its own memory location where it's stored as a vector. This allows for faster and more efficient processing of data in batches. |
| 121 | + |
| 122 | +The following is example of how a vector can be stored in SQL Database: |
| 123 | + |
| 124 | +```sql |
| 125 | +CREATE TABLE [dbo].[wikipedia_articles_embeddings_titles_vector] |
| 126 | +( |
| 127 | + [article_id] [int] NOT NULL, |
| 128 | + [vector_value_id] [int] NOT NULL, |
| 129 | + [vector_value] [float] NOT NULL |
| 130 | +) |
| 131 | +GO |
| 132 | + |
| 133 | +CREATE CLUSTERED COLUMNSTORE INDEX ixc |
| 134 | + ON dbo.wikipedia_articles_embeddings_titles_vector |
| 135 | + ORDER (article_id); |
| 136 | +GO |
| 137 | +``` |
| 138 | + |
| 139 | +For an example that uses a common subset of Wikipedia articles with embeddings already generated using OpenAI, see [Vector similarity search with Azure SQL Database and OpenAI](https://devblogs.microsoft.com/azure-sql/vector-similarity-search-with-azure-sql-database-and-openai/). |
| 140 | + |
| 141 | +## Azure Cognitive Search |
| 142 | + |
| 143 | +Implement RAG-patterns with Azure SQL Database and Azure Cognitive Search. You can run supported chat models on data stored in Azure SQL Database, without having to train or fine-tune models, thanks to the integration of Azure Cognitive Search with Azure OpenAI and Azure SQL Database. Running models on your data enables you to chat on top of, and analyze, your data with greater accuracy and speed. |
| 144 | + |
| 145 | +- [Azure OpenAI on your data](/azure/ai-services/openai/concepts/use-your-data) |
| 146 | +- [Retrieval Augmented Generation (RAG) in Azure Cognitive Search](/azure/search/retrieval-augmented-generation-overview) |
| 147 | + |
| 148 | +## Intelligent applications |
| 149 | + |
| 150 | +Azure SQL Database can be used to build intelligent applications that include AI features, such as recommenders, and Retrieval Augmented Generation (RAG) as the following diagram demonstrates: |
| 151 | + |
| 152 | +:::image type="content" source="media/ai-artificial-intelligence-intelligent-applications/session-recommender-architecture.png" alt-text="Diagram of different AI features to build intelligent applications with Azure SQL Database."::: |
| 153 | + |
| 154 | +For an end-to-end sample to build a recommender using session abstract as a sample dataset, see [How I built a session recommender in 1 hour using Open AI](https://devblogs.microsoft.com/azure-sql/how-i-built-a-session-recommender-in-1-hour-using-open-ai/). |
| 155 | + |
| 156 | +### LangChain integration |
| 157 | + |
| 158 | +LangChain is a well-known framework for developing applications powered by language models. |
| 159 | + |
| 160 | +For an example that shows how LangChain can be used to create a Chatbot on your own data, see [Building your own DB Copilot for Azure SQL with Azure OpenAI GPT-4](https://devblogs.microsoft.com/azure-sql/building-your-own-db-copilot-for-azure-sql-with-azure-openai-gpt-4/). |
| 161 | + |
| 162 | +## Related content |
| 163 | + |
| 164 | +- [Create and deploy an Azure OpenAI Service resource](/azure/ai-services/openai/how-to/create-resource?pivots=web-portal) |
| 165 | +- [Embeddings models](/azure/ai-services/openai/concepts/models#embeddings-models) |
0 commit comments