Skip to content

Commit 8216d78

Browse files
Roopan-Microsoftross-p-smithgpickettFr4nc3Pavan-Microsoft
authored
fix: Merging main branch into demo (#1616)
Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Ross Smith <[email protected]> Co-authored-by: gpickett <[email protected]> Co-authored-by: Francia Riesco <[email protected]> Co-authored-by: Pavan-Microsoft <[email protected]> Co-authored-by: Francia Riesco <[email protected]> Co-authored-by: Prajwal D C <[email protected]> Co-authored-by: Ajit Padhi <[email protected]> Co-authored-by: Harmanpreet-Microsoft <[email protected]> Co-authored-by: UtkarshMishra-Microsoft <[email protected]> Co-authored-by: Priyanka-Microsoft <[email protected]> Co-authored-by: Prasanjeet-Microsoft <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
1 parent 1ae5e9c commit 8216d78

File tree

93 files changed

+10861
-2775
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

93 files changed

+10861
-2775
lines changed

.env.sample

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,9 @@ AZURE_SEARCH_DATASOURCE_NAME=
2222
# Azure OpenAI for generating the answer and computing the embedding of the documents
2323
AZURE_OPENAI_RESOURCE=
2424
AZURE_OPENAI_API_KEY=
25-
AZURE_OPENAI_MODEL_INFO="{\"model\":\"gpt-35-turbo-16k\",\"modelName\":\"gpt-35-turbo-16k\",\"modelVersion\":\"0613\"}"
26-
AZURE_OPENAI_EMBEDDING_MODEL_INFO="{\"model\":\"text-embedding-ada-002\",\"modelName\":\"text-embedding-ada-002\",\"modelVersion\":\"2\"}"
25+
AZURE_OPENAI_MODEL=gpt-35-turbo
26+
AZURE_OPENAI_MODEL_NAME=gpt-35-turbo
27+
AZURE_OPENAI_EMBEDDING_MODEL=text-embedding-ada-002
2728
AZURE_OPENAI_TEMPERATURE=0
2829
AZURE_OPENAI_TOP_P=1.0
2930
AZURE_OPENAI_MAX_TOKENS=1000
@@ -35,7 +36,6 @@ AZURE_OPENAI_STREAM=True
3536
AzureWebJobsStorage=
3637
BACKEND_URL=http://localhost:7071
3738
DOCUMENT_PROCESSING_QUEUE_NAME=
38-
# Azure Blob Storage for storing the original documents to be processed
3939
AZURE_BLOB_ACCOUNT_NAME=
4040
AZURE_BLOB_ACCOUNT_KEY=
4141
AZURE_BLOB_CONTAINER_NAME=
@@ -63,6 +63,11 @@ AZURE_KEY_VAULT_ENDPOINT=
6363
# Chat conversation type to decide between custom or byod (bring your own data) conversation type
6464
CONVERSATION_FLOW=
6565
# Chat History CosmosDB Integration Settings
66-
AZURE_COSMOSDB_INFO="{\"accountName\":\"cosmos-abc123\",\"databaseName\":\"db_conversation_history\",\"containerName\":\"conversations\"}"
67-
AZURE_COSMOSDB_ACCOUNT_KEY=
66+
AZURE_COSMOSDB_ACCOUNT_NAME=
67+
AZURE_COSMOSDB_DATABASE_NAME=
68+
AZURE_COSMOSDB_CONVERSATIONS_CONTAINER_NAME=
6869
AZURE_COSMOSDB_ENABLE_FEEDBACK=
70+
AZURE_POSTGRESQL_HOST_NAME=
71+
AZURE_POSTGRESQL_DATABASE_NAME=
72+
AZURE_POSTGRESQL_USER=
73+
DATABASE_TYPE="CosmosDB"

.github/workflows/build-docker-images.yml

Lines changed: 4 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,7 @@
11
name: Build Docker Images
22

33
on:
4-
workflow_run:
5-
workflows: [Tests]
6-
types: [completed]
4+
push:
75
branches:
86
- main
97
- dev
@@ -22,7 +20,6 @@ on:
2220

2321
jobs:
2422
docker-build:
25-
if: ${{ github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success' }}
2623
strategy:
2724
matrix:
2825
include:
@@ -34,9 +31,9 @@ jobs:
3431
dockerfile: docker/Frontend.Dockerfile
3532
uses: ./.github/workflows/build-docker.yml
3633
with:
37-
registry: ${{ github.event.workflow_run.head_branch == 'main' && 'fruoccopublic.azurecr.io' || 'cwydcontainerreg.azurecr.io'}}
38-
username: ${{ github.event.workflow_run.head_branch == 'main' && 'fruoccopublic' || 'cwydcontainerreg'}}
34+
registry: ${{ github.ref_name == 'main' && 'fruoccopublic.azurecr.io' || 'cwydcontainerreg.azurecr.io'}}
35+
username: ${{ github.ref_name == 'main' && 'fruoccopublic' || 'cwydcontainerreg'}}
3936
app_name: ${{ matrix.app_name }}
4037
dockerfile: ${{ matrix.dockerfile }}
41-
push: ${{ github.event.workflow_run.head_branch == 'main' || github.event.workflow_run.head_branch == 'dev' || github.event.workflow_run.head_branch == 'demo' }}
38+
push: ${{ github.ref_name == 'main' || github.ref_name == 'dev' || github.ref_name == 'demo' }}
4239
secrets: inherit

.github/workflows/build-docker.yml

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,6 @@ jobs:
2828
docker-build:
2929
runs-on: ubuntu-latest
3030
steps:
31-
3231
- name: Checkout
3332
uses: actions/checkout@v4
3433

@@ -61,7 +60,7 @@ jobs:
6160
context: .
6261
file: ${{ inputs.dockerfile }}
6362
push: ${{ inputs.push }}
64-
cache-from: type=registry,ref=${{ inputs.registry }}/${{ inputs.app_name}}:${{ github.ref_name == 'main' && 'latest' || github.ref_name == 'dev' && 'dev' || github.ref_name == 'demo' && 'demo' || 'latest' }}
63+
cache-from: type=registry,ref=${{ inputs.registry }}/${{ inputs.app_name}}:${{ github.ref_name == 'main' && 'latest' || github.ref_name == 'dev' && 'dev' || github.ref_name == 'demo' && 'demo' || github.head_ref || github.ref_name }}
6564
tags: |
66-
${{ inputs.registry }}/${{ inputs.app_name}}:${{ github.ref_name == 'main' && 'latest' || github.ref_name == 'dev' && 'dev' || github.ref_name == 'demo' && 'demo' || 'latest' }}
65+
${{ inputs.registry }}/${{ inputs.app_name}}:${{ github.ref_name == 'main' && 'latest' || github.ref_name == 'dev' && 'dev' || github.ref_name == 'demo' && 'demo' || github.head_ref || 'default' }}
6766
${{ inputs.registry }}/${{ inputs.app_name}}:${{ steps.date.outputs.date }}_${{ github.run_number }}

.github/workflows/ci.yml

Lines changed: 1 addition & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,6 @@ jobs:
5252
env:
5353
AZURE_ENV_NAME: ${{ github.run_id }}
5454
AZURE_LOCATION: ${{ vars.AZURE_LOCATION }}
55-
AZURE_RESOURCE_GROUP: ${{ vars.AZURE_RESOURCE_GROUP }}
5655
with:
5756
imageName: ghcr.io/azure-samples/chat-with-your-data-solution-accelerator
5857
cacheFrom: ghcr.io/azure-samples/chat-with-your-data-solution-accelerator
@@ -66,15 +65,12 @@ jobs:
6665
AZURE_SUBSCRIPTION_ID
6766
AZURE_ENV_NAME
6867
AZURE_LOCATION
69-
AZURE_RESOURCE_GROUP
70-
7168
- name: Tidy up
7269
uses: devcontainers/[email protected]
7370
if: always()
7471
env:
7572
AZURE_ENV_NAME: ${{ github.run_id }}
7673
AZURE_LOCATION: ${{ vars.AZURE_LOCATION }}
77-
AZURE_RESOURCE_GROUP: ${{ vars.AZURE_RESOURCE_GROUP }}
7874
with:
7975
push: never
8076
imageName: ghcr.io/azure-samples/chat-with-your-data-solution-accelerator
@@ -87,8 +83,7 @@ jobs:
8783
AZURE_SUBSCRIPTION_ID
8884
AZURE_ENV_NAME
8985
AZURE_LOCATION
90-
AZURE_RESOURCE_GROUP
91-
86+
9287
- name: Send Notification on Failure
9388
if: failure()
9489
run: |

.github/workflows/sync-branches.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ jobs:
1515

1616
steps:
1717
- name: Checkout repository
18-
uses: actions/checkout@v3
18+
uses: actions/checkout@v4
1919
with:
2020
fetch-depth: 0 # Fetch all history for accurate branch comparison
2121

Makefile

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -57,14 +57,10 @@ azd-login: ## 🔑 Login to Azure with azd and a SPN
5757
@echo -e "\e[34m$@\e[0m" || true
5858
@azd auth login --client-id ${AZURE_CLIENT_ID} --client-secret ${AZURE_CLIENT_SECRET} --tenant-id ${AZURE_TENANT_ID}
5959

60-
az-login: ## 🔑 Login to Azure with az and a SPN
61-
az login --service-principal -u ${AZURE_CLIENT_ID} -p ${AZURE_CLIENT_SECRET} --tenant ${AZURE_TENANT_ID}
62-
63-
deploy: azd-login az-login ## 🚀 Deploy everything to Azure
60+
deploy: azd-login ## 🚀 Deploy everything to Azure
6461
@echo -e "\e[34m$@\e[0m" || true
6562
@azd env new ${AZURE_ENV_NAME}
6663
@azd env set AZURE_APP_SERVICE_HOSTING_MODEL code --no-prompt
67-
@az group create --name ${AZURE_RESOURCE_GROUP} --location ${AZURE_LOCATION}
6864
@azd provision --no-prompt
6965
@azd deploy web --no-prompt
7066
@azd deploy function --no-prompt

README.md

Lines changed: 35 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,8 @@ urlFragment: chat-with-your-data-solution-accelerator
4848
## User story
4949
Welcome to the *Chat with your data* Solution accelerator repository! The *Chat with your data* Solution accelerator is a powerful tool that combines the capabilities of Azure AI Search and Large Language Models (LLMs) to create a conversational search experience. This solution accelerator uses an Azure OpenAI GPT model and an Azure AI Search index generated from your data, which is integrated into a web application to provide a natural language interface, including [speech-to-text](docs/speech_to_text.md) functionality, for search queries. Users can drag and drop files, point to storage, and take care of technical setup to transform documents. Everything can be deployed in your own subscription to accelerate your use of this technology.
5050

51-
![Solution Architecture - Chat with your data](/docs/images/cwyd-solution-architecture.png)
51+
52+
5253

5354
### About this repo
5455

@@ -91,12 +92,15 @@ Here is a comparison table with a few features offered by Azure, an available Gi
9192
- **Single application access to your full data set**: Minimize endpoints required to access internal company knowledgebases. Reuse the same backend with the [Microsoft Teams Extension](docs/teams_extension.md)
9293
- **Natural language interaction with your unstructured data**: Use natural language to quickly find the answers you need and ask follow-up queries to get the supplemental details, including [Speech-to-text](docs/speech_to_text.md).
9394
- **Easy access to source documentation when querying**: Review referenced documents in the same chat window for additional context.
95+
- **Chat history**: Prior conversations and context are maintained and accessible through chat history.
9496
- **Data upload**: Batch upload documents of [various file types](docs/supported_file_types.md)
9597
- **Accessible orchestration**: Prompt and document configuration (prompt engineering, document processing, and data retrieval)
98+
- **Database flexibility**: Dynamic database switching allows users to choose between PostgreSQL and Cosmos DB based on their requirements. If no preference is specified the platform defaults to PostgreSQL.
9699

97100

98101
**Note**: The current model allows users to ask questions about unstructured data, such as PDF, text, and docx files. See the [supported file types](docs/supported_file_types.md).
99102

103+
100104
### Target end users
101105
Company personnel (employees, executives) looking to research against internal unstructured company data would leverage this accelerator using natural language to find what they need quickly.
102106

@@ -107,6 +111,11 @@ Tech administrators can use this accelerator to give their colleagues easy acces
107111

108112
### Use Case scenarios
109113

114+
#### Employee Onboarding Scenario
115+
The sample data illustrates how this accelerator could be used for an employee onboarding scenario in across industries.
116+
117+
In this scenario, a newly hired employee is in the process of onboarding to their organization. Leveraging the solution accelerator, she navigates through the extensive offerings of her organization’s health and retirement benefits. With the newly integrated chat history capabilities, they can revisit previous conversations, ensuring continuity and context across multiple days of research. This functionality allows the new employee to efficiently gather and consolidate information, streamlining their onboarding experience. [For more details, refer to the README](docs/employee_assistance.md).
118+
110119
#### Financial Advisor Scenario
111120
The sample data illustrates how this accelerator could be used in the financial services industry (FSI).
112121

@@ -120,12 +129,6 @@ Additionally, we have implemented a Legal Review and Summarization Assistant sce
120129
Note: Some of the sample data included with this accelerator was generated using AI and is for illustrative purposes only.
121130

122131

123-
#### Employee Onboarding Scenario
124-
The sample data illustrates how this accelerator could be used for an employee onboarding scenario in across industries.
125-
126-
In this scenario, a newly hired employee is in the process of onboarding to their organization. Leveraging the solution accelerator, she navigates through the extensive offerings of her organization’s health and retirement benefits. With the newly integrated chat history capabilities, they can revisit previous conversations, ensuring continuity and context across multiple days of research. This functionality allows the new employee to efficiently gather and consolidate information, streamlining their onboarding experience. [For more details, refer to the README](docs/employee_assistance.md).
127-
128-
129132
---
130133

131134
![One-click Deploy](/docs/images/oneClickDeploy.png)
@@ -146,6 +149,7 @@ In this scenario, a newly hired employee is in the process of onboarding to thei
146149
- Azure Storage Account
147150
- Azure Speech Service
148151
- Azure CosmosDB
152+
- Azure PostgreSQL
149153
- Teams (optional: Teams extension only)
150154

151155
### Required licenses
@@ -163,13 +167,30 @@ The following are links to the pricing details for some of the resources:
163167
- [Azure AI Document Intelligence pricing](https://azure.microsoft.com/pricing/details/ai-document-intelligence/)
164168
- [Azure Web App Pricing](https://azure.microsoft.com/pricing/details/app-service/windows/)
165169

170+
### Deployment options: PostgreSQL or Cosmos DB
171+
With the addition of PostgreSQL, customers can leverage the power of a relationship-based AI solution to enhance historical conversation access, improve data privacy, and open the possibilities for scalability.
172+
173+
Customers have the option to deploy this solution with PostgreSQL or Cosmos DB. Consider the following when deciding which database to use:
174+
- PostgreSQL enables a relationship-based AI solution and search indexing for Retrieval Augmented Generation (RAG)
175+
- Cosmos DB enables chat history and is a NoSQL-based solution. With Cosmos DB, Azure AI Search is used for storing extracted documents and embeddings.
176+
177+
178+
To review PostgreSQL configuration overview and steps, follow the link [here](docs/postgreSQL.md).
179+
![Solution Architecture - Chat with your data PostgreSQL](/docs/images/architrecture_pg.png)
180+
181+
To review Cosmos DB configuration overview and steps, follow the link [here](docs/employee_assistance.md).
182+
![Solution Architecture - Chat with your data CosmosDB](/docs/images/architecture_cdb.png)
183+
166184
### Deploy instructions
185+
The "Deploy to Azure" button offers a one-click deployment where you don’t have to clone the code. If you would like a developer experience instead, follow the [local deployment instructions](./docs/LOCAL_DEPLOYMENT.md).
186+
187+
Once you deploy to Azure, you will have the option to select PostgreSQL or Cosmos DB, see screenshot below.
167188

168-
There are two choices; the "Deploy to Azure" offers a one click deployment where you don't have to clone the code, alternatively if you would like a developer experience, follow the [Local deployment instructions](./docs/LOCAL_DEPLOYMENT.md).
189+
[![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure-Samples%2Fchat-with-your-data-solution-accelerator%2Frefs%2Fheads%2Fmain%2Finfra%2Fmain.json)
169190

170-
The demo, which uses containers pre-built from the main branch is available by clicking this button:
191+
Select either "PostgreSQL" or "Cosmos DB":
192+
![Solution Architecture - DB Selection](/docs/images/db_selection.png)
171193

172-
[![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure-Samples%2Fchat-with-your-data-solution-accelerator%2Fmain%2Finfra%2Fmain.json)
173194

174195
When Deployment is complete, follow steps in [Set Up Authentication in Azure App Service](./docs/azure_app_service_auth_setup.md) to add app authentication to your web app running on Azure App Service
175196

@@ -195,9 +216,11 @@ switch to a lower version. To find out which versions are supported in different
195216

196217
![A screenshot of the chat app.](./docs/images/web-unstructureddata.png)
197218

198-
\
199-
\
219+
220+
221+
200222
![Supporting documentation](/docs/images/supportingDocuments.png)
223+
201224
## Supporting documentation
202225

203226
### Resource links

azure.yaml

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,13 @@ metadata:
55
66
hooks:
77
postprovision:
8-
run: ./infra/prompt-flow/create-prompt-flow.sh
8+
# run: ./infra/prompt-flow/create-prompt-flow.sh
9+
posix:
10+
shell: sh
11+
run: chmod +x ./scripts/parse_env.sh && ./scripts/parse_env.sh
12+
windows:
13+
shell: pwsh
14+
run: ./scripts/parse_env.ps1
915
services:
1016
web:
1117
project: ./code

0 commit comments

Comments
 (0)