Skip to content

Commit 4eeb055

Browse files
20231204 edit pass, topic type review
1 parent 7560148 commit 4eeb055

9 files changed

Lines changed: 95 additions & 82 deletions

docs/big-data-cluster/active-directory-deployment-aks-tutorial.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -5,26 +5,27 @@ description: Learn how to deploy SQL Server Big Data Clusters in AD mode on Azur
55
author: HugoMSFT
66
ms.author: hudequei
77
ms.reviewer: wiassaf
8-
ms.date: 07/16/2021
8+
ms.date: 12/04/2023
99
ms.service: sql
1010
ms.subservice: big-data-cluster
11-
ms.topic: conceptual
12-
ms.custom: intro-deployment
11+
ms.topic: tutorial
12+
ms.custom:
13+
- intro-deployment
1314
---
1415

1516
# Tutorial: Deploy SQL Server Big Data Clusters in AD mode on Azure Kubernetes Services (AKS)
1617

1718
This article explains how to deploy a SQL Server big data cluster in the Active Directory authentication mode with a reference architecture. The reference architecture extends your on-premises Active Directory domain Service (AD DS) to Azure. You can deploy it from [Azure Architecture Center](https://github.com/mspnp/identity-reference-architectures/tree/master/adds-extend-domain) with [Azure building blocks](https://github.com/mspnp/template-building-blocks/wiki/Install-Azure-Building-Blocks).
1819

19-
[!INCLUDE[big-data-clusters-banner-retirement](../includes/bdc-banner-retirement.md)]
20+
[!INCLUDE [big-data-clusters-banner-retirement](../includes/bdc-banner-retirement.md)]
2021

2122
## Prerequisites
2223

2324
Before deploying a SQL Server big data cluster, you need to:
2425

25-
* Access an Azure VM for management. This VM requires access to the Azure Virtual Network (VNet) where you will deploy the big data cluster. It must either reside on the same VNet, or on [peered VNet](/azure/virtual-network/virtual-network-manage-peering).
26-
* [Install the big data tools](deploy-big-data-tools.md) on the management VM.
27-
* Prepare to deploy the cluster in the [Active Directory authentication mode](active-directory-prerequisites.md) in your on-premises AD controller.
26+
- Access an Azure VM for management. This VM requires access to the Azure Virtual Network (VNet) where you will deploy the big data cluster. It must either reside on the same VNet, or on [peered VNet](/azure/virtual-network/virtual-network-manage-peering).
27+
- [Install the big data tools](deploy-big-data-tools.md) on the management VM.
28+
- Prepare to deploy the cluster in the [Active Directory authentication mode](active-directory-prerequisites.md) in your on-premises AD controller.
2829

2930
## Create AKS subnet
3031

@@ -50,7 +51,7 @@ Before deploying a SQL Server big data cluster, you need to:
5051

5152
The following screenshot shows how we plan the subnets resides in the VNet in the architecture.
5253

53-
:::image type="content" source="media/active-directory-deployment-aks/ad-in-aks-diagram.png" alt-text="AKS cluster with AD and SQL Server Big Data Cluster":::
54+
:::image type="content" source="media/active-directory-deployment-aks/ad-in-aks-diagram.png" alt-text="Diagram of the AKS cluster with AD and SQL Server Big Data Cluster.":::
5455

5556
## Create an AKS private cluster
5657

@@ -130,8 +131,7 @@ The following command initiates a big data cluster deployment:
130131
azdata bdc create --config-profile bdc-ad-aks --accept-eula yes
131132
```
132133

133-
## Next steps
134+
## Related content
134135

135-
[Connect to a SQL Server big data cluster with Azure Data Studio](connect-to-big-data-cluster.md)
136-
137-
[Tutorial: Load sample data into a SQL Server big data cluster](tutorial-load-sample-data.md)
136+
- [Connect to a SQL Server big data cluster with Azure Data Studio](connect-to-big-data-cluster.md)
137+
- [Tutorial: Load sample data into a SQL Server big data cluster](tutorial-load-sample-data.md)

docs/big-data-cluster/app-create.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,18 +5,19 @@ description: Deploy a Python or R script as an application on SQL Server 2019 bi
55
author: HugoMSFT
66
ms.author: hudequei
77
ms.reviewer: wiassaf
8-
ms.date: 04/13/2021
8+
ms.date: 12/04/2023
99
ms.service: sql
1010
ms.subservice: big-data-cluster
11-
ms.topic: conceptual
12-
ms.custom: intro-deployment
11+
ms.topic: how-to
12+
ms.custom:
13+
- intro-deployment
1314
---
1415

1516
# How to deploy an app on SQL Server Big Data Clusters
1617

17-
[!INCLUDE[SQL Server 2019](../includes/applies-to-version/sqlserver2019.md)]
18+
[!INCLUDE [SQL Server 2019](../includes/applies-to-version/sqlserver2019.md)]
1819

19-
[!INCLUDE[big-data-clusters-banner-retirement](../includes/bdc-banner-retirement.md)]
20+
[!INCLUDE [big-data-clusters-banner-retirement](../includes/bdc-banner-retirement.md)]
2021

2122
Applications deployed on SQL Server Big Data Clusters not only benefit from many advantages such as the computational power of the cluster but also access massive data that is available on the cluster. It dramatically improve the performance since your app sits in the same cluster where the data lives.
2223

@@ -218,7 +219,7 @@ To delete an app from your big data cluster, use the following syntax:
218219
azdata app delete --name add-app --version v1
219220
```
220221

221-
## Next steps
222+
## Related content
222223

223224
Explore how to integrate apps deployed on [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ss-nover.md)] in your own applications at [Run applications on big data clusters](app-run.md) and [Consume applications on big data clusters](app-consume.md) for more information. You can also check out additional samples at [App Deploy Samples](https://aka.ms/sql-app-deploy).
224225

docs/big-data-cluster/app-run.md

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,17 +5,17 @@ description: Running applications with azdata on SQL Server 2019 big data cluste
55
author: HugoMSFT
66
ms.author: hudequei
77
ms.reviewer: wiassaf
8-
ms.date: 08/16/2020
8+
ms.date: 12/04/2023
99
ms.service: sql
1010
ms.subservice: big-data-cluster
11-
ms.topic: conceptual
11+
ms.topic: how-to
1212
---
1313

1414
# Run apps with azdata - SQL Server Big Data Clusters
1515

1616
This article describes how to run an application inside a SQL Server Big Data Clusters.
1717

18-
[!INCLUDE[big-data-clusters-banner-retirement](../includes/bdc-banner-retirement.md)]
18+
[!INCLUDE [big-data-clusters-banner-retirement](../includes/bdc-banner-retirement.md)]
1919

2020
## Prerequisites
2121

@@ -67,7 +67,7 @@ If the run was successful, you should see your output as specified when you crea
6767

6868
## Describe an app
6969

70-
The describe command provides detailed information about the app including the end point in your cluster. This is typically used by an app developer to build an app using the swagger client and using the webservice to interact with the app in a RESTful manner. For more information, see [Consume applications on big data clusters](app-consume.md) for more information.
70+
The describe command provides detailed information about the app including the endpoint in your cluster. This is typically used by an app developer to build an app using the swagger client and using the webservice to interact with the app in a RESTful manner. For more information, see [Consume applications on big data clusters](app-consume.md) for more information.
7171

7272
```json
7373
{
@@ -97,6 +97,8 @@ The describe command provides detailed information about the app including the e
9797
}
9898
```
9999

100-
Explore how to integrate apps deployed on [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ss-nover.md)] in your own applications at [Consume applications on big data clusters](app-consume.md) for more information. You can also check out additional samples at [App Deploy Samples](https://aka.ms/sql-app-deploy).
100+
## Related content
101101

102-
For more information about [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ss-nover.md)], see [Introducing [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ver15.md)]](big-data-cluster-overview.md).
102+
Explore how to integrate apps deployed on [!INCLUDE [big-data-clusters-2019](../includes/ssbigdataclusters-ss-nover.md)] in your own applications at [Consume applications on big data clusters](app-consume.md) for more information. You can also check out additional samples at [App Deploy Samples](https://aka.ms/sql-app-deploy).
103+
104+
For more information about [!INCLUDE [big-data-clusters-2019](../includes/ssbigdataclusters-ss-nover.md)], see [Introducing [!INCLUDE [big-data-clusters-2019](../includes/ssbigdataclusters-ver15.md)]](big-data-cluster-overview.md).

docs/big-data-cluster/cluster-logging-kibana.md

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -5,39 +5,39 @@ description: Monitoring cluster with Kibana Dashboard on SQL Server 2019 big dat
55
author: HugoMSFT
66
ms.author: hudequei
77
ms.reviewer: wiassaf
8-
ms.date: 07/12/2021
8+
ms.date: 12/04/2023
99
ms.service: sql
1010
ms.subservice: big-data-cluster
11-
ms.topic: conceptual
11+
ms.topic: how-to
1212
---
1313

1414
# Review cluster logs with Kibana Dashboard
1515

16-
This article describes how to visualize the logs an application inside [!INCLUDE[ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)]. [!INCLUDE[ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)] use Fluent Bit, an open-source log processor and forwarder. Fluent Bit fetches the logs from big data cluster components in the cluster and stores them in [Elastic Stack Elasticsearch](https://azure.microsoft.com/overview/linux-on-azure/elastic/). From Kibana Dashboard, you can visualize and search the log of your interest.
16+
This article describes how to visualize the logs an application inside [!INCLUDE [ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)]. [!INCLUDE [ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)] use Fluent Bit, an open-source log processor and forwarder. Fluent Bit fetches the logs from big data cluster components in the cluster and stores them in [Elastic Stack Elasticsearch](https://azure.microsoft.com/overview/linux-on-azure/elastic/). From Kibana Dashboard, you can visualize and search the log of your interest.
1717

18-
[!INCLUDE[big-data-clusters-banner-retirement](../includes/bdc-banner-retirement.md)]
18+
[!INCLUDE [big-data-clusters-banner-retirement](../includes/bdc-banner-retirement.md)]
1919

2020
## Logs stored in Elasticsearch
2121

2222
Big Data Cluster-related logs stored in Elasticsearch includes the standard output and error logs of all services, including SQL Server, Spark, HDFS, and platform services.
2323

2424
Those logs can be searched by components From Kibana Dashboard. You can use filters such as 'kubernetes_container_name', 'kubernetes_pod_name', 'log_filename' and 'service_name' to help you quickly visualize all the logs such as logs from Big Data Clusters controller, from SQL Server, or any logs from different pods, services, and more.
2525

26-
Specifically, the controller log records the status and process of the cluster deployments and cluster events by filtering 'service_name: controller'. For those who look into [!INCLUDE[ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)] in AD mode, you may find the security-support log to be quite useful, it records the events during the process Big Data Cluster obtains AD tokens from the on-premises Active Directory(AD) domain controller, you can access it by filtering 'service_name: secsupp' under the controller log.
26+
Specifically, the controller log records the status and process of the cluster deployments and cluster events by filtering 'service_name: controller'. For those who look into [!INCLUDE [ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)] in AD mode, you might find the security-support log to be quite useful, it records the events during the process Big Data Cluster obtains AD tokens from the on-premises Active Directory(AD) domain controller, you can access it by filtering 'service_name: secsupp' under the controller log.
2727

2828

2929
## Prerequisites
3030

31-
- [[!INCLUDE[ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)]](deployment-guidance.md)
31+
- [[!INCLUDE [ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)]](deployment-guidance.md)
3232
- [azdata command-line utility](../azdata/install/deploy-install-azdata.md)
3333

3434
## Capabilities
3535

36-
In [!INCLUDE[sssql19-md](../includes/sssql19-md.md)] you can create, delete, describe, initialize, list run and update your application. The following table describes the application deployment commands that you can use with `azdata`.
36+
In [!INCLUDE [sssql19-md](../includes/sssql19-md.md)] you can create, delete, describe, initialize, list run and update your application. The following table describes the application deployment commands that you can use with `azdata`.
3737

3838
|Command |Description |
3939
|:---|:---|
40-
|`azdata bdc endpoint list` | Lists the endpoints for the [!INCLUDE[ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)]. |
40+
|`azdata bdc endpoint list` | Lists the endpoints for the [!INCLUDE [ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)]. |
4141

4242

4343
You can use the following example to list the endpoint of **Kibana Dashboard**:
@@ -48,16 +48,16 @@ azdata bdc endpoint list --endpoint-name logsui
4848

4949
The output will give you the endpoint, which you can use your cluster username and password to sign in.
5050

51-
![Kibana Dashboard](media/big-data-cluster-monitor-cluster/kibana-dashboard-endpoint.png)
51+
:::image type="content" source="media/big-data-cluster-monitor-cluster/kibana-dashboard-endpoint.png" alt-text="A screenshot of the text of a dashboard endpoint.":::
5252

5353

5454
The link to a Kibana dashboard:
5555

56-
![Kibana dashboard](media/view-cluster-status/kibana-dashboard.png)
56+
:::image type="content" source="media/view-cluster-status/kibana-dashboard.png" alt-text="A screenshot of the link to the Kibana dashboard.":::
5757

5858
> [!IMPORTANT]
5959
> The Internet Explorer browser and older Microsoft Edge browsers are not compatible with Kibana. You will see a blank page when loading the dashboards using an unsupported browser. Consider the [Chromium-based Microsoft Edge](https://microsoftedgewelcome.microsoft.com/), or review [supported browsers for Kibana](https://www.elastic.co/support/matrix#matrix_browsers).
6060
61-
## Next steps
61+
## Related content
6262

6363
For more information about [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ss-nover.md)], see [Introducing [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ver15.md)]](big-data-cluster-overview.md).

docs/big-data-cluster/cluster-logging-notebooks.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,4 +64,7 @@ A set of notebooks to gather and analyze logs from a SQL Server big data cluster
6464

6565
## Next steps
6666

67-
For more information about [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ss-nover.md)], see [Introducing [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ver15.md)]](big-data-cluster-overview.md).
67+
- [Monitor Big Data Clusters status by using Azure Data Studio](cluster-monitor-ads.md)
68+
- [Monitor Big Data Clusters by using azdata and Grafana Dashboard](cluster-monitor-grafana.md)
69+
- [Monitor Big Data Clusters by using azdata and kubectl](cluster-monitor-cmdlet.md)
70+
- [Monitor Big Data Clusters by using Jupyter Notebooks and Azure Data Studio](cluster-monitor-notebooks.md)

0 commit comments

Comments
 (0)