You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/big-data-cluster/active-directory-deployment-aks-tutorial.md
+12-12Lines changed: 12 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,26 +5,27 @@ description: Learn how to deploy SQL Server Big Data Clusters in AD mode on Azur
5
5
author: HugoMSFT
6
6
ms.author: hudequei
7
7
ms.reviewer: wiassaf
8
-
ms.date: 07/16/2021
8
+
ms.date: 12/04/2023
9
9
ms.service: sql
10
10
ms.subservice: big-data-cluster
11
-
ms.topic: conceptual
12
-
ms.custom: intro-deployment
11
+
ms.topic: tutorial
12
+
ms.custom:
13
+
- intro-deployment
13
14
---
14
15
15
16
# Tutorial: Deploy SQL Server Big Data Clusters in AD mode on Azure Kubernetes Services (AKS)
16
17
17
18
This article explains how to deploy a SQL Server big data cluster in the Active Directory authentication mode with a reference architecture. The reference architecture extends your on-premises Active Directory domain Service (AD DS) to Azure. You can deploy it from [Azure Architecture Center](https://github.com/mspnp/identity-reference-architectures/tree/master/adds-extend-domain) with [Azure building blocks](https://github.com/mspnp/template-building-blocks/wiki/Install-Azure-Building-Blocks).
Before deploying a SQL Server big data cluster, you need to:
24
25
25
-
* Access an Azure VM for management. This VM requires access to the Azure Virtual Network (VNet) where you will deploy the big data cluster. It must either reside on the same VNet, or on [peered VNet](/azure/virtual-network/virtual-network-manage-peering).
26
-
*[Install the big data tools](deploy-big-data-tools.md) on the management VM.
27
-
* Prepare to deploy the cluster in the [Active Directory authentication mode](active-directory-prerequisites.md) in your on-premises AD controller.
26
+
- Access an Azure VM for management. This VM requires access to the Azure Virtual Network (VNet) where you will deploy the big data cluster. It must either reside on the same VNet, or on [peered VNet](/azure/virtual-network/virtual-network-manage-peering).
27
+
-[Install the big data tools](deploy-big-data-tools.md) on the management VM.
28
+
- Prepare to deploy the cluster in the [Active Directory authentication mode](active-directory-prerequisites.md) in your on-premises AD controller.
28
29
29
30
## Create AKS subnet
30
31
@@ -50,7 +51,7 @@ Before deploying a SQL Server big data cluster, you need to:
50
51
51
52
The following screenshot shows how we plan the subnets resides in the VNet in the architecture.
52
53
53
-
:::image type="content" source="media/active-directory-deployment-aks/ad-in-aks-diagram.png" alt-text="AKS cluster with AD and SQL Server Big Data Cluster":::
54
+
:::image type="content" source="media/active-directory-deployment-aks/ad-in-aks-diagram.png" alt-text="Diagram of the AKS cluster with AD and SQL Server Big Data Cluster.":::
54
55
55
56
## Create an AKS private cluster
56
57
@@ -130,8 +131,7 @@ The following command initiates a big data cluster deployment:
Applications deployed on SQL Server Big Data Clusters not only benefit from many advantages such as the computational power of the cluster but also access massive data that is available on the cluster. It dramatically improve the performance since your app sits in the same cluster where the data lives.
22
23
@@ -218,7 +219,7 @@ To delete an app from your big data cluster, use the following syntax:
218
219
azdata app delete --name add-app --version v1
219
220
```
220
221
221
-
## Next steps
222
+
## Related content
222
223
223
224
Explore how to integrate apps deployed on [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ss-nover.md)] in your own applications at [Run applications on big data clusters](app-run.md) and [Consume applications on big data clusters](app-consume.md) for more information. You can also check out additional samples at [App Deploy Samples](https://aka.ms/sql-app-deploy).
@@ -67,7 +67,7 @@ If the run was successful, you should see your output as specified when you crea
67
67
68
68
## Describe an app
69
69
70
-
The describe command provides detailed information about the app including the end point in your cluster. This is typically used by an app developer to build an app using the swagger client and using the webservice to interact with the app in a RESTful manner. For more information, see [Consume applications on big data clusters](app-consume.md) for more information.
70
+
The describe command provides detailed information about the app including the endpoint in your cluster. This is typically used by an app developer to build an app using the swagger client and using the webservice to interact with the app in a RESTful manner. For more information, see [Consume applications on big data clusters](app-consume.md) for more information.
71
71
72
72
```json
73
73
{
@@ -97,6 +97,8 @@ The describe command provides detailed information about the app including the e
97
97
}
98
98
```
99
99
100
-
Explore how to integrate apps deployed on [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ss-nover.md)] in your own applications at [Consume applications on big data clusters](app-consume.md) for more information. You can also check out additional samples at [App Deploy Samples](https://aka.ms/sql-app-deploy).
100
+
## Related content
101
101
102
-
For more information about [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ss-nover.md)], see [Introducing [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ver15.md)]](big-data-cluster-overview.md).
102
+
Explore how to integrate apps deployed on [!INCLUDE [big-data-clusters-2019](../includes/ssbigdataclusters-ss-nover.md)] in your own applications at [Consume applications on big data clusters](app-consume.md) for more information. You can also check out additional samples at [App Deploy Samples](https://aka.ms/sql-app-deploy).
103
+
104
+
For more information about [!INCLUDE [big-data-clusters-2019](../includes/ssbigdataclusters-ss-nover.md)], see [Introducing [!INCLUDE [big-data-clusters-2019](../includes/ssbigdataclusters-ver15.md)]](big-data-cluster-overview.md).
Copy file name to clipboardExpand all lines: docs/big-data-cluster/cluster-logging-kibana.md
+11-11Lines changed: 11 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,39 +5,39 @@ description: Monitoring cluster with Kibana Dashboard on SQL Server 2019 big dat
5
5
author: HugoMSFT
6
6
ms.author: hudequei
7
7
ms.reviewer: wiassaf
8
-
ms.date: 07/12/2021
8
+
ms.date: 12/04/2023
9
9
ms.service: sql
10
10
ms.subservice: big-data-cluster
11
-
ms.topic: conceptual
11
+
ms.topic: how-to
12
12
---
13
13
14
14
# Review cluster logs with Kibana Dashboard
15
15
16
-
This article describes how to visualize the logs an application inside [!INCLUDE[ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)]. [!INCLUDE[ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)] use Fluent Bit, an open-source log processor and forwarder. Fluent Bit fetches the logs from big data cluster components in the cluster and stores them in [Elastic Stack Elasticsearch](https://azure.microsoft.com/overview/linux-on-azure/elastic/). From Kibana Dashboard, you can visualize and search the log of your interest.
16
+
This article describes how to visualize the logs an application inside [!INCLUDE[ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)]. [!INCLUDE[ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)] use Fluent Bit, an open-source log processor and forwarder. Fluent Bit fetches the logs from big data cluster components in the cluster and stores them in [Elastic Stack Elasticsearch](https://azure.microsoft.com/overview/linux-on-azure/elastic/). From Kibana Dashboard, you can visualize and search the log of your interest.
Big Data Cluster-related logs stored in Elasticsearch includes the standard output and error logs of all services, including SQL Server, Spark, HDFS, and platform services.
23
23
24
24
Those logs can be searched by components From Kibana Dashboard. You can use filters such as 'kubernetes_container_name', 'kubernetes_pod_name', 'log_filename' and 'service_name' to help you quickly visualize all the logs such as logs from Big Data Clusters controller, from SQL Server, or any logs from different pods, services, and more.
25
25
26
-
Specifically, the controller log records the status and process of the cluster deployments and cluster events by filtering 'service_name: controller'. For those who look into [!INCLUDE[ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)] in AD mode, you may find the security-support log to be quite useful, it records the events during the process Big Data Cluster obtains AD tokens from the on-premises Active Directory(AD) domain controller, you can access it by filtering 'service_name: secsupp' under the controller log.
26
+
Specifically, the controller log records the status and process of the cluster deployments and cluster events by filtering 'service_name: controller'. For those who look into [!INCLUDE[ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)] in AD mode, you might find the security-support log to be quite useful, it records the events during the process Big Data Cluster obtains AD tokens from the on-premises Active Directory(AD) domain controller, you can access it by filtering 'service_name: secsupp' under the controller log.
In [!INCLUDE[sssql19-md](../includes/sssql19-md.md)] you can create, delete, describe, initialize, list run and update your application. The following table describes the application deployment commands that you can use with `azdata`.
36
+
In [!INCLUDE[sssql19-md](../includes/sssql19-md.md)] you can create, delete, describe, initialize, list run and update your application. The following table describes the application deployment commands that you can use with `azdata`.
37
37
38
38
|Command |Description |
39
39
|:---|:---|
40
-
|`azdata bdc endpoint list`| Lists the endpoints for the [!INCLUDE[ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)]. |
40
+
|`azdata bdc endpoint list`| Lists the endpoints for the [!INCLUDE[ssbigdataclusters-ss-nover](../includes/ssbigdataclusters-ss-nover.md)]. |
41
41
42
42
43
43
You can use the following example to list the endpoint of **Kibana Dashboard**:
@@ -48,16 +48,16 @@ azdata bdc endpoint list --endpoint-name logsui
48
48
49
49
The output will give you the endpoint, which you can use your cluster username and password to sign in.
:::image type="content" source="media/big-data-cluster-monitor-cluster/kibana-dashboard-endpoint.png" alt-text="A screenshot of the text of a dashboard endpoint.":::
:::image type="content" source="media/view-cluster-status/kibana-dashboard.png" alt-text="A screenshot of the link to the Kibana dashboard.":::
57
57
58
58
> [!IMPORTANT]
59
59
> The Internet Explorer browser and older Microsoft Edge browsers are not compatible with Kibana. You will see a blank page when loading the dashboards using an unsupported browser. Consider the [Chromium-based Microsoft Edge](https://microsoftedgewelcome.microsoft.com/), or review [supported browsers for Kibana](https://www.elastic.co/support/matrix#matrix_browsers).
60
60
61
-
## Next steps
61
+
## Related content
62
62
63
63
For more information about [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ss-nover.md)], see [Introducing [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ver15.md)]](big-data-cluster-overview.md).
Copy file name to clipboardExpand all lines: docs/big-data-cluster/cluster-logging-notebooks.md
+4-1Lines changed: 4 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -64,4 +64,7 @@ A set of notebooks to gather and analyze logs from a SQL Server big data cluster
64
64
65
65
## Next steps
66
66
67
-
For more information about [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ss-nover.md)], see [Introducing [!INCLUDE[big-data-clusters-2019](../includes/ssbigdataclusters-ver15.md)]](big-data-cluster-overview.md).
67
+
-[Monitor Big Data Clusters status by using Azure Data Studio](cluster-monitor-ads.md)
68
+
-[Monitor Big Data Clusters by using azdata and Grafana Dashboard](cluster-monitor-grafana.md)
69
+
-[Monitor Big Data Clusters by using azdata and kubectl](cluster-monitor-cmdlet.md)
70
+
-[Monitor Big Data Clusters by using Jupyter Notebooks and Azure Data Studio](cluster-monitor-notebooks.md)
0 commit comments