Skip to content

Commit ea1cc3b

Browse files
authored
Merge pull request #20388 from DaniBunny/20211006-cu13-docs-fix
CU13 quick docs fix
2 parents d60c344 + e62d415 commit ea1cc3b

3 files changed

Lines changed: 28 additions & 6 deletions

File tree

docs/big-data-cluster/distributed-data-copy-hdfs.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,23 @@ In this guide we will cover the following data copy scenarios:
4545

4646
Certificates are required to create a trusted relationship between source and destination clusters. These steps are required only once per source/destination cluster combination.
4747

48+
> [!IMPORTANT]
49+
> If a SQL Server big data cluster with basic authentication (non-AD) is __upgraded to CU13__, the distcp functionality won't work.
50+
>
51+
> To enable the distcp functionality in this scenario execute the following additional steps once:
52+
>
53+
> ```bash
54+
> kubectl -n $CLUSTER_NAME exec -it nmnode-0-0 -- bash
55+
> export HADOOP_USER_NAME=hdfs
56+
> hadoop fs -mkdir -p /tmp/hadoop-yarn/staging/history
57+
> hadoop fs -chown yarn /tmp/hadoop-yarn
58+
> hadoop fs -chown yarn /tmp/hadoop-yarn/staging
59+
> hadoop fs -chown yarn /tmp/hadoop-yarn/staging/history
60+
> hadoop fs -chmod 733 /tmp/hadoop-yarn
61+
> hadoop fs -chmod 733 /tmp/hadoop-yarn/staging
62+
> hadoop fs -chmod 733 /tmp/hadoop-yarn/staging/history
63+
> ```
64+
4865
The required notebooks in the next steps are part of the Operational notebooks for [!INCLUDE[big-data-clusters-nover](../includes/ssbigdataclusters-ss-nover.md)]. For more information how to install and use the notebooks, see [Operational notebooks](cluster-manage-notebooks.md)
4966
5067
### Step 1 - Certificate creation and installation

docs/big-data-cluster/package-management-delta-lake.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,13 +6,13 @@ author: DaniBunny
66
ms.author: dacoelho
77
ms.reviewer: wiassaf
88
ms.metadata: seo-lt-2019
9-
ms.date: 10/05/2021
9+
ms.date: 10/06/2021
1010
ms.topic: guide
1111
ms.prod: sql
1212
ms.technology: big-data-cluster
1313
---
1414

15-
# Configuring Delta Lake on SQL Server Big Data Clusters
15+
# Delta Lake on SQL Server Big Data Clusters
1616

1717
[!INCLUDE[SQL Server 2019](../includes/applies-to-version/sqlserver2019.md)]
1818

docs/big-data-cluster/release-notes-cumulative-update-13.md

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ description: This article describes the SQL Server Big Data Clusters Cumulative
55
author: WilliamDAssafMSFT
66
ms.author: wiassaf
77
ms.reviewer: melqin,dacoelho
8-
ms.date: 10/05/2021
8+
ms.date: 10/06/2021
99
ms.topic: conceptual
1010
ms.prod: sql
1111
ms.technology: big-data-cluster
@@ -27,8 +27,12 @@ The following release notes apply to [!INCLUDE[big-data-clusters-2019](../includ
2727
> [!CAUTION]
2828
> Before upgrading make sure to review the [Spark 3 upgrade guide](spark-3-upgrade.md).
2929
30+
* Delta Lake 1.0.0 available out-of-the-box. Additional libraries doesn't need to be installed and loaded. Read more at [Delta Lake on SQL Server Big Data Clusters](package-management-delta-lake.md)
31+
* Custom time zone configuration for all services. Read more at [How to configure big data clusters settings post deployment](configure-bdc-postdeployment.md#step-by-step-scenario-configure-timezone-on-)
3032
* [Password rotation for big data cluster autogenerated Active Directory service accounts](active-directory-password-rotation.md)
31-
* [New Advanced Encryption Standard (AES) optional parameter for the automatically generated AD accounts](active-directory-prerequisites.md)
33+
* [New Advanced Encryption Standard (AES) optional parameter for the automatically generated AD accounts](active-directory-deploy.md)
34+
35+
For detailed SQL Server engine changes, check the [official SQL Server CU13 knowledge base article](https://support.microsoft.com/topic/kb5005679-cumulative-update-13-for-sql-server-2019-5c1be850-460a-4be4-a569-fe11f0adc535).
3236

3337
## Tested configurations for CU13
3438

@@ -53,6 +57,7 @@ Reference Architecture White Papers for [!INCLUDE[big-data-clusters-nover](../in
5357
* __R__: Microsoft R 3.5.2
5458
* __Microsoft Spark Runtime 2021.1__
5559
* __Spark__: 3.1.2
60+
* __Delta Lake__: 1.0.0
5661
* __Java__: Azul Zulu JRE 1.8.0_275
5762
* __Scala__: 2.12
5863
* __Python__: 3.8 (miniforge 4.9)
@@ -79,7 +84,7 @@ Reference Architecture White Papers for [!INCLUDE[big-data-clusters-nover](../in
7984
| [Telegraf](https://docs.influxdata.com/telegraf/) | 1.16.1 |
8085
| [ZooKeeper](https://cwiki.apache.org/confluence/display/zookeeper) | 3.5.8 |
8186

82-
## Microsoft Spark Runtime 2021.1 - Installed Python libraries
87+
## Runtime for Apache Spark release 2021.1 (BDC.3.2021.1) - Installed Python libraries
8388

8489
| Library | Version | Library | Version | Library | Version |
8590
|--|--|--|--|--|--|
@@ -213,7 +218,7 @@ Reference Architecture White Papers for [!INCLUDE[big-data-clusters-nover](../in
213218
| pyspark | 3.1.2 | pyparsing | 2.4.7 | | |
214219

215220

216-
## Microsoft Spark Runtime 2021.1 - Installed R libraries
221+
## Runtime for Apache Spark release 2021.1 (BDC.3.2021.1) - Installed R libraries
217222

218223
| Library | Version | Library | Version | Library | Version |
219224
|--|--|--|--|--|--|

0 commit comments

Comments
 (0)