You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/relational-databases/polybase/polybase-faq.yml
+44-23Lines changed: 44 additions & 23 deletions
Original file line number
Diff line number
Diff line change
@@ -1,8 +1,8 @@
1
1
### YamlMime:FAQ
2
2
metadata:
3
3
title: "Frequently Asked Questions in PolyBase"
4
-
description: Compare PolyBase and linked servers and compare PolyBase in Big Data Clusters and PolyBase in stand-alone instances. Find out what's new in PolyBase 2019.
5
-
ms.date: 08/13/2021
4
+
description: Frequently asked questions about PolyBase. Compare PolyBase and linked servers and compare PolyBase in Big Data Clusters and PolyBase in stand-alone instances. Find out what's new in PolyBase.
5
+
ms.date: 11/03/2022
6
6
ms.prod: sql
7
7
ms.technology: polybase
8
8
ms.topic: conceptual
@@ -19,23 +19,30 @@ summary: |
19
19
sections:
20
20
- name: Ignored
21
21
questions:
22
-
- question: PolyBase VS. linked servers
22
+
- question: PolyBase vs. linked servers
23
23
answer: |
24
24
The following table highlights the differences between PolyBase and linked server features:
|Supports read-only operations for all data sources and insert operation for HADOOP & data pool data source only|Supports both read and write operations|
30
+
|Supports read-only operations for all data sources and write to some external data sources|Supports both read and write operations|
31
31
|Queries to remote data source from a single connection can be scaled-out |Queries to remote data source from a single connection cannot be scaled-out|
32
32
|Predicates pushdown is supported|Predicates pushdown is supported|
33
33
|No separate configuration needed for availability group|Separate configuration needed for each instance in availability group|
|Suitable for analytic queries processing large number of rows|Suitable for OLTP queries returning single or few rows|
36
36
|Queries using external table cannot participate in distributed transaction|Distributed queries can participate in distributed transaction|
37
+
- question: What's new in PolyBase in [!INCLUDE[sssql22](../../includes/sssql22-md.md)]?
38
+
answer: |
39
+
[!INCLUDE[sssql22](../../includes/sssql22-md.md)] now supports CSV, Parquet, and Delta files stored on Azure Storage Account v2, Azure Data Lake Storage Gen2, or any S3–compliant object storage using the S3 REST API, on-premises or in the cloud.
40
+
41
+
[!INCLUDE[sssql22](../../includes/sssql22-md.md)] can now use CREATE EXTERNAL TABLE as SELECT (CETAS), together with commands like OPENROWSET, CREATE EXTERNAL TABLE (CET), and all the new T-SQL enhancements.
37
42
38
-
- question: What's new in PolyBase 2019?
43
+
Read more on [Data Virtualization with PolyBase for SQL Server 2022](https://cloudblogs.microsoft.com/sqlserver/2022/10/05/data-virtualization-with-polybase-for-sql-server-2022/) and review [SQL Server 2022 PolyBase enhancements](polybase-guide.md#sql-server-2022-polybase-enhancements).
44
+
45
+
- question: What's new in PolyBase in [!INCLUDE[sssql19](../../includes/sssql19-md.md)]?
39
46
answer: |
40
47
PolyBase in [!INCLUDE[sssql19](../../includes/sssql19-md.md)] can now read data from a larger variety of data sources. The data from these external data sources can be stored as external tables on your SQL Server. PolyBase also supports pushdown computation to these external data sources, excluding ODBC generic types.
41
48
@@ -48,24 +55,11 @@ sections:
48
55
- Compatible ODBC generic types
49
56
50
57
> [!NOTE]
51
-
> PolyBase can allow connection to external data sources using thirdparty ODBC drivers. These drivers are not provided along with PolyBase and may not work as intended. For more information, visit our [guide](../../relational-databases/polybase/polybase-configure-odbc-generic.md) for PolyBase ODBC generic configuration.
52
-
53
-
- question: PolyBase in Big Data Clusters vs. PolyBase in stand-alone instances
58
+
> PolyBase can allow connection to external data sources using third-party ODBC drivers. These drivers are not provided along with PolyBase and may not work as intended. For more information, see [Configure PolyBase to access external data with ODBC generic types](../../relational-databases/polybase/polybase-configure-odbc-generic.md).
59
+
60
+
- question: What CONNECTION_OPTIONS can I specify for connection to third-party data sources?
54
61
answer: |
55
-
The following table highlights the PolyBase features available in [!INCLUDE[sssql19](../../includes/sssql19-md.md)] stand-alone install and [!INCLUDE[sssql19](../../includes/sssql19-md.md)] big data cluster:
|Create external data source for SQL Server, Oracle, Teradata, and Mongo DB |X|X|
60
-
|Create external data source using a compatible third-party ODBC Driver | | X|
61
-
|Create external data source for HADOOP data source | X| X|
62
-
|Create external data source for Azure Blob Storage | X| X|
63
-
|Create external table on a SQL Server data pool | X| |
64
-
|Create external table on a SQL Server storage pool | X| |
65
-
|Scale-out query execution | X| X (Windows only) |
66
-
67
-
> [!NOTE]
68
-
> For more information on connections using the ODBC generic connector visit our [How to guide for configuring ODBC generic types](polybase-configure-odbc-generic.md).
62
+
Review [CREATE EXTERNAL DATA SOURCE (Transact-SQL) CONNECTION_OPTIONS](../../t-sql/statements/create-external-data-source-connection-options.md) for supported data provider connection options for SQL Server, Oracle, Teradata, the MongoDB API for Cosmos DB, and generic ODBC data providers.
69
63
70
64
- question: How can I view the remote query execution plan of PolyBase pushdown computation?
71
65
answer: |
@@ -75,6 +69,14 @@ sections:
75
69
answer: |
76
70
See [How to tell if external pushdown occurred](polybase-how-to-tell-pushdown-computation.md).
77
71
72
+
- question: Is PolyBase supported on Azure SQL Database?
73
+
answer: |
74
+
No, but you can query data in the files placed on Azure Blob Storage using OPENROWSET function, or use CREATE EXTERNAL TABLE to read from a serverless SQL pool in Synapse Analytics. For more information, see [Azure SQL can read Azure Data Lake storage files using Synapse SQL external tables](https://devblogs.microsoft.com/azure-sql/read-azure-storage-files-using-synapse-sql-external-tables/).
75
+
76
+
- question: Is PolyBase supported on Azure SQL Managed Instance?
77
+
answer: |
78
+
Azure SQL Managed Instance has its own data virtualization capabilities for Azure Data Lake Storage (ADLS) and Azure Blob Storage as data sources. For more information, see [Data virtualization with Azure SQL Managed Instance](/azure/azure-sql/managed-instance/data-virtualization-overview).
79
+
78
80
- question: Which ports should I allow through my firewall for PolyBase?
79
81
answer: |
80
82
No additional ports are needed to open for PolyBase itself, except in the case of a PolyBase scale-out group. Without a scale-out group, the PolyBase service alone should not require any firewall rule.
@@ -90,12 +92,31 @@ sections:
90
92
| Microsoft SQL Server | [Ports Used By SQL Server](../../sql-server/install/configure-the-windows-firewall-to-allow-sql-server-access.md#ports-used-by-sql-server)|
| Cloudera CDH | [Ports Used by CDH Components](https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/cdh_ports.html)|
93
-
| | |
94
95
95
96
- question: What ports are used for PolyBase scale-out group?
96
97
answer: |
97
98
By default, the PolyBase data movement service connects to the head node of a scale-out group over a range of ports from TCP 16450-16460, typically using 16450-16453, plus port 17001. These ports are only used when a PolyBase scale-out group is configured and should be allowed through server firewalls. SQL Server Setup should automatically create a local firewall rule allowing these ports upon feature installation. For more information, see [Configure PolyBase scale-out groups on Windows](configure-scale-out-groups-windows.md).
99
+
100
+
> [!NOTE]
101
+
> The Microsoft SQL Server PolyBase scale-out groups will be retired. Scale-out group functionality will be removed from the product in [!INCLUDE[sssql22](../../includes/sssql22-md.md)].
102
+
103
+
- question: PolyBase in Big Data Clusters vs. PolyBase in stand-alone instances
104
+
answer: |
105
+
For more information on connections using the ODBC generic connector, visit our [How to guide for configuring ODBC generic types](polybase-configure-odbc-generic.md).
106
+
107
+
The following table highlights the PolyBase features available in [!INCLUDE[sssql19](../../includes/sssql19-md.md)] stand-alone install and [!INCLUDE[sssql19](../../includes/sssql19-md.md)] big data cluster:
0 commit comments