Skip to content

Commit d965a61

Browse files
committed
Refresh Format files article and address multiple issues
1 parent 6ee3b96 commit d965a61

3 files changed

Lines changed: 138 additions & 143 deletions

File tree

docs/relational-databases/import-export/examples-of-bulk-access-to-data-in-azure-blob-storage.md

Lines changed: 29 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ title: "Bulk access to data in Azure Blob Storage"
33
description: "Transact-SQL examples that use BULK INSERT and OPENROWSET to access data in an Azure Blob Storage account."
44
author: rwestMSFT
55
ms.author: randolphwest
6-
ms.date: 10/04/2022
6+
ms.date: 12/20/2023
77
ms.service: sql
88
ms.subservice: data-movement
99
ms.topic: conceptual
@@ -12,16 +12,17 @@ helpviewer_keywords:
1212
- "Azure Blob Storage, bulk import to SQL Server"
1313
- "BULK INSERT, Azure Blob Storage"
1414
- "OPENROWSET, Azure Blob Storage"
15-
monikerRange: ">=sql-server-2017||>=sql-server-linux-2017||=azuresqldb-mi-current"
15+
monikerRange: ">=sql-server-2017 || >=sql-server-linux-2017 || =azuresqldb-mi-current"
1616
---
1717
# Examples of bulk access to data in Azure Blob Storage
1818

19-
[!INCLUDE[sqlserver2017-asdb](../../includes/applies-to-version/sqlserver2017-asdb.md)]
19+
[!INCLUDE [sqlserver2017-asdb](../../includes/applies-to-version/sqlserver2017-asdb.md)]
2020

2121
The `BULK INSERT` and `OPENROWSET` statements can directly access a file in Azure Blob Storage. The following examples use data from a CSV (comma separated value) file (named `inv-2017-01-19.csv`), stored in a container (named `Week3`), stored in a storage account (named `newinvoices`).
2222

23-
> [!IMPORTANT]
24-
> All the paths to the container and to the files on Blob Storage are **case-sensitive**. If not correct, it might return an error like "Cannot bulk load. The file "file.csv" does not exist or you don't have file access rights."
23+
All the paths to the container and to the files on Blob Storage are **case-sensitive**. If not correct, it might return an error like the following example:
24+
25+
> Cannot bulk load. The file "file.csv" does not exist or you don't have file access rights.
2526
2627
## Create the credential
2728

@@ -34,12 +35,12 @@ For example:
3435
```sql
3536
CREATE DATABASE SCOPED CREDENTIAL UploadInvoices
3637
WITH IDENTITY = 'SHARED ACCESS SIGNATURE',
37-
SECRET = 'sv=2018-03-28&ss=b&srt=sco&sp=rwdlac&se=2019-08-31T02:25:19Z&st=2019-07-30T18:25:19Z&spr=https&sig=KS51p%2BVnfUtLjMZtUTW1siyuyd2nlx294tL0mnmFsOk%3D';
38+
SECRET = 'sv=2018-03-28&ss=b&srt=sco&sp=rwdlac&se=2019-08-31T02:25:19Z&st=2019-07-30T18:25:19Z&spr=https&sig=KS51p%2BVnfUtLjMZtUTW1siyuyd2nlx294tL0mnmFsOk%3D';
3839
```
3940

4041
## Known issues
4142

42-
Requests from Azure SQL Database and Azure SQL Managed Instance using SAS tokens may be blocked with the following error:
43+
Requests from Azure SQL Database and Azure SQL Managed Instance using SAS tokens might be blocked with the following error:
4344

4445
```text
4546
Msg 4861, Level 16, State 1, Line 27
@@ -58,11 +59,11 @@ The following example uses an external data source pointing to an Azure storage
5859

5960
```sql
6061
CREATE EXTERNAL DATA SOURCE MyAzureInvoices
61-
WITH (
62-
TYPE = BLOB_STORAGE,
63-
LOCATION = 'https://newinvoices.blob.core.windows.net',
64-
CREDENTIAL = UploadInvoices
65-
);
62+
WITH (
63+
TYPE = BLOB_STORAGE,
64+
LOCATION = 'https://newinvoices.blob.core.windows.net',
65+
CREDENTIAL = UploadInvoices
66+
);
6667
```
6768

6869
Then the `OPENROWSET` statement adds the container name (`week3`) to the file description. The file is named `inv-2017-01-19.csv`.
@@ -72,18 +73,17 @@ SELECT * FROM OPENROWSET(
7273
BULK 'week3/inv-2017-01-19.csv',
7374
DATA_SOURCE = 'MyAzureInvoices',
7475
FORMAT = 'CSV',
75-
FORMATFILE='invoices.fmt',
76+
FORMATFILE = 'invoices.fmt',
7677
FORMATFILE_DATA_SOURCE = 'MyAzureInvoices'
77-
) AS DataFile;
78+
) AS DataFile;
7879
```
7980

8081
Using `BULK INSERT`, use the container and file description:
8182

8283
```sql
8384
BULK INSERT Colors2
8485
FROM 'week3/inv-2017-01-19.csv'
85-
WITH (DATA_SOURCE = 'MyAzureInvoices',
86-
FORMAT = 'CSV');
86+
WITH (DATA_SOURCE = 'MyAzureInvoices', FORMAT = 'CSV');
8787
```
8888

8989
### Access data in a CSV file referencing a container in an Azure Blob Storage location
@@ -92,11 +92,11 @@ The following example uses an external data source pointing to a container (name
9292

9393
```sql
9494
CREATE EXTERNAL DATA SOURCE MyAzureInvoicesContainer
95-
WITH (
96-
TYPE = BLOB_STORAGE,
97-
LOCATION = 'https://newinvoices.blob.core.windows.net/week3',
98-
CREDENTIAL = UploadInvoices
99-
);
95+
WITH (
96+
TYPE = BLOB_STORAGE,
97+
LOCATION = 'https://newinvoices.blob.core.windows.net/week3',
98+
CREDENTIAL = UploadInvoices
99+
);
100100
```
101101

102102
Then the `OPENROWSET` statement doesn't include the container name in the file description:
@@ -106,23 +106,22 @@ SELECT * FROM OPENROWSET(
106106
BULK 'inv-2017-01-19.csv',
107107
DATA_SOURCE = 'MyAzureInvoicesContainer',
108108
FORMAT = 'CSV',
109-
FORMATFILE='invoices.fmt',
109+
FORMATFILE = 'invoices.fmt',
110110
FORMATFILE_DATA_SOURCE = 'MyAzureInvoices'
111-
) AS DataFile;
111+
) AS DataFile;
112112
```
113113

114114
Using `BULK INSERT`, don't use the container name in the file description:
115115

116116
```sql
117117
BULK INSERT Colors2
118118
FROM 'inv-2017-01-19.csv'
119-
WITH (DATA_SOURCE = 'MyAzureInvoicesContainer',
120-
FORMAT = 'CSV');
119+
WITH (DATA_SOURCE = 'MyAzureInvoicesContainer', FORMAT = 'CSV');
121120
```
122121

123-
## See also
122+
## Related content
124123

125-
- [CREATE DATABASE SCOPED CREDENTIAL](../../t-sql/statements/create-database-scoped-credential-transact-sql.md)
126-
- [CREATE EXTERNAL DATA SOURCE](../../t-sql/statements/create-external-data-source-transact-sql.md)
127-
- [BULK INSERT](../../t-sql/statements/bulk-insert-transact-sql.md)
128-
- [OPENROWSET](../../t-sql/functions/openrowset-transact-sql.md)
124+
- [CREATE DATABASE SCOPED CREDENTIAL (Transact-SQL)](../../t-sql/statements/create-database-scoped-credential-transact-sql.md)
125+
- [CREATE EXTERNAL DATA SOURCE (Transact-SQL)](../../t-sql/statements/create-external-data-source-transact-sql.md)
126+
- [BULK INSERT (Transact-SQL)](../../t-sql/statements/bulk-insert-transact-sql.md)
127+
- [OPENROWSET (Transact-SQL)](../../t-sql/functions/openrowset-transact-sql.md)
Lines changed: 44 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -1,74 +1,75 @@
11
---
2-
title: "Format files to import & export data"
2+
title: "Format files to import and export data"
33
description: When you bulk import to a SQL Server table or bulk export from a table, a format file can store field format information for a data file relative to a table.
44
author: rwestMSFT
55
ms.author: randolphwest
6-
ms.date: 08/29/2022
6+
ms.date: 12/20/2023
77
ms.service: sql
88
ms.subservice: data-movement
99
ms.topic: conceptual
1010
helpviewer_keywords:
1111
- "bulk exporting [SQL Server], format files"
1212
- "bulk importing [SQL Server], format files"
1313
- "format files [SQL Server]"
14-
monikerRange: ">=aps-pdw-2016||=azuresqldb-current||>=sql-server-2016||>=sql-server-linux-2017||=azuresqldb-mi-current"
14+
monikerRange: ">=aps-pdw-2016 || =azuresqldb-current || >=sql-server-2016 || >=sql-server-linux-2017 || =azuresqldb-mi-current"
1515
---
1616
# Format files to import or export data (SQL Server)
1717

18-
[!INCLUDE[SQL Server Azure SQL Database PDW](../../includes/applies-to-version/sql-asdb-asdbmi-pdw.md)]
18+
[!INCLUDE [SQL Server Azure SQL Database PDW](../../includes/applies-to-version/sql-asdb-asdbmi-pdw.md)]
1919

20-
When you bulk import data into a [!INCLUDE[ssNoVersion](../../includes/ssnoversion-md.md)] table or bulk export data from a table, you can use a *format file* to store all the format information that is required to bulk export or bulk import data. This includes format information for each field in a data file relative to that table.
20+
When you bulk import data into a [!INCLUDE [ssNoVersion](../../includes/ssnoversion-md.md)] table or bulk export data from a table, you can use a *format file* to store all the format information that is required to bulk export or bulk import data. This includes format information for each field in a data file relative to that table.
2121

22-
[!INCLUDE[ssnoversion](../../includes/ssnoversion-md.md)] supports two types of format files: XML formats and non-XML format files. Both non-XML format files and XML format files contain descriptions of every field in a data file, and XML format files also contain descriptions of the corresponding table columns. Generally, XML and non-XML format files are interchangeable. However, we recommend that you use the XML syntax for new format files because they provide several advantages over non-XML format files. For more information, see [XML Format Files (SQL Server)](../../relational-databases/import-export/xml-format-files-sql-server.md).
22+
[!INCLUDE [ssnoversion](../../includes/ssnoversion-md.md)] supports two types of format files: XML formats and non-XML format files. Both non-XML format files and XML format files contain descriptions of every field in a data file, and XML format files also contain descriptions of the corresponding table columns. Generally, XML and non-XML format files are interchangeable. However, we recommend that you use the XML syntax for new format files because they provide several advantages over non-XML format files. For more information, see [XML Format Files (SQL Server)](xml-format-files-sql-server.md).
2323

2424
> [!NOTE]
25-
> This syntax, including bulk insert, is not supported in Azure Synapse Analytics. [!INCLUDE [Use ADF or PolyBase instead of Synapse Bulk Insert](includes/bulk-insert-synapse.md)]
25+
> This syntax, including bulk insert, isn't supported in Azure Synapse Analytics. [!INCLUDE [Use ADF or PolyBase instead of Synapse Bulk Insert](includes/bulk-insert-synapse.md)]
2626
27-
## <a id="Benefits"></a> Benefits of format files
27+
## Benefits of format files
2828

29-
- Provides a flexible system for writing data files that requires little or no editing to comply with other data formats or to read data files from other software.
30-
- Enables you to bulk import data without having to add or delete unnecessary data or to reorder existing data in the data file. Format files are particularly useful when a mismatch exists between fields in the data file and columns in the table.
29+
Format files provide a flexible system for writing data files that requires little or no editing to comply with other data formats or to read data files from other software.
3130

32-
## <a id="ExamplesOfFFs"></a> Examples of format files
31+
You can bulk import data without having to add or delete unnecessary data or to reorder existing data in the data file. Format files can be useful when a mismatch exists between fields in the data file and columns in the table.
3332

34-
The following examples show the layout of a non-XML format file and of an XML format file. These format files correspond to the `HumanResources.myTeam` table in the [!INCLUDE[ssSampleDBobject](../../includes/sssampledbobject-md.md)] sample database. This table contains four columns: `EmployeeID`, `Name`, `Title`, and `ModifiedDate`.
33+
## Examples of format files
34+
35+
The following examples show the layout of a non-XML format file and of an XML format file. These format files correspond to the `HumanResources.myTeam` table in the [!INCLUDE [ssSampleDBobject](../../includes/sssampledbobject-md.md)] sample database. This table contains four columns: `EmployeeID`, `Name`, `Title`, and `ModifiedDate`.
3536

3637
> [!NOTE]
37-
> For information about this table and how to create it, see [HumanResources.myTeam Sample Table &#40;SQL Server&#41;](../../relational-databases/import-export/humanresources-myteam-sample-table-sql-server.md).
38+
> For information about this table and how to create it, see [HumanResources.myTeam sample table (SQL Server)](humanresources-myteam-sample-table-sql-server.md).
3839
3940
### A. Use a non-XML format file
4041

41-
The following non-XML format file uses the [!INCLUDE[ssNoVersion](../../includes/ssnoversion-md.md)] native data format for the `HumanResources.myTeam` table. This format file was created by using the following `bcp` command.
42+
The following non-XML format file uses the [!INCLUDE [ssNoVersion](../../includes/ssnoversion-md.md)] native data format for the `HumanResources.myTeam` table. This format file was created by using the following `bcp` command.
4243

4344
```cmd
4445
bcp AdventureWorks2022.HumanResources.myTeam format nul -f myTeam.Fmt -n -T
4546
```
4647

47-
The `bcp` command defaults to a local, default instance of SQL Server with Windows Authentication. You can specify other instance and login information as desired, for more information see [bcp Utility](../../tools/bcp-utility.md). For example, to specify a remote server named instance with Windows Authentication, use:
48+
The `bcp` command defaults to a local, default instance of [!INCLUDE [ssnoversion-md](../../includes/ssnoversion-md.md)] with Windows Authentication. You can specify other instance and login information as desired, for more information, see [bcp Utility](../../tools/bcp-utility.md). For example, to specify a remote server named instance with Windows Authentication, use:
4849

4950
```cmd
5051
bcp AdventureWorks2022.HumanResources.myTeam format nul -f myTeam.Fmt -n -T -S servername/instancename
5152
```
5253

53-
The contents of this format file are as follows, starting with the major version number of SQL Server, and the table metadata information.
54+
The contents of this format file are as follows, starting with the major version number of [!INCLUDE [ssnoversion-md](../../includes/ssnoversion-md.md)], and the table metadata information.
5455

5556
```output
5657
14.0
5758
4
58-
1 SQLSMALLINT 0 2 "" 1 EmployeeID ""
59-
2 SQLNCHAR 2 100 "" 2 Name SQL_Latin1_General_CP1_CI_AS
60-
3 SQLNCHAR 2 100 "" 3 Title SQL_Latin1_General_CP1_CI_AS
59+
1 SQLSMALLINT 0 2 "" 1 EmployeeID ""
60+
2 SQLNCHAR 2 100 "" 2 Name SQL_Latin1_General_CP1_CI_AS
61+
3 SQLNCHAR 2 100 "" 3 Title SQL_Latin1_General_CP1_CI_AS
6162
4 SQLNCHAR 2 100 "" 4 Background SQL_Latin1_General_CP1_CI_AS
6263
```
6364

64-
For more information, see [Non-XML Format Files &#40;SQL Server&#41;](../../relational-databases/import-export/non-xml-format-files-sql-server.md).
65+
For more information, see [Use Non-XML format files (SQL Server)](non-xml-format-files-sql-server.md).
6566

66-
### B. Using an XML format file
67+
### B. Use an XML format file
6768

68-
The following XML format file uses the [!INCLUDE[ssNoVersion](../../includes/ssnoversion-md.md)] native data format for the `HumanResources.myTeam` table. This format file was created by using the following `bcp` command.
69+
The following XML format file uses the [!INCLUDE [ssNoVersion](../../includes/ssnoversion-md.md)] native data format for the `HumanResources.myTeam` table. This format file was created by using the following `bcp` command.
6970

7071
```cmd
71-
bcp AdventureWorks2022.HumanResources.myTeam format nul -f myTeam.Xml -x -n -T
72+
bcp AdventureWorks2022.HumanResources.myTeam format nul -f myTeam.xml -x -n -T
7273
```
7374

7475
The format file contains:
@@ -91,39 +92,39 @@ The format file contains:
9192
</BCPFORMAT>
9293
```
9394

94-
For more information, see [XML Format Files &#40;SQL Server&#41;](../../relational-databases/import-export/xml-format-files-sql-server.md).
95-
96-
## <a id="WhenFFrequired"></a> When is a format file required?
95+
For more information, see [XML Format Files (SQL Server)](xml-format-files-sql-server.md).
9796

98-
- An INSERT ... SELECT * FROM OPENROWSET(BULK...) statement always requires a format file.
99-
- For **bcp** or BULK INSERT, in simple situations, using a format file is optional and rarely necessary. However, for complex bulk-import situations, a format file is frequently required.
97+
## When is a format file required?
10098

101-
Format files are required if:
99+
Format files are usually required in the following circumstances:
102100

101+
- When you use an `INSERT ... SELECT * FROM OPENROWSET(BULK...)` statement.
102+
- For complex bulk-import situations using **bcp** or `BULK INSERT`.
103103
- The same data file is used as a source for multiple tables that have different schemas.
104104
- The data file has a different number of fields that the target table has columns; for example:
105105

106-
- The target table contains at least one column for which either a default value is defined or NULL is allowed.
107-
- The users do not have SELECT/INSERT permissions on one or more columns in the table.
106+
- The target table contains at least one column for which either a default value is defined or `NULL` is allowed.
107+
- The users don't have `SELECT`/`INSERT` permissions on one or more columns in the table.
108108
- A single data file is used with two or more tables that have different schemas.
109109

110110
- The column order is different for the data file and table.
111111
- The terminating characters or prefix lengths differ among the columns of the data file.
112112

113113
> [!NOTE]
114-
> In the absence of a format file, if a **bcp** command specifies a data-format switch (**-n**, **-c**, **-w**, or **-N**) or a BULK INSERT operation specifies the DATAFILETYPE option, the specified data format is used as the default method of interpreting the fields of the data file.
114+
> In the absence of a format file, if a **bcp** command specifies a data-format switch (`-n`, `-c`, `-w`, or `-N`) or a `BULK INSERT` operation specifies the `DATAFILETYPE` option, the specified data format is used as the default method of interpreting the fields of the data file.
115115
116-
## See also
116+
## Related tasks
117117

118-
- [Non-XML Format Files &#40;SQL Server&#41;](../../relational-databases/import-export/non-xml-format-files-sql-server.md)
119-
- [XML Format Files &#40;SQL Server&#41;](../../relational-databases/import-export/xml-format-files-sql-server.md)
120-
- [Data Formats for Bulk Import or Bulk Export &#40;SQL Server&#41;](../../relational-databases/import-export/data-formats-for-bulk-import-or-bulk-export-sql-server.md)
121-
- [bcp Utility](../../tools/bcp-utility.md)
118+
- [Using BCP native/format file vs text file and the BOM](https://techcommunity.microsoft.com/t5/sql-server-blog/using-sqlbcp-native-format-file-vs-text-file-and-the-bom/ba-p/3194629)
119+
- [Use Non-XML format files (SQL Server)](non-xml-format-files-sql-server.md)
120+
- [XML Format Files (SQL Server)](xml-format-files-sql-server.md)
121+
- [Data formats for bulk import or bulk export (SQL Server)](data-formats-for-bulk-import-or-bulk-export-sql-server.md)
122+
- [Create a format file with bcp (SQL Server)](create-a-format-file-sql-server.md)
122123

123-
## <a id="RelatedTasks"></a> Next steps
124+
## Related content
124125

125-
- [Create a Format File &#40;SQL Server&#41;](../../relational-databases/import-export/create-a-format-file-sql-server.md)
126-
- [Use a Format File to Bulk Import Data &#40;SQL Server&#41;](../../relational-databases/import-export/use-a-format-file-to-bulk-import-data-sql-server.md)
127-
- [Use a Format File to Skip a Table Column &#40;SQL Server&#41;](../../relational-databases/import-export/use-a-format-file-to-skip-a-table-column-sql-server.md)
128-
- [Use a Format File to Skip a Data Field &#40;SQL Server&#41;](../../relational-databases/import-export/use-a-format-file-to-skip-a-data-field-sql-server.md)
129-
- [Use a Format File to Map Table Columns to Data-File Fields &#40;SQL Server&#41;](../../relational-databases/import-export/use-a-format-file-to-map-table-columns-to-data-file-fields-sql-server.md)
126+
- [bcp Utility](../../tools/bcp-utility.md)
127+
- [Use a format file to bulk import data (SQL Server)](use-a-format-file-to-bulk-import-data-sql-server.md)
128+
- [Use a Format file to skip a table column (SQL Server)](use-a-format-file-to-skip-a-table-column-sql-server.md)
129+
- [Use a format file to skip a data field (SQL Server)](use-a-format-file-to-skip-a-data-field-sql-server.md)
130+
- [Use a format file to map table columns to data-file fields (SQL Server)](use-a-format-file-to-map-table-columns-to-data-file-fields-sql-server.md)

0 commit comments

Comments
 (0)