You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/integration-services/load-data-to-sql-data-warehouse.md
+89-34Lines changed: 89 additions & 34 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,52 +1,56 @@
1
1
---
2
-
title: Load data from SQL Server into Azure SQL Data Warehouse (SSIS) | Microsoft Docs
3
-
description: Shows you how to create a SQL Server Integration Services (SSIS) package to move data from a wide variety of data sources to SQL Data Warehouse.
2
+
title: Load data into Azure SQL Data Warehouse with SQL Server Integration Services (SSIS) | Microsoft Docs
3
+
description: Shows you how to create a SQL Server Integration Services (SSIS) package to move data from a wide variety of data sources to Azure SQL Data Warehouse.
4
4
documentationcenter: NA
5
-
ms.service: sql-data-warehouse
6
-
ms.component: data-movement
5
+
ms.prod: sql
6
+
ms.prod_service: "integration-services"
7
+
ms.suite: "sql"
8
+
ms.technology: "integration-services"
7
9
ms.devlang: NA
8
10
ms.topic: conceptual
9
11
ms.tgt_pltfrm: NA
10
12
ms.custom: loading
11
-
ms.date: 04/04/2018
13
+
ms.date: 08/09/2018
12
14
ms.author: douglasl
13
15
author: douglaslMS
14
16
manager: craigg-msft
15
17
---
16
-
# Load data from SQL Server to Azure SQL Data Warehouse with SQL Server Integration Services (SSIS)
18
+
# Load data into Azure SQL Data Warehouse with SQL Server Integration Services (SSIS)
17
19
18
-
Create a SQL Server Integration Services (SSIS) package to load data from SQL Server into [Azure SQL Data Warehouse](/azure/sql-data-warehouse/index). You can optionally restructure, transform, and cleanse the data as it passes through the SSIS data flow.
20
+
Create a SQL Server Integration Services (SSIS) package to load data into [Azure SQL Data Warehouse](/azure/sql-data-warehouse/index). You can optionally restructure, transform, and cleanse the data as it passes through the SSIS data flow.
19
21
20
-
In this tutorial, you will:
22
+
This article shows you how to do the following things.
21
23
22
24
* Create a new Integration Services project in Visual Studio.
23
-
* Connect to data sources, including SQL Server (as a source) and SQL Data Warehouse (as a destination).
24
25
* Design an SSIS package that loads data from the source into the destination.
25
26
* Run the SSIS package to load the data.
26
27
27
-
This tutorial uses SQL Server as the data source. SQL Server could be running on premises or in an Azure virtual machine.
28
-
29
28
## Basic concepts
30
-
The package is the unit of work in SSIS. Related packages are grouped in projects. You create projects and design packages in Visual Studio with SQL Server Data Tools. The design process is a visual process in which you drag and drop components from the Toolbox to the design surface, connect them, and set their properties. After you finish your package, you can optionally deploy it to SQL Server for comprehensive management, monitoring, and security.
31
29
32
-
## Options for loading data with SSIS
30
+
The package is the basic unit of work in SSIS. Related packages are grouped in projects. You create projects and design packages in Visual Studio with SQL Server Data Tools. The design process is a visual process in which you drag and drop components from the Toolbox to the design surface, connect them, and set their properties. After you finish your package, you can optionally deploy it to SQL Server for comprehensive management, monitoring, and security.
31
+
32
+
A detailed introduction to SSIS is beyond the scope of this article. To learn more, see:
33
+
34
+
-[SQL Server Integration Services](sql-server-integration-services.md)
35
+
36
+
-[How to Create an ETL Package](ssis-how-to-create-an-etl-package.md)
37
+
38
+
## Options for loading data into SQL Data Warehouse with SSIS
33
39
SQL Server Integration Services (SSIS) is a flexible set of tools that provides a variety of options for connecting to, and loading data into, SQL Data Warehouse.
34
40
35
-
1. Use an ADO NET Destination to connect to SQL Data Warehouse. This tutorial uses an ADO NET Destination because it has the fewest configuration options.
36
-
2. Use an OLE DB Destination to connect to SQL Data Warehouse. This option may provide slightly better performance than the ADO NET Destination.
37
-
3. Use the Azure Blob Upload Task to stage the data in Azure Blob Storage. Then use the SSIS Execute SQL task to launch a Polybase script that loads the data into SQL Data Warehouse. This option provides the best performance of the three options listed here. To get the Azure Blob Upload task, download the [Microsoft SQL Server 2016 Integration Services Feature Pack for Azure][Microsoft SQL Server 2016 Integration Services Feature Pack for Azure]. To learn more about Polybase, see [PolyBase Guide][PolyBase Guide].
41
+
1. The preferred method, which provides the best performance, is to create a package that uses the [Azure SQL DW Upload Task](control-flow/azure-sql-dw-upload-task.md) to load the data. This task encapsulates both source and destination information. It assumes that your source data is stored locally in delimited text files.
38
42
39
-
## Before you start
40
-
To step through this tutorial, you need:
43
+
2. Alternately, you can create a package that uses a Data Flow task that contains a source and a destination. This approach supports a wide range of data sources, including SQL Server and Azure SQL Data Warehouse.
41
44
42
-
1.**SQL Server Integration Services (SSIS)**. SSIS is a component of SQL Server and requires an evaluation version or a licensed version of SQL Server. To get an evaluation version of SQL Server 2016 Preview, see [SQL Server Evaluations][SQL Server Evaluations].
43
-
2.**Visual Studio**. To get the free Visual Studio Community Edition, see [Visual Studio Community][Visual Studio Community].
45
+
## Prerequisites
46
+
To step through this tutorial, you need the following things:
47
+
48
+
1.**SQL Server Integration Services (SSIS)**. SSIS is a component of SQL Server and requires a licensed version, or the developer or evaluation version, of SQL Server. To get an evaluation version of SQL Server, see [Evaluate SQL Server][https://www.microsoft.com/evalcenter/evaluate-sql-server-2017-rtm].
49
+
2.**Visual Studio** (optional). To get the free Visual Studio Community Edition, see [Visual Studio Community][Visual Studio Community]. If you don't want to install Visual Studio, you can install SQL Server Data Tools (SSDT). SSDT installs a version of Visual Studio with limited functionality.
44
50
3.**SQL Server Data Tools for Visual Studio (SSDT)**. To get SQL Server Data Tools for Visual Studio, see [Download SQL Server Data Tools (SSDT)][Download SQL Server Data Tools (SSDT)].
45
-
4.**Sample data**. This tutorial uses sample data stored in SQL Server in the AdventureWorks sample database as the source data to be loaded into SQL Data Warehouse. To get the AdventureWorks sample database, see [AdventureWorks 2014 Sample Databases][AdventureWorks 2014 Sample Databases].
46
-
5.**A SQL Data Warehouse database and permissions**. This tutorial connects to a SQL Data Warehouse instance and loads data into it. You have to have permissions to create a table and to load data.
47
-
6.**A firewall rule**. You have to create a firewall rule on SQL Data Warehouse with the IP address of your local computer before you can upload data to the SQL Data Warehouse.
51
+
4.**An Azure SQL Data Warehouse database and permissions**. This tutorial connects to a SQL Data Warehouse instance and loads data into it. You have to have permissions to connect, to create a table, and to load data.
48
52
49
-
## Step 1: Create a new Integration Services project
53
+
## Create a new Integration Services project
50
54
1. Launch Visual Studio.
51
55
2. On the **File** menu, select **New | Project**.
52
56
3. Navigate to the **Installed | Templates | Business Intelligence | Integration Services** project types.
@@ -60,7 +64,55 @@ Visual Studio opens and creates a new Integration Services (SSIS) project. Then
60
64
61
65
![][01]
62
66
63
-
## Step 2: Create the basic data flow
67
+
## Option 1 - Use the SQL DW Upload task
68
+
69
+
The first approach is a package that uses the SQL DW Upload task. This task encapsulates both source and destination information. It assumes that your source data is stored in delimited text files, either locally or in Azure Blob Storage.
70
+
71
+
### Prerequisites for Option 1
72
+
73
+
To continue the tutorial with this option, you need the following things:
74
+
75
+
- The [Microsoft SQL Server Integration Services Feature Pack for Azure][Microsoft SQL Server 2017 Integration Services Feature Pack for Azure]. The SQL DW Upload task is a component of the Feature Pack.
76
+
77
+
- An [Azure Blob Storage](https://docs.microsoft.com/azure/storage/) account. The SQL DW Upload task loads data from Azure Blob Storage into Azure SQL Data Warehouse. You can load files that are already in Blob Storage, or you can load files from your computer. If you select files on your computer, the SQL DW Upload task uploads them to Blob Storage first for staging, and then loads them into SQL Data Warehouse.
78
+
79
+
### Add and configure the SQL DW Upload Task
80
+
81
+
1. Drag a SQL DW Upload Task from the Toolbox to the center of the design surface (on the **Control Flow** tab).
82
+
83
+
2. Double-click the task to open the **SQL DW Upload Task Editor**.
84
+
85
+
86
+
87
+
3. Configure the task with the help of the guidance in the article [Azure SQL DW Upload Task](control-flow/azure-sql-dw-upload-task.md). Since this task encapsulates both source and destination information, and the mappings between source and destination tables, the task editor has several pages of settings to configure.
88
+
89
+
### Create a similar solution manually
90
+
91
+
For more control, you can manually create a package that emulates the work done by the SQL DW Upload task. To get the Azure Blob Upload task, download the [Microsoft SQL Server Integration Services Feature Pack for Azure][Microsoft SQL Server 2017 Integration Services Feature Pack for Azure].
92
+
93
+
1. Use the Azure Blob Upload Task to stage the data in Azure Blob Storage.
94
+
95
+
2. Then use the SSIS Execute SQL task to launch a Polybase script that loads the data into SQL Data Warehouse. To learn more about Polybase, see [PolyBase Guide][PolyBase Guide].
96
+
97
+
## Option 2 - Use a source and destination
98
+
99
+
The second approach is a typical package which uses a Data Flow task that contains a source and a destination. This approach supports a wide range of data sources, including SQL Server and Azure SQL Data Warehouse.
100
+
101
+
This tutorial uses SQL Server as the data source. SQL Server runs on premises or on an Azure virtual machine.
102
+
103
+
To connect to SQL Server and to SQL Data Warehouse, you can use an ADO.NET connection manager and source and destination, or an OLE DB connection manager and source and destination. This tutorial uses ADO.NET because it has the fewest configuration options. OLE DB may provide slightly better performance than ADO.NET.
104
+
105
+
As a shortcut, you can use the SQL Server Import and Export Wizard to create the basic package. Then, save the package, and open it in Visual Studio or SSDT to view and customize it.
106
+
107
+
### Prerequisites for Option 2
108
+
109
+
To continue the tutorial with this option, you need the following things:
110
+
111
+
1.**Sample data**. This tutorial uses sample data stored in SQL Server in the AdventureWorks sample database as the source data to be loaded into SQL Data Warehouse. To get the AdventureWorks sample database, see [AdventureWorks Sample Databases][AdventureWorks 2014 Sample Databases].
112
+
113
+
2.**A firewall rule**. You have to create a firewall rule on SQL Data Warehouse with the IP address of your local computer before you can upload data to the SQL Data Warehouse.
114
+
115
+
### Create the basic data flow
64
116
1. Drag a Data Flow Task from the Toolbox to the center of the design surface (on the **Control Flow** tab).
65
117
66
118
![][02]
@@ -70,7 +122,7 @@ Visual Studio opens and creates a new Integration Services (SSIS) project. Then
70
122
71
123
![][09]
72
124
73
-
##Step 3: Configure the source adapter
125
+
###Configure the source adapter
74
126
1. Double-click the source adapter to open the **ADO.NET Source Editor**.
75
127
76
128
![][03]
@@ -101,15 +153,15 @@ Visual Studio opens and creates a new Integration Services (SSIS) project. Then
101
153
8. In the **Preview Query Results** dialog box, click **Close** to return to the **ADO.NET Source Editor**.
102
154
9. In the **ADO.NET Source Editor**, click **OK** to finish configuring the data source.
103
155
104
-
##Step 4: Connect the source adapter to the destination adapter
156
+
###Connect the source adapter to the destination adapter
105
157
1. Select the source adapter on the design surface.
106
158
2. Select the blue arrow that extends from the source adapter and drag it to the destination editor until it snaps into place.
107
159
108
160
![][10]
109
161
110
162
In a typical SSIS package, you use a number of other components from the SSIS Toolbox in between the source and the destination to restructure, transform, and cleanse your data as it passes through the SSIS data flow. To keep this example as simple as possible, we’re connecting the source directly to the destination.
111
163
112
-
##Step 5: Configure the destination adapter
164
+
###Configure the destination adapter
113
165
1. Double-click the destination adapter to open the **ADO.NET Destination Editor**.
114
166
115
167
![][11]
@@ -140,9 +192,11 @@ Visual Studio opens and creates a new Integration Services (SSIS) project. Then
140
192
![][13]
141
193
9. Click **OK** to finish configuring the data source.
142
194
143
-
## Step 6: Run the package to load the data
195
+
## Run the package to load the data
144
196
Run the package by clicking the **Start** button on the toolbar or by selecting one of the **Run** options on the **Debug** menu.
145
197
198
+
The following paragraphs describe what you see if you created the package with the second option described in this article, that is, with a data flow containing a source and destination.
199
+
146
200
As the package begins to run, you see yellow spinning wheels to indicate activity as well as the number of rows processed so far.
147
201
148
202
![][14]
@@ -154,9 +208,10 @@ When the package has finished running, you see green check marks to indicate suc
154
208
Congratulations! You’ve successfully used SQL Server Integration Services to load data into Azure SQL Data Warehouse.
155
209
156
210
## Next steps
157
-
* Learn more about the SSIS data flow. Start here: [Data Flow][Data Flow].
158
-
* Learn how to debug and troubleshoot your packages right in the design environment. Start here: [Troubleshooting Tools for Package Development][Troubleshooting Tools for Package Development].
159
-
* Learn how to deploy your SSIS projects and packages to Integration Services Server or to another storage location. Start here: [Deployment of Projects and Packages][Deployment of Projects and Packages].
211
+
212
+
- Learn how to debug and troubleshoot your packages right in the design environment. Start here: [Troubleshooting Tools for Package Development][Troubleshooting Tools for Package Development].
213
+
214
+
- Learn how to deploy your SSIS projects and packages to Integration Services Server or to another storage location. Start here: [Deployment of Projects and Packages][Deployment of Projects and Packages].
0 commit comments