Skip to content

Commit 9e297a2

Browse files
committed
Bringing even with master.
2 parents cfa604e + 5c046f6 commit 9e297a2

33 files changed

Lines changed: 1111 additions & 635 deletions

File tree

.openpublishing.redirection.json

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1490,6 +1490,26 @@
14901490
"redirect_url": "/sql/analysis-services/analysis-services",
14911491
"redirect_document_id": false
14921492
},
1493+
{
1494+
"source_path": "docs/analysis-services/deprecated-analysis-services-features-in-sql-server-2016.md",
1495+
"redirect_url": "/sql/analysis-services/analysis-services-backward-compatibility",
1496+
"redirect_document_id": false
1497+
},
1498+
{
1499+
"source_path": "docs/analysis-services/discontinued-analysis-services-functionality-in-sql-server-2016.md",
1500+
"redirect_url": "/sql/analysis-services/analysis-services-backward-compatibility",
1501+
"redirect_document_id": false
1502+
},
1503+
{
1504+
"source_path": "docs/analysis-services/breaking-changes-to-analysis-services-features-in-sql-server-2016.md",
1505+
"redirect_url": "/sql/analysis-services/analysis-services-backward-compatibility",
1506+
"redirect_document_id": false
1507+
},
1508+
{
1509+
"source_path": "docs/analysis-services/behavior-changes-to-analysis-services-features-in-sql-server-2016.md",
1510+
"redirect_url": "/sql/analysis-services/analysis-services-backward-compatibility",
1511+
"redirect_document_id": false
1512+
},
14931513
{
14941514
"source_path": "docs/analysis-services/troubleshoot-a-power-pivot-for-sharepoint-installation.md",
14951515
"redirect_url": "/sql/analysis-services/analysis-services",
Lines changed: 10 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: "Architecture | Microsoft Docs"
33
ms.custom: ""
4-
ms.date: "04/18/2017"
4+
ms.date: "07/11/2017"
55
ms.prod: "sql-server-2016"
66
ms.reviewer: ""
77
ms.suite: ""
@@ -13,21 +13,20 @@ author: "jeannt"
1313
ms.author: "jeannt"
1414
manager: "jhubbard"
1515
---
16-
17-
# Architecture Overview - Machine Learning Services with Python
16+
# Architecture overview for Machine Learning Services with Python
1817

1918
This topic provides an overview of how Python is integrated with SQL Server, including the security model, the components in the database engine that support external script execution, and new components that enable interoperability of Python with SQL Server. For details see the linked topics.
2019

2120
> [!IMPORTANT]
22-
> Support for Python is available beginning with SQL Server 2017. This is a pre-release feature and subject to change.
21+
> Support for Python is available beginning with SQL Server 2017 CTP 2.0. This is a pre-release feature and subject to change.
2322
24-
## Python Interoperability
23+
## Python interoperability
2524

26-
SQL Server Machine Learning Services (In-Database) installs the Anaconda distribution of Python, and the Python 3.5 runtime and interpreter. This ensure near-complete compatibility with standard Python solutions. Python runs in a separate process from SQL Server, to ensure that database operations are not compromised.
25+
SQL Server Machine Learning Services (In-Database) installs the Anaconda distribution of Python, and the Python 3.5 runtime and interpreter. This ensures near-complete compatibility with standard Python solutions. Python runs in a separate process from SQL Server, to guarantee that database operations are not compromised.
2726

28-
For more information about the interaction of SQL Server with Python, see [Python Interoperability](/python-interoperability.md)
27+
For more information about the interaction of SQL Server with Python, see [Python interoperability](/python-interoperability.md)
2928

30-
## New Components that Support Python Integration
29+
## Components that support Python integration
3130

3231
The extensibility framework introduced in SQL Server 2016 now supports execution of Python script, through the addition of new language-specific components. These components improve data exchange speed and compression, while providing a secure, high-performance platform for running external scripts.
3332

@@ -41,12 +40,12 @@ All tasks are secured by Windows job objects or SQL Server security. Data is kep
4140

4241
For details, see [Security for Python](../../advanced-analytics/python/security-overview-sql-server-python-services.md)
4342

44-
## Resource Governance
43+
## Resource governance
4544

4645
In SQL Server Enterprise Edition, you can use Resource Governor to manage and monitor resource use of external script operations, including R script and Python scripts.
4746

4847
For more information, see [Resource Governance for R](../../advanced-analytics/r/resource-governance-for-r-services.md).
4948

50-
## See Also
49+
## Next steps
5150

52-
[Run Python using T-SQl](../tutorials/run-python-using-t-sql.md)
51+
[Run Python using T-SQL](../tutorials/run-python-using-t-sql.md)
21.8 KB
Loading

docs/advanced-analytics/python/new-components-in-sql-server-to-support-python-integration.md

Lines changed: 28 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: "New Components for Python Integration | Microsoft Docs"
2+
title: "Components for Python integration with SQL Server | Microsoft Docs"
33
ms.custom: ""
44
ms.date: "03/30/2017"
55
ms.prod: "sql-server-2016"
@@ -13,13 +13,13 @@ author: "jeannt"
1313
ms.author: "jeannt"
1414
manager: "jhubbard"
1515
---
16-
# New components in SQL Server to Support Python Integration
16+
# Components in SQL Server to support Python integration
1717

18-
Beginning in SQL Server 2017, Machine Learning Services supports Python as an external language that can be executed from T-SQL, or executed remotely using SQL Server as the compute context.
18+
Beginning in SQL Server 2017, Machine Learning Services supports Python as an external language that can be executed from T-SQL, or executed remotely using SQL Server as the compute context.
1919

2020
This topic describes the components in SQL Server 2017 that support extensibility in general and the Python language specifically.
2121

22-
## SQL Server Components and Providers
22+
## SQL Server components and providers
2323

2424
To configure SQL Server 2017 to allow Python script execution is a multi-step process.
2525

@@ -29,29 +29,34 @@ To configure SQL Server 2017 to allow Python script execution is a multi-step pr
2929

3030
Additional steps might be required to support remote script execution.
3131

32-
For more information, see [Set up Machine LEarnign Services](setup-python-machine-learning-services.md)
32+
For more information, see [Set up Machine Learning Services](setup-python-machine-learning-services.md)
3333

34-
### **Launchpad**
34+
### Launchpad
3535

3636
The SQL Server Trusted Launchpad is a service introduced in SQL Server 2016 that manages and executes external scripts, similar to the way that the full-text indexing and query service launches a separate host for processing full-text queries.
3737

38-
The Launchpad service will start only trusted launchers that are published by Microsoft, or that have been certified by Microsoft as meeting requirements for performance and resource management.
38+
The Launchpad service can start only trusted launchers that are published by Microsoft, or that have been certified by Microsoft as meeting requirements for performance and resource management.
39+
3940
+ SQL Server 2016 supports R
4041
+ SQL Server 2017 supports R and Python 3.5
4142

4243
The [!INCLUDE[rsql_launchpad_md](../../includes/rsql-launchpad-md.md)] service runs under its own user account.
4344

45+
> [!TIP]
46+
> If you change the account that runs Launchpad, be sure to do so using SQL Server Configuration Manager, to ensure that changes are written to related files.
47+
4448
To execute tasks in a specific supported language, the Launchpad gets a secured worker account from the pool, and starts a satellite process to manage the external runtime:
49+
4550
+ RLauncher.dll for the R language
4651
+ Pythonlauncher.dll for Python 3.5
4752

4853
Each satellite process inherits the user account of the Launchpad and uses that worker account for the duration of script execution. If the Python script uses parallel processes, they are created under the same, single worker account.
4954

5055
For more information about the security context of the Launchpad, see [Security](security-overview-sql-server-python-services.md).
5156

52-
### **BxlServer and SQL Satellite**
57+
### BxlServer and SQL Satellite
5358

54-
If you run Process Explorer while a Python job is running, you might see one or multiple instances of BxlServer.
59+
If you run [Process Explorer](https://technet.microsoft.com/sysinternals/processexplorer.aspx) while a Python job is running, you might see one or multiple instances of BxlServer.
5560

5661
**BxlServer** is an executable provided by Microsoft that manages communication between [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)] and Python (or R). It creates the Windows job objects that are used to contain external script sessions, provisions secure working folders for each external script job, and uses SQL Satellite to manage data transfer between the external runtime and [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)].
5762

@@ -72,28 +77,29 @@ SQL Satellite uses a custom data format that is optimized for fast data transfer
7277

7378
The SQL Satellite can be monitored by using windows extended events (xEvents). For more information, see [Extended Events for R](../../advanced-analytics/r/extended-events-for-sql-server-r-services.md).
7479

75-
76-
## Communication Channels Between Components
80+
## Communication channels between components
7781

7882
+ **TCP/IP**
83+
7984
By default, internal communications between [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)] and the SQL Satellite use TCP/IP.
8085

8186
+ **Named Pipes**
8287

8388
Internal data transport between the BxlServer and [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)] through SQL Satellite uses a proprietary, compressed data format to enhance performance. Data is exchanged between Python and BxlServer in BXL format, using Named Pipes.
8489

8590
+ **ODBC**
91+
8692
Communications between external data science clients and the [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)] instance use ODBC. The account that sends the script jobs to [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)] must have both permissions to connect to the instance and to run external scripts.
8793

8894
Additionally, depending on the task, the account might need these permissions:
95+
8996
+ Read data used by the job
9097
+ Write data to tables: for example, when saving results to a table
9198
+ Create database objects: for example, if saving external script as part of a new stored procedure.
9299

93100
When [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)] is used as the compute context for Python script executed from a remote client, and the Python executable must retrieve data from an external source, ODBC is used for writeback. [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)] will map the identity of the user issuing the remote command to the identity of the user on the current instance, and run the ODBC command using that user's credentials. The connection string needed to perform this ODBC call is obtained from the client code.
94101

95-
96-
## Interaction of Components
102+
## Interaction of components
97103

98104
The following diagrams depict the interaction of SQL Server components with the Python runtime in each of the supported scenarios: running script in-database, and remote execution from a Python terminal, using a SQL Server compute context.
99105

@@ -111,27 +117,29 @@ After the script has been embedded in the stored procedure, any application that
111117
4. BxlServer coordinates with the Python runtime to manage exchanges of data, and storage of working results.
112118
5. SQL Satellite manages communications about related tasks and processes with [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)].
113119
6. BxlServer uses SQL Satellite to communicate status and results to [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)].
114-
7. [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)] gets results and closes related tasks and processes.
115-
120+
7. [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)] gets results and closes related tasks and processes.
116121

117122
### Python scripts executed from a remote client
118123

119124
You can run Python scripts from a remote computer, such as a laptop, and have them execute in the context of the SQl Server computer, if these conditions are met:
125+
120126
+ You design the scripts appropriately
121-
+ The remote computer has installed the extensibility libraries that are used by Machine Learning Services.
127+
+ The remote computer has installed the extensibility libraries that are used by Machine Learning Services
122128

123129
The following diagram summarizes the overall workflow when scripts are sent from a remote computer.
124130

125-
![remote-sqlcc-from-python](../../advanced-analytics/python/media/remote-sqlcc-from-python.png)
131+
![remote-sqlcc-from-python](../../advanced-analytics/python/media/remote-sqlcc-from-python2.png)
126132

127-
1. For function that are supported in RevoScalePy functions, the Python runtime calls a linking function, which in turn calls BxlServer.
128-
2. BxlServer is included with machine Learning Services (In-Database) and runs in a separate process from the Python runtime.
133+
1. For functions that are supported in **revoscalepy**, the Python runtime calls a linking function, which in turn calls BxlServer.
134+
2. BxlServer is included with Machine Learning Services (In-Database) and runs in a separate process from the Python runtime.
129135
3. BxlServer determines the connection target and initiates a connection using ODBC, passing credentials supplied as part of the connection string in the Python script.
130136
4. BxlServer opens a connection to the [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)] instance.
131-
5. When an external script runtime is called, the Launchpad service is invoked, which is turn starts the appropriate launcher: in this case, PythonLauncher.dll. Thereafter, processing of Python code is handled in a workflow similar to that when Python code is invoked from a stored procedure in T-SQL.
137+
5. When an external script runtime is called, the Launchpad service is invoked, which in turn starts the appropriate launcher: in this case, PythonLauncher.dll. Thereafter, processing of Python code is handled in a workflow similar to that when Python code is invoked from a stored procedure in T-SQL.
132138
6. PythonLauncher makes a call to the instance of the Python that is installed on the [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)] computer.
133139
7. Results are returned to BxlServer.
134140
8. SQL Satellite manages communication with [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)] and cleanup of related job objects.
135141
9. [!INCLUDE[ssNoVersion_md](../../includes/ssnoversion-md.md)] passes results back to the client.
136142

143+
## Next steps
137144

145+
[Architecture overview for Python in SQL Server](architecture-overview-sql-server-python.md)
Lines changed: 14 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: "Architecture Overview (SQL Server R Services) | Microsoft Docs"
33
ms.custom: ""
4-
ms.date: "04/18/2017"
4+
ms.date: "07/11/2017"
55
ms.prod: "sql-server-2016"
66
ms.reviewer: ""
77
ms.suite: ""
@@ -15,27 +15,27 @@ author: "jeannt"
1515
ms.author: "jeannt"
1616
manager: "jhubbard"
1717
---
18-
# Architecture Overview (SQL Server R Services)
18+
# Architecture overview for R in SQL Server
1919

20-
This section provides an overview of the architecture of SQL Server 2016 R Services, and SQL Server 2017 Machine Learning Services.
20+
This section provides an overview of the architecture of SQL Server 2016 R Services, and of SQL Server 2017 Machine Learning Services.
2121

2222
The architecture for the extensibility architecture is the same or very similar for the SQL Server 2016 and SQL Server 2017 releases, and similar also for R and Python. However, to simplify the discussion, this topic discusses only the R components, including new components added in the SQL Server database engine to support external script execution, security, R libraries, and interoperability with open source R.
2323

2424
Additional details are provided in the links for each section.
2525

26-
## R Interoperability
26+
## R interoperability
2727

28-
Both SQL Server 2016 R Services and SQL Server Machine Learning Services (In-Database) install an open source distribution of R, as well as packages provided by Microsoft that support distributed and/or parallel processing.
28+
Both SQL Server 2016 R Services and SQL Server 2017 Machine Learning Services (In-Database) install an open source distribution of R, as well as packages provided by Microsoft that support distributed and/or parallel processing.
2929

30-
The architecture is designed such that external scripts using R or Python run in a separate process from SQL Server. Current users of R should be able to port their R code and execute it in T-SQL with relatively minor modifications.
30+
The architecture is designed such that external scripts using R run in a separate process from SQL Server. Current users of R should be able to port their R code and execute it in T-SQL with relatively minor modifications.
3131

32-
To scale your solution or use parallel processing, we recommend that you use the RevoScaleR package. If you do not use the distributed computing capabilities provided by these libraries, you can still get some performance improvements by running your R code in the context of SQL Server.
32+
To scale your solution or use parallel processing, we recommend that you use the [RevoScaleR](https://docs.microsoft.com/r-server/r-reference/revoscaler/revoscaler) package or the [MicrosoftML](https://docs.microsoft.com/r-server/r-reference/microsoftml/microsoftml-package) package. If you do not use the distributed computing capabilities provided by these libraries, you can still get some performance improvements by running your R code in the context of SQL Server.
3333

3434
For more information about the external scripting components that are installed, or the interaction of SQL Server with R, see [R Interoperability](../../advanced-analytics/r/r-interoperability-in-sql-server.md)
3535

36-
## New Components to Support R Integration
36+
## Components to support R integration
3737

38-
The extensibility framework introduced in SQL Server 2016 is continued in SQL Server 2017. The extensibility components are used by SQL Server to start the external runtime for R, pass data between R and the database engine, and coordinate parallel tasks needed for an R job.
38+
The extensibility framework introduced in SQL Server 2016 is continued in SQL Server 2017. The extensibility components are used by SQL Server to start the external runtime for R, to pass data between R and the database engine, and to coordinate parallel tasks needed for an R job.
3939

4040
The role of these additional components is to improve data exchange speed and compression, while providing a secure, high-performance platform for running external scripts.
4141

@@ -47,12 +47,12 @@ When you run R code using Machine Learning Services or SQL Server R Services, al
4747

4848
SQL Server intercepts all job requests, secures the task and its data using Windows job objects, and maintains security over data using SQL Server user accounts and database roles.
4949

50-
Data is kept within the compliance boundary by enforcing SQL Server security at the table, database, and instance level. The database administrator can control who has the ability to run R jobs, and who has the ability to install or share R packages. The administrator can also monitor the use of R scripts by either rremote or local users, and monitor and manage the resources consumed.
50+
Data is kept within the compliance boundary by enforcing SQL Server security at the table, database, and instance level. The database administrator can control who has the ability to run R jobs, and who has the ability to install or share R packages. The administrator can also monitor the use of R scripts by either remote or local users, and monitor and manage the resources consumed.
5151

52-
For details, see [Security for R](../../advanced-analytics/r/security-overview-sql-server-r.md)
52+
## Next steps
5353

54+
[Components that support R integration](new-components-in-sql-server-to-support-r.md)
5455

55-
## See Also
56-
57-
[Machine Learning Tutorials](../../advanced-analytics/tutorials/machine-learning-services-tutorials.md)
56+
[R interoperability](r-interoperability-in-sql-server.md)
5857

58+
[Security overview](security-overview-sql-server-r.md)
18.3 KB
Loading

0 commit comments

Comments
 (0)