You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
title: "Architecture and Overview | Microsoft Docs"
3
3
ms.custom: ""
4
-
ms.date: "06/01/2017"
4
+
ms.date: "06/20/2017"
5
5
ms.prod: "sql-server-2016"
6
6
ms.reviewer: ""
7
7
ms.suite: ""
@@ -16,19 +16,19 @@ manager: "jhubbard"
16
16
17
17
# Architecture and Overview of Machine Learning Services
18
18
19
-
This topic describes the goals of the extensibility framework that supports execution of Python and R script in SQL Server.
19
+
This topic describes the goals of the extensibility framework that supports execution of Python and R script in SQL Server.
20
20
21
-
It also provides an overview of how the architecture is designed to meet these goals, how R and Python are supported and executed by SQL Server, and the benefits of integration.
21
+
It also provides an overview of how the architecture is designed to meet these goals, how R and Python are supported and executed by SQL Server, and the benefits of integration.
22
22
23
23
Overall, the extensibility framework is almost identical for R and Python, with some minor differences in the details of the launchers that are called, configuration options, and so forth. For further information about the implementation for a specific language, see these topics:
24
24
25
25
-[Architecture Overview for SQL Server R Services](r/architecture-overview-sql-server-r.md)
26
26
-[Architecture Overview for Python in SQL Server](python/architecture-overview-sql-server-python.md)
27
27
28
28
29
-
## Design Goals
29
+
## Background
30
30
31
-
In SQL Server 2016, numerous changes were introduced to the database engine to support execution of R scripts using SQL Server. In SQL Server 2017, this underlying infrastructure was improved to add support for the Python language.
31
+
In SQL Server 2016, numerous changes were introduced to the database engine to support execution of R scripts using SQL Server. In SQL Server 2017, this underlying infrastructure was improved to add support for the Python language.
32
32
33
33
The goal of the extensibility framework was to create a better interface between SQL Server and data science languages such as R and Python, both to reduce the friction that occurs when data science solutions are moved into production, and to protect data that might be exposed during the data science development process.
34
34
@@ -41,53 +41,66 @@ By executing a trusted scripting language within a secure framework managed by S
41
41
- The database administrator should be able to manage resources used by external scripts, manage users, and manage and monitor external code libraries.
42
42
- The system must support solutions based entirely on open source distributions of R and Python, but use proprietary components developed by Microsoft to provide greater security and performance.
43
43
44
-
## Architecture Pillars
44
+
## Architecture core concepts
45
45
46
-
To meet these goals, the architecture of SQL Server 2016 R Services and SQL Server 2017 Machine Machine Learning Services for R and Python is based on these core concepts:
46
+
To meet these goals, the architecture of SQL Server 2016 R Services and SQL Server 2017 Machine Learning Services for R and Python is based on these core concepts:
47
47
48
48
+**Multi-process architecture**
49
-
Both R and Python are open-source languages with rich and enthusiastic community support. Therefore, it is important to maintain full interoperability with open source R and Python.
50
49
51
-
Open source distributions of R and Python are installed with SQL Server under license, and can function independently from SQL Server if needed.
50
+
Both R and Python are open-source languages with rich and enthusiastic community support. Therefore, it is important to maintain full interoperability with open source R and Python.
52
51
53
-
In addition, Microsoft provides a set of proprietary DLLs that provide integration with SQL Server, including data translation, compression and optimization targeted at each supported language.
52
+
Open source distributions of R and Python are installed with SQL Server under license, and can function independently from SQL Server if needed.
53
+
54
+
In addition, Microsoft provides a set of proprietary libraries that provide integration with SQL Server, including data translation, compression, and optimization targeted at each supported language.
54
55
55
56
+**Security**
56
-
Better security means support for both integrated Windows authentication and password-based SQL logins, as well as secure handling of credentials, reliance on SQL Server for data protection, and use of the SQL Server Trusted Launchpad to manage external script execution and secure data used in scripts.
57
+
58
+
Better security means support for both integrated Windows authentication and password-based SQL logins, as well as secure handling of credentials, reliance on SQL Server for data protection, and use of the SQL Server Trusted Launchpad to manage external script execution and secure data used in scripts.
57
59
58
60
+**Scalability and performance**
61
+
59
62
Integration with SQL Server is key to improving the usefulness of R and Python in the enterprise. Any R or Python script can be run by calling a stored procedure, and the results are returned as tabular results directly to SQL Server, making it easy to generate or consume machine learning from any application that can send a SQL query and handle the results.
60
63
61
64
Performance optimization relies on two equally powerful aspects of the platform: resource governance and parallel processing using SQL Server, and distributed computing provided by the algorithms in **RevoScaleR** and **revoscalepy**.
62
65
63
66
64
-
## Solution Development and Deployment Workflow
67
+
## Solution development and deployment
65
68
66
69
In addition to these core goals for the extensibility platform, the machine learning services in SQL Server are designed to provide strong integration with the database engine and the BI stack, with these benefits:
67
70
68
71
+ Performance and resource management through traditional monitoring tools
69
72
+ Easy use of Python and R data by BI suites or any application that can consume SQL query results
70
73
+ Much lower barrier for enterprise development of machine learning solutions
71
-
+
74
+
72
75
Let's see how it works in practice.
73
76
74
77

75
78
76
-
1. Data is kept within the compliance boundary and use of data can be managed and monitored by SQL Server. Meanwhile, the DBA has full control over who can install packages or run scripts on the server. If so desired, the DBA can also delegate permissions on a database level to data scientists or managers.
77
-
2. Data scientists can build and test solutions in their preferred R or Python environments, disconnected from the server.
78
-
3. The SQL developer can use familiar tools such as Management Studio or Visual Studio to integrate the R or Python code with SQL Server. The tight integration means that the savvy developer can choose the best tool to optimize each task. For example, you might use SQL for some feature engineering tasks and R for others. You might embed Python script in an Integration Services task to perform sophisticated text analytics.
79
+
1. Data is kept within the compliance boundary and use of data can be managed and monitored by SQL Server. Meanwhile, the DBA has full control over who can install packages or run scripts on the server. If so desired, the DBA can also delegate permissions on a database level to data scientists or managers.
80
+
2. Data scientists can build and test solutions in their preferred R or Python environments, disconnected from the server.
81
+
3. The SQL developer can use familiar tools such as Management Studio or Visual Studio to integrate the R or Python code with SQL Server. The tight integration means that the savvy developer can choose the best tool to optimize each task. For example, you might use SQL for some feature engineering tasks and R for others. You might embed Python script in an Integration Services task to perform sophisticated text analytics.
79
82
4. Tested and ready-to-deploy solutions can be optimized using SQL Server technologies, such as columnstore indexes, for better performance. Newer features let you batch-train many small models in parallel on partitioned data set, or score millions of rows in using native SQL code optimized for machine learning tasks.
80
83
5. Ready to lift off? You can easily expose your predictive solutions to the BI stack or external applications by using stored procedures.
81
84
82
-
## Related Products
85
+
## Related products
83
86
84
87
Not sure which machine learning solution meets your needs? In addition to embedded analytics in SQL Server 2016 and SQL Server 2017, Microsoft provides the following machine learning platforms and services:
85
88
86
89
+[Microsoft R Server](https://msdn.microsoft.com/microsoft-r/rserver)
90
+
91
+
A multi-platform environment for developing, distributing, and managing machine learning jobs
All the tools you need for machine learning, preinstalled. Use Jupyter notebooks, Python, or R.
95
+
96
+
Try the new [Windows 2016 preview edition](http://aka.ms/dsvm/win2016), which includes GPU versions of popular deep learning frameworks such as CNTK and mxNet, as well as support for Windows containers!
A variety of cloud services for adding AI and ML into your applications, including natural language indexing of video, facial recognition, emotion detection, text analytics, machine translation, and much, more
A cloud-based drag-and-drop interface for designing machine learning workflows, coupled with the ability to automate and integrate with applications via web services and PowerShell
103
+
91
104
## See Also
92
105
93
106
[R Server Standalone](https://docs.microsoft.com/sql/advanced-analytics/r/r-server-standalone)
0 commit comments