Skip to content

Commit 2fe2244

Browse files
committed
Bringing even with master.
2 parents e20ed33 + 0aaa16b commit 2fe2244

59 files changed

Lines changed: 1240 additions & 1211 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

docs/TOC.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
## [XQuery](xquery/xquery-language-reference-sql-server.md)
1111
1212
# [Samples](sample/microsoft-sql-server-samples.md)
13-
# [Tools](tools/command-prompt-utility-reference-database-engine.md)
13+
# [Tools](tools/overview-sql-tools.md)
1414
1515
# Developer
1616
## [SQL Server Data Tools (SSDT)](ssdt/download-sql-server-data-tools-ssdt.md)
Lines changed: 30 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: "Architecture and Overview | Microsoft Docs"
33
ms.custom: ""
4-
ms.date: "06/01/2017"
4+
ms.date: "06/20/2017"
55
ms.prod: "sql-server-2016"
66
ms.reviewer: ""
77
ms.suite: ""
@@ -16,19 +16,19 @@ manager: "jhubbard"
1616

1717
# Architecture and Overview of Machine Learning Services
1818

19-
This topic describes the goals of the extensibility framework that supports execution of Python and R script in SQL Server.
19+
This topic describes the goals of the extensibility framework that supports execution of Python and R script in SQL Server.
2020

21-
It also provides an overview of how the architecture is designed to meet these goals, how R and Python are supported and executed by SQL Server, and the benefits of integration.
21+
It also provides an overview of how the architecture is designed to meet these goals, how R and Python are supported and executed by SQL Server, and the benefits of integration.
2222

2323
Overall, the extensibility framework is almost identical for R and Python, with some minor differences in the details of the launchers that are called, configuration options, and so forth. For further information about the implementation for a specific language, see these topics:
2424

2525
- [Architecture Overview for SQL Server R Services](r/architecture-overview-sql-server-r.md)
2626
- [Architecture Overview for Python in SQL Server](python/architecture-overview-sql-server-python.md)
2727

2828

29-
## Design Goals
29+
## Background
3030

31-
In SQL Server 2016, numerous changes were introduced to the database engine to support execution of R scripts using SQL Server. In SQL Server 2017, this underlying infrastructure was improved to add support for the Python language.
31+
In SQL Server 2016, numerous changes were introduced to the database engine to support execution of R scripts using SQL Server. In SQL Server 2017, this underlying infrastructure was improved to add support for the Python language.
3232

3333
The goal of the extensibility framework was to create a better interface between SQL Server and data science languages such as R and Python, both to reduce the friction that occurs when data science solutions are moved into production, and to protect data that might be exposed during the data science development process.
3434

@@ -41,53 +41,66 @@ By executing a trusted scripting language within a secure framework managed by S
4141
- The database administrator should be able to manage resources used by external scripts, manage users, and manage and monitor external code libraries.
4242
- The system must support solutions based entirely on open source distributions of R and Python, but use proprietary components developed by Microsoft to provide greater security and performance.
4343

44-
## Architecture Pillars
44+
## Architecture core concepts
4545

46-
To meet these goals, the architecture of SQL Server 2016 R Services and SQL Server 2017 Machine Machine Learning Services for R and Python is based on these core concepts:
46+
To meet these goals, the architecture of SQL Server 2016 R Services and SQL Server 2017 Machine Learning Services for R and Python is based on these core concepts:
4747

4848
+ **Multi-process architecture**
49-
Both R and Python are open-source languages with rich and enthusiastic community support. Therefore, it is important to maintain full interoperability with open source R and Python.
5049

51-
Open source distributions of R and Python are installed with SQL Server under license, and can function independently from SQL Server if needed.
50+
Both R and Python are open-source languages with rich and enthusiastic community support. Therefore, it is important to maintain full interoperability with open source R and Python.
5251

53-
In addition, Microsoft provides a set of proprietary DLLs that provide integration with SQL Server, including data translation, compression and optimization targeted at each supported language.
52+
Open source distributions of R and Python are installed with SQL Server under license, and can function independently from SQL Server if needed.
53+
54+
In addition, Microsoft provides a set of proprietary libraries that provide integration with SQL Server, including data translation, compression, and optimization targeted at each supported language.
5455

5556
+ **Security**
56-
Better security means support for both integrated Windows authentication and password-based SQL logins, as well as secure handling of credentials, reliance on SQL Server for data protection, and use of the SQL Server Trusted Launchpad to manage external script execution and secure data used in scripts.
57+
58+
Better security means support for both integrated Windows authentication and password-based SQL logins, as well as secure handling of credentials, reliance on SQL Server for data protection, and use of the SQL Server Trusted Launchpad to manage external script execution and secure data used in scripts.
5759

5860
+ **Scalability and performance**
61+
5962
Integration with SQL Server is key to improving the usefulness of R and Python in the enterprise. Any R or Python script can be run by calling a stored procedure, and the results are returned as tabular results directly to SQL Server, making it easy to generate or consume machine learning from any application that can send a SQL query and handle the results.
6063

6164
Performance optimization relies on two equally powerful aspects of the platform: resource governance and parallel processing using SQL Server, and distributed computing provided by the algorithms in **RevoScaleR** and **revoscalepy**.
6265

6366

64-
## Solution Development and Deployment Workflow
67+
## Solution development and deployment
6568

6669
In addition to these core goals for the extensibility platform, the machine learning services in SQL Server are designed to provide strong integration with the database engine and the BI stack, with these benefits:
6770

6871
+ Performance and resource management through traditional monitoring tools
6972
+ Easy use of Python and R data by BI suites or any application that can consume SQL query results
7073
+ Much lower barrier for enterprise development of machine learning solutions
71-
+
74+
7275
Let's see how it works in practice.
7376

7477
![ML solution development process](media/ml-solution-development-process.png "Develop and deploy using Machine Learning Services")
7578

76-
1. Data is kept within the compliance boundary and use of data can be managed and monitored by SQL Server. Meanwhile, the DBA has full control over who can install packages or run scripts on the server. If so desired, the DBA can also delegate permissions on a database level to data scientists or managers.
77-
2. Data scientists can build and test solutions in their preferred R or Python environments, disconnected from the server.
78-
3. The SQL developer can use familiar tools such as Management Studio or Visual Studio to integrate the R or Python code with SQL Server. The tight integration means that the savvy developer can choose the best tool to optimize each task. For example, you might use SQL for some feature engineering tasks and R for others. You might embed Python script in an Integration Services task to perform sophisticated text analytics.
79+
1. Data is kept within the compliance boundary and use of data can be managed and monitored by SQL Server. Meanwhile, the DBA has full control over who can install packages or run scripts on the server. If so desired, the DBA can also delegate permissions on a database level to data scientists or managers.
80+
2. Data scientists can build and test solutions in their preferred R or Python environments, disconnected from the server.
81+
3. The SQL developer can use familiar tools such as Management Studio or Visual Studio to integrate the R or Python code with SQL Server. The tight integration means that the savvy developer can choose the best tool to optimize each task. For example, you might use SQL for some feature engineering tasks and R for others. You might embed Python script in an Integration Services task to perform sophisticated text analytics.
7982
4. Tested and ready-to-deploy solutions can be optimized using SQL Server technologies, such as columnstore indexes, for better performance. Newer features let you batch-train many small models in parallel on partitioned data set, or score millions of rows in using native SQL code optimized for machine learning tasks.
8083
5. Ready to lift off? You can easily expose your predictive solutions to the BI stack or external applications by using stored procedures.
8184

82-
## Related Products
85+
## Related products
8386

8487
Not sure which machine learning solution meets your needs? In addition to embedded analytics in SQL Server 2016 and SQL Server 2017, Microsoft provides the following machine learning platforms and services:
8588

8689
+ [Microsoft R Server](https://msdn.microsoft.com/microsoft-r/rserver)
90+
91+
A multi-platform environment for developing, distributing, and managing machine learning jobs
8792
+ [Data Science Virtual Machine](https://docs.microsoft.com/azure/machine-learning/machine-learning-data-science-virtual-machine-overview)
93+
94+
All the tools you need for machine learning, preinstalled. Use Jupyter notebooks, Python, or R.
95+
96+
Try the new [Windows 2016 preview edition](http://aka.ms/dsvm/win2016), which includes GPU versions of popular deep learning frameworks such as CNTK and mxNet, as well as support for Windows containers!
8897
+ [Azure Cognitive Services](https://azure.microsoft.com/services/cognitive-services/)
98+
99+
A variety of cloud services for adding AI and ML into your applications, including natural language indexing of video, facial recognition, emotion detection, text analytics, machine translation, and much, more
89100
+ [Azure Machine Learning](https://azure.microsoft.com/services/machine-learning/)
90101

102+
A cloud-based drag-and-drop interface for designing machine learning workflows, coupled with the ability to automate and integrate with applications via web services and PowerShell
103+
91104
## See Also
92105

93106
[R Server Standalone](https://docs.microsoft.com/sql/advanced-analytics/r/r-server-standalone)

0 commit comments

Comments
 (0)