Skip to content

Commit 243053a

Browse files
committed
Bringing even with master.
2 parents 7ff1af0 + bf2a84e commit 243053a

5 files changed

Lines changed: 100 additions & 39 deletions

File tree

docs/relational-databases/TOC.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
## [Thread and Task Architecture](thread-and-task-architecture-guide.md)
1515
## [Transaction Log Architecture and Management Guide](sql-server-transaction-log-architecture-and-management-guide.md)
1616
## [Transaction Locking and Row Versioning Guide](sql-server-transaction-locking-and-row-versioning-guide.md)
17+
## [Ghost record cleanup process](ghost-record-cleanup-process-guide.md)
1718

1819
# [Backup and restore](../relational-databases/backup-restore/back-up-and-restore-of-sql-server-databases.md)
1920
# [Blob](../relational-databases/blob/binary-large-object-blob-data-sql-server.md)
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
---
2+
title: "Ghost cleanup process guide | Microsoft Docs"
3+
ms.custom: ""
4+
ms.date: "05/02/2018"
5+
ms.prod: "sql"
6+
ms.prod_service: "database-engine, sql-database, sql-data-warehouse, pdw"
7+
ms.service: ""
8+
ms.component: "relational-databases-misc"
9+
ms.reviewer: ""
10+
ms.suite: "sql"
11+
ms.technology:
12+
- "database-engine"
13+
ms.tgt_pltfrm: ""
14+
ms.topic: conceptual
15+
helpviewer_keywords:
16+
- "ghost cleanup"
17+
- "ghost records"
18+
- "ghost clean up process"
19+
author: "MashaMSFT"
20+
ms.author: "mathoma"
21+
manager: "craigg"
22+
---
23+
# Ghost cleanup process guide
24+
25+
The ghost cleanup process is a single-threaded background process that deletes records off of pages that have been marked for deletion. The following article provides an overview of this process.
26+
27+
## Ghost records
28+
29+
Records that are deleted from a leaf level of an index page aren't physically removed from the page - instead, the record is marked as 'to be deleted', or *ghosted*. This means that the row stays on the page but a bit is changed in the row header to indicate that the row is really a ghost. This is to optimize performance during a delete operation. Ghosts are necessary for row-level locking, but are also necessary for snapshot isolation where we need to maintain the older versions of rows.
30+
31+
## Ghost record cleanup task
32+
33+
Records that are marked for deletion, or *ghosted*, are cleaned up by the background ghost cleanup process. This background process runs sometime after the delete transaction is committed, and physically removes ghosted records from pages. The ghost cleanup process runs automatically on an interval (every 5 seconds for SQL Server 2012+, every 10 seconds for SQL Server 2008/2008R2) and checks to see if any pages have been marked with ghost records. If it finds any, then it goes and deletes the records that are marked for deletion, or *ghosted*, touching at most 10 pages with each execution.
34+
35+
When a record is ghosted, the database is marked as having ghosted entries, and the ghost cleanup process will only scan those databases. The ghost cleanup process will also mark the database as 'having no ghosted records' once all ghosted records have been deleted, and it will skip this database the next time it runs. The process will also skip any databases it is unable to take a shared lock on, and will try again the next time it runs.
36+
37+
The below query can identify how many ghosted records exist in a single database.
38+
39+
```sql
40+
SELECT sum(ghost_record_count) total_ghost_records, db_name(database_id)
41+
FROM sys.dm_db_index_physical_stats (NULL, NULL, NULL, NULL, NULL)
42+
group by database_id
43+
order by total_ghost_records desc
44+
```
45+
46+
## Disable the ghost cleanup
47+
48+
On high-load systems with many deletes, the ghost cleanup process can cause a performance issue from keeping pages in the buffer pool and generating IO. As such, it is possible to disable this process with the use of trace flag 661. More information about this can be found in [Tuning options for SQL Server when running high performance workloads](https://support.microsoft.com/en-us/help/920093/tuning-options-for-sql-server-when-running-in-high-performance-workloa). However, there are performance implications from disabling the process.
49+
50+
Disabling the ghost cleanup process can cause your database to grow unnecessarily large and can lead to performance issues. Since the ghost cleanup process removes records that are marked as ghosts, disabling the process will leave these records on the page, preventing SQL Server from reusing this space. This forces SQL Server to add data to new pages instead, leading to bloated database files, and can also cause [page splits](/indexes/specify-fill-factor-for-an-index.md). Page splits lead to performance issues when creating execution plans, and when doing scan operations.
51+
52+
Once the ghost cleanup process is disabled, some action needs to be taken to remove the ghosted records. One option is to execute an index rebuild, which will move data around on pages. Another option is to manually run [sp_clean_db_free_space](/system-stored-procedures/sp-clean-db-free-space-transact-sql.md) (to clean all database data files) or [sp_clean_db_file_free_space](/system-stored-procedures/sp-clean-db-file-free-space-transact-sql.md) (to clean a single database datafile), which will delete ghosted records.
53+
54+
>[!warning]
55+
> Disabling the ghost cleanup process is not generally recommended. Doing so should be tested thoroughly in a controlled environment before being implemented permanently in a production environment.
56+
57+
58+
## Next steps
59+
[Disabling the ghost clean up process](https://support.microsoft.com/en-us/help/920093/tuning-options-for-sql-server-when-running-in-high-performance-workloa)
60+
<br>[Remove ghost records from a single database file](/system-stored-procedures/sp-clean-db-file-free-space-transact-sql.md)
61+
<br>[Remove ghost records from all database data files](/system-stored-procedures/sp-clean-db-free-space-transact-sql.md)
62+

docs/relational-databases/system-stored-procedures/sp-clean-db-file-free-space-transact-sql.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@ EXEC sp_clean_db_file_free_space
7777
```
7878

7979
## See Also
80-
[Database Engine Stored Procedures &#40;Transact-SQL&#41;](../../relational-databases/system-stored-procedures/database-engine-stored-procedures-transact-sql.md)
80+
[Database Engine Stored Procedures &#40;Transact-SQL&#41;](../../relational-databases/system-stored-procedures/database-engine-stored-procedures-transact-sql.md)
81+
<br>[Ghost Cleanup Process Guide](../ghost-record-cleanup-process-guide.md)
8182

8283

docs/relational-databases/system-stored-procedures/sp-clean-db-free-space-transact-sql.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ EXEC sp_clean_db_free_space
7373
```
7474

7575
## See Also
76-
[Database Engine Stored Procedures &#40;Transact-SQL&#41;](../../relational-databases/system-stored-procedures/database-engine-stored-procedures-transact-sql.md)
76+
[Database Engine Stored Procedures &#40;Transact-SQL&#41;](../../relational-databases/system-stored-procedures/database-engine-stored-procedures-transact-sql.md)
77+
<br>[Ghost Cleanup Process Guide](../ghost-record-cleanup-process-guide.md)
7778

7879

docs/t-sql/functions/string-split-transact-sql.md

Lines changed: 33 additions & 37 deletions
Original file line numberDiff line numberDiff line change
@@ -29,10 +29,8 @@ manager: craigg
2929
Splits the character expression using specified separator.
3030

3131
> [!NOTE]
32-
> The **STRING_SPLIT** function is available only under compatibility level 130. If your database compatibility level is lower than 130, SQL Server will not be able to find and execute **STRING_SPLIT** function. You can change a compatibility level of database using the following command:
33-
> ALTER DATABASE DatabaseName SET COMPATIBILITY_LEVEL = 130
34-
>
35-
> Note that compatibility level 120 might be default even in new Azure SQL Databases.
32+
> The **STRING_SPLIT** function is available only under compatibility level 130 and above. If your database compatibility level is lower than 130, [!INCLUDE[ssNoVersion](../../includes/ssnoversion-md.md)] will not be able to find and execute **STRING_SPLIT** function. To change the compatibility level of a database, refer to [View or Change the Compatibility Level of a Database](../../relational-databases/databases/view-or-change-the-compatibility-level-of-a-database.md).
33+
> Note that compatibility level 120 might be default even in new [!INCLUDE[ssSDSfull](../../includes/sssdsfull-md.md)].
3634
3735
![Topic link icon](../../database-engine/configure-windows/media/topic-link.gif "Topic link icon") [Transact-SQL Syntax Conventions](../../t-sql/language-elements/transact-sql-syntax-conventions-transact-sql.md)
3836

@@ -44,16 +42,16 @@ STRING_SPLIT ( string , separator )
4442

4543
## Arguments
4644
*string*
47-
Is an [expression](../../t-sql/language-elements/expressions-transact-sql.md) of any character type (i.e. **nvarchar**, **varchar**, **nchar** or **char**).
45+
Is an [expression](../../t-sql/language-elements/expressions-transact-sql.md) of any character type (for example, **nvarchar**, **varchar**, **nchar**, or **char**).
4846

4947
*separator*
50-
Is a single character [expression](../../t-sql/language-elements/expressions-transact-sql.md) of any character type (e.g. **nvarchar(1)**, **varchar(1)**, **nchar(1)** or **char(1)**) that is used as separator for concatenated strings.
48+
Is a single character [expression](../../t-sql/language-elements/expressions-transact-sql.md) of any character type (for example, **nvarchar(1)**, **varchar(1)**, **nchar(1)**, or **char(1)**) that is used as separator for concatenated strings.
5149

5250
## Return Types
5351
Returns a single-column table with fragments. The name of the column is **value**. Returns **nvarchar** if any of the input arguments are either **nvarchar** or **nchar**. Otherwise returns **varchar**. The length of the return type is the same as the length of the string argument.
5452

5553
## Remarks
56-
**STRING_SPLIT** takes a string that should be divided and the separator that will be used to divide string. It returns a single-column table with substrings. For example, the following statement `SELECT value FROM STRING_SPLIT('Lorem ipsum dolor sit amet.', ' ');` using the space character as the separator, returns following result table:
54+
**STRING_SPLIT** takes a string that should be divided and the separator that will be used to divide string. It returns a single-column table with substrings. For example, the following statement `SELECT value FROM STRING_SPLIT('Lorem ipsum dolor sit amet.', ' ');` using the space character as the separator, returns following result table:
5755

5856
|value|
5957
|-----------|
@@ -63,39 +61,37 @@ STRING_SPLIT ( string , separator )
6361
|sit|
6462
|amet.|
6563

66-
If the input string is **NULL**, the **STRING_SPLIT** table-valued function returns an empty table.
64+
If the input string is **NULL**, the **STRING_SPLIT** table-valued function returns an empty table.
6765

68-
**STRING_SPLIT** requires at least compatibility mode 130.
66+
**STRING_SPLIT** requires at least compatibility mode 130.
6967

7068
## Examples
7169

72-
### A. Split comma separated value string
73-
Parse a comma separated list of values and return all non-empty tokens:
74-
75-
```
70+
### A. Split comma-separated value string
71+
Parse a comma separated list of values and return all non-empty tokens:
7672

73+
```sql
7774
DECLARE @tags NVARCHAR(400) = 'clothing,road,,touring,bike'
7875

7976
SELECT value
8077
FROM STRING_SPLIT(@tags, ',')
8178
WHERE RTRIM(value) <> '';
82-
8379
```
8480

85-
STRING_SPLIT will return empty string if there is nothing between separator. Condition RTRIM(value) <> '' will remove empty tokens.
81+
STRING_SPLIT will return empty string if there is nothing between separator. Condition RTRIM(value) <> '' will remove empty tokens.
8682

87-
### B. Split comma separated value string in a column
88-
Product table has a column with comma-separate list of tags shown in the following example:
83+
### B. Split comma-separated value string in a column
84+
Product table has a column with comma-separate list of tags shown in the following example:
8985

9086
|ProductId|Name|Tags|
9187
|---------------|----------|----------|
9288
|1|Full-Finger Gloves|clothing,road,touring,bike|
9389
|2|LL Headset|bike|
9490
|3|HL Mountain Frame|bike,mountain|
9591

96-
Following query transforms each list of tags and joins them with the original row:
92+
Following query transforms each list of tags and joins them with the original row:
9793

98-
```
94+
```sql
9995
SELECT ProductId, Name, value
10096
FROM Product
10197
CROSS APPLY STRING_SPLIT(Tags, ',');
@@ -114,9 +110,9 @@ FROM Product
114110
|3|HL Mountain Frame|mountain|
115111

116112
### C. Aggregation by values
117-
Users must create a report that shows the number of products per each tag, ordered by number of products, and to filter only the tags with more than 2 products.
113+
Users must create a report that shows the number of products per each tag, ordered by number of products, and to filter only the tags with more than two products.
118114

119-
```
115+
```sql
120116
SELECT value as tag, COUNT(*) AS [Number of articles]
121117
FROM Product
122118
CROSS APPLY STRING_SPLIT(Tags, ',')
@@ -126,19 +122,19 @@ ORDER BY COUNT(*) DESC;
126122
```
127123

128124
### D. Search by tag value
129-
Developers must create queries that find articles by keywords. They can use following queries:
125+
Developers must create queries that find articles by keywords. They can use following queries:
130126

131-
To find products with a single tag (clothing):
127+
To find products with a single tag (clothing):
132128

133-
```
129+
```sql
134130
SELECT ProductId, Name, Tags
135131
FROM Product
136132
WHERE 'clothing' IN (SELECT value FROM STRING_SPLIT(Tags, ','));
137133
```
138134

139-
Find products with two specified tags (clothing and road):
135+
Find products with two specified tags (clothing and road):
140136

141-
```
137+
```sql
142138

143139
SELECT ProductId, Name, Tags
144140
FROM Product
@@ -148,30 +144,30 @@ WHERE EXISTS (SELECT *
148144
```
149145

150146
### E. Find rows by list of values
151-
Developers must create a query that finds articles by a list of ids. They can use following query:
147+
Developers must create a query that finds articles by a list of IDs. They can use following query:
152148

153-
```
149+
```sql
154150
SELECT ProductId, Name, Tags
155151
FROM Product
156152
JOIN STRING_SPLIT('1,2,3',',')
157153
ON value = ProductId;
158154
```
159155

160-
This is replacement for common anti-pattern such as creating a dynamic SQL string in application layer or [!INCLUDE[tsql](../../includes/tsql-md.md)], or by using LIKE operator:
156+
This is replacement for common anti-pattern such as creating a dynamic SQL string in application layer or [!INCLUDE[tsql](../../includes/tsql-md.md)], or by using LIKE operator:
161157

162-
```
158+
```sql
163159
SELECT ProductId, Name, Tags
164160
FROM Product
165161
WHERE ',1,2,3,' LIKE '%,' + CAST(ProductId AS VARCHAR(20)) + ',%';
166162
```
167163

168164
## See Also
169-
[LEFT &#40;Transact-SQL&#41;](../../t-sql/functions/left-transact-sql.md)
170-
[LTRIM &#40;Transact-SQL&#41;](../../t-sql/functions/ltrim-transact-sql.md)
171-
[RIGHT &#40;Transact-SQL&#41;](../../t-sql/functions/right-transact-sql.md)
172-
[RTRIM &#40;Transact-SQL&#41;](../../t-sql/functions/rtrim-transact-sql.md)
173-
[SUBSTRING &#40;Transact-SQL&#41;](../../t-sql/functions/substring-transact-sql.md)
174-
[TRIM &#40;Transact-SQL&#41;](../../t-sql/functions/trim-transact-sql.md)
175-
[String Functions &#40;Transact-SQL&#41;](../../t-sql/functions/string-functions-transact-sql.md)
165+
[LEFT &#40;Transact-SQL&#41;](../../t-sql/functions/left-transact-sql.md)
166+
[LTRIM &#40;Transact-SQL&#41;](../../t-sql/functions/ltrim-transact-sql.md)
167+
[RIGHT &#40;Transact-SQL&#41;](../../t-sql/functions/right-transact-sql.md)
168+
[RTRIM &#40;Transact-SQL&#41;](../../t-sql/functions/rtrim-transact-sql.md)
169+
[SUBSTRING &#40;Transact-SQL&#41;](../../t-sql/functions/substring-transact-sql.md)
170+
[TRIM &#40;Transact-SQL&#41;](../../t-sql/functions/trim-transact-sql.md)
171+
[String Functions &#40;Transact-SQL&#41;](../../t-sql/functions/string-functions-transact-sql.md)
176172

177173

0 commit comments

Comments
 (0)