Skip to content

Commit cf8a335

Browse files
amvin87-zzamvin87-zz
committed
AG on SQL server containers with DH2i
1 parent 94e50c0 commit cf8a335

1 file changed

Lines changed: 380 additions & 0 deletions

File tree

‎docs/linux/tutorial-sql-server-containers-kubernetes-DH2i.md‎

Lines changed: 380 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,380 @@
1+
---
2+
title: Deploy Always on availability group using DH2i cluster solution for SQL Server Containers deployed on Azure Kubernetes Services (AKS)
3+
description: This tutorial shows how to deploy a SQL Server Always On availability group with DH2i Clustering solution for SQL Server containers on Azure Kubernetes Service.
4+
ms.custom: seo-lt-2019
5+
author: amvin87
6+
ms.author: amvin87
7+
ms.reviewer: amvin87
8+
ms.date: 09/07/2021
9+
ms.topic: tutorial
10+
ms.prod: sql
11+
ms.technology: linux
12+
---
13+
14+
# Deploy Always on availability group using DH2i cluster solution for SQL Server containers deployed on Azure Kubernetes Services (AKS)
15+
16+
This tutorial explains how to configure SQL Server Always On availability group for SQL Server Linux based containers deployed in Kubernetes cluster. In this case, Azure Kubernetes Service(AKS) is used as the kubernetes cluster and the tutorial consists of the following tasks:
17+
18+
1. Deploy Azure Kubernetes Service.
19+
2. Prepare the SQL Server & DhH2i container image.
20+
3. Deploy Containers on Azure Kubernetes Service.
21+
4. Configure the DxEnterprise cluster.
22+
5. Configure Read_Write_Routing_URL for listener functionality - Optional.
23+
24+
## Prerequisites
25+
1. You would need an Azure account to deploy Azure Kubernetes Service. For this tutorial a two node cluster is sufficient.
26+
2. Create Azure Container Registry this will be used in our deployment scripts to pull the custom image and deploy the containers to Azure Kubernetes service. You could use your choice of container registry instead of ACR to push the custom container images.
27+
28+
## Deploy Azure Kubernetes Service
29+
30+
To setup a two-node Kubernetes cluster using the Azure Kubernetes Service, please follow this [quickstart tutorial](https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough-portal#create-an-aks-cluster). Once you create the cluster you can connect to the cluster by following the steps documented in the ["connect to the cluster"](https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough-portal#connect-to-the-cluster) section of the article.
31+
32+
You should now a two-node kubernetes cluster, and running a command like : kubectl get nodes from your client machine should show results similar to this:
33+
34+
```bash
35+
C:\>kubectl get nodes
36+
NAME STATUS ROLES AGE VERSION
37+
aks-nodepool1-75119571-vmss000000 Ready agent 61d v1.19.9
38+
aks-nodepool1-75119571-vmss000001 Ready agent 61d v1.19.9
39+
```
40+
41+
## Prepare the SQL Server & DH2i DxEnterprise custom container image
42+
43+
Next, we are going to prepare the custom container image which will be used in our deployment scripts to deploy SQL Server, .Net and DxEnterprise inside a container deployed using this custom image. The sample dockerfile for the deployment is shared below, you can change the SQL Server version in the below sample dockerfile.
44+
45+
```bash
46+
FROM mcr.microsoft.com/mssql/server:2019-latest
47+
USER root
48+
49+
#Install dotnet
50+
RUN apt-get update \
51+
&& ACCEPT_EULA=Y apt-get upgrade -y \
52+
&& apt-get install -y wget \
53+
&& wget --no-dns-cache https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb -O packages-microsoft-prod.deb \
54+
&& dpkg -i packages-microsoft-prod.deb \
55+
&& apt-get update \
56+
&& apt-get install -y dotnet-runtime-3.1 zip \
57+
&& dpkg --purge packages-microsoft-prod \
58+
&& apt-get purge -y wget \
59+
&& apt-get clean \
60+
&& rm packages-microsoft-prod.deb \
61+
&& rm -rf /var/lib/apt/lists/*
62+
63+
#Download and unpack DxE, setup permissions
64+
ADD https://repos.dh2i.com/container/ ./dxe.tgz
65+
RUN tar zxvf dxe.tgz && rm dxe.tgz \
66+
&& chown -R mssql /var/opt/mssql \
67+
&& chmod -R 777 /opt/dh2i /etc/dh2i
68+
69+
#Finish setup
70+
EXPOSE 7979 7985
71+
ENV DX_HAS_MSSQLSERVER=1
72+
USER mssql
73+
ENTRYPOINT ["/opt/dh2i/sbin/dxstart.sh"]
74+
```
75+
76+
On a Linux machine, you can build the image using the commands shown below:
77+
78+
```bash
79+
$mkdir dockefiles
80+
$cd dockerfiles
81+
$nano Dockerfile
82+
# paste the sample dockerfile content shared above
83+
# now build the image using the command:
84+
$docker build -t <tagname> .
85+
# you should now be able to see the new image sqlimage when you run the docker images command
86+
```
87+
88+
Now tag the image and push it to ACR using the following commands, you need to ensure that you have already logged in to the ACR using the docker login command, for more details see [login to ACR](https://docs.microsoft.com/en-us/azure/container-registry/container-registry-get-started-portal#log-in-to-registry).
89+
90+
```bash
91+
$docker tag sqlimage/latest amvinacr.azurecr.io/sqlimage:latest
92+
#now push to the ACR repo:
93+
$docker push amvinacr.azurecr.io/sqlimage:latest
94+
#you can browse your ACR through the portal and should see the repo and the tag listed in the ACR.
95+
```
96+
This ensures that you now have the custom image pushed to Azure Container Registry (ACR) and now to integrate your Azure Kubernetes Service with Azure Container Registry, please run the below command, for more details refer this (article)[https://docs.microsoft.com/en-us/azure/aks/cluster-container-registry-integration]
97+
98+
```bash
99+
az aks update -n myAKSCluster -g amvindomain --attach-acr amvinacr
100+
```
101+
102+
## Deploy containers on Azure Kubernetes Service
103+
104+
We will be deploying SQL Server containers as statefulset deployments, a sample deployment file is shared below for reference which deploys the containers on the Azure Kubernetes Service. Please note the below points:
105+
106+
1. We are going to deploy three SQL Server instances, 1-Primary replica and 2-Secondary replicas. You can also optinally added labels to the node, to ensure that primary always runs on a specific node and the secondary replicas run on another node. Here are the steps to label the nodes:
107+
1. Get the node names of the cluster using the command:
108+
```bash
109+
kubectl get nodes
110+
```
111+
2. Now label the nodes using the commands:
112+
```bash
113+
kubectl label node aks-nodepool1-75119571-vmss000000 <role=ags-primary>
114+
kubectl label node aks-nodepool1-75119571-vmss000001 <role=ags-secondary>
115+
```
116+
2. Before you deploy the SQL Server containers, please create the SA password secret on kubernetes using the command:
117+
118+
```bash
119+
kubectl create secret generic mssql --from-literal=SA_PASSWORD="MyC0m9l&xP@ssw0rd"
120+
```
121+
Replace MyC0m9l&xP@ssw0rd with a complex password
122+
123+
3. Create a manifest (a YAML file) to describe the deployment. The following example describes our current deployment which uses the custom container image created in the preceding steps
124+
125+
```bash
126+
kind: StorageClass
127+
apiVersion: storage.k8s.io/v1
128+
metadata:
129+
name: azure-disk
130+
provisioner: kubernetes.io/azure-disk
131+
parameters:
132+
storageaccounttype: Standard_LRS
133+
kind: Managed
134+
---
135+
apiVersion: apps/v1
136+
kind: StatefulSet
137+
metadata:
138+
name: mssql-pri
139+
labels:
140+
app: mssql
141+
spec:
142+
serviceName: "mssql-pri"
143+
replicas: 1
144+
selector:
145+
matchLabels:
146+
app: mssql
147+
template:
148+
metadata:
149+
labels:
150+
app: mssql
151+
spec:
152+
securityContext:
153+
fsGroup: 10001
154+
containers:
155+
- name: mssql
156+
image: amvinacr.azurecr.io/sqldh2i:latest
157+
env:
158+
- name: ACCEPT_EULA
159+
value: "Y"
160+
- name: MSSQL_AGENT_ENABLED
161+
value: "Y"
162+
- name: MSSQL_ENABLE_HADR
163+
value: "1"
164+
- name: SA_PASSWORD
165+
valueFrom:
166+
secretKeyRef:
167+
name: mssql
168+
key: SA_PASSWORD
169+
volumeMounts:
170+
- name: dxe
171+
mountPath: "/etc/dh2i"
172+
- name: mssql
173+
mountPath: "/var/opt/mssql"
174+
nodeSelector:
175+
role: ags-primary
176+
volumeClaimTemplates:
177+
- metadata:
178+
name: dxe
179+
spec:
180+
accessModes:
181+
- ReadWriteOnce
182+
resources:
183+
requests:
184+
storage: 1Gi
185+
- metadata:
186+
name: mssql
187+
spec:
188+
accessModes:
189+
- ReadWriteOnce
190+
resources:
191+
requests:
192+
storage: 8Gi
193+
---
194+
apiVersion: apps/v1
195+
kind: StatefulSet
196+
metadata:
197+
name: mssql-sec
198+
labels:
199+
app: mssql
200+
spec:
201+
serviceName: "mssql-sec"
202+
replicas: 2
203+
selector:
204+
matchLabels:
205+
app: mssql
206+
template:
207+
metadata:
208+
labels:
209+
app: mssql
210+
spec:
211+
securityContext:
212+
fsGroup: 10001
213+
containers:
214+
- name: mssql
215+
image: amvinacr.azurecr.io/sqldh2i:latest
216+
env:
217+
- name: ACCEPT_EULA
218+
value: "Y"
219+
- name: MSSQL_AGENT_ENABLED
220+
value: "Y"
221+
- name: MSSQL_ENABLE_HADR
222+
value: "1"
223+
- name: SA_PASSWORD
224+
valueFrom:
225+
secretKeyRef:
226+
name: mssql
227+
key: SA_PASSWORD
228+
volumeMounts:
229+
- name: dxe
230+
mountPath: "/etc/dh2i"
231+
- name: mssql
232+
mountPath: "/var/opt/mssql"
233+
nodeSelector:
234+
role: ags-secondary
235+
volumeClaimTemplates:
236+
- metadata:
237+
name: dxe
238+
spec:
239+
accessModes:
240+
- ReadWriteOnce
241+
resources:
242+
requests:
243+
storage: 1Gi
244+
- metadata:
245+
name: mssql
246+
spec:
247+
accessModes:
248+
- ReadWriteOnce
249+
resources:
250+
requests:
251+
storage: 8Gi
252+
---
253+
apiVersion: v1
254+
kind: Service
255+
metadata:
256+
name: mssql-pri-0
257+
spec:
258+
type: LoadBalancer
259+
selector:
260+
statefulset.kubernetes.io/pod-name: mssql-pri-0
261+
ports:
262+
- name: sql
263+
protocol: TCP
264+
port: 1433
265+
targetPort: 1433
266+
- name: dxe
267+
protocol: TCP
268+
port: 7979
269+
targetPort: 7979
270+
---
271+
apiVersion: v1
272+
kind: Service
273+
metadata:
274+
name: mssql-sec-0
275+
spec:
276+
type: LoadBalancer
277+
selector:
278+
statefulset.kubernetes.io/pod-name: mssql-sec-0
279+
ports:
280+
- name: sql
281+
protocol: TCP
282+
port: 1433
283+
targetPort: 1433
284+
- name: dxe
285+
protocol: TCP
286+
port: 7979
287+
targetPort: 7979
288+
---
289+
apiVersion: v1
290+
kind: Service
291+
metadata:
292+
name: mssql-sec-1
293+
spec:
294+
type: LoadBalancer
295+
selector:
296+
statefulset.kubernetes.io/pod-name: mssql-sec-1
297+
ports:
298+
- name: sql
299+
protocol: TCP
300+
port: 1433
301+
targetPort: 1433
302+
- name: dxe
303+
protocol: TCP
304+
port: 7979
305+
targetPort: 7979
306+
```
307+
308+
Copy the preceding code into a new file, named sqldeployment.yaml, update the values like port, image and storage details as per your configuration. Create the deployment using the command below:
309+
310+
```bash
311+
kubectl apply -f <Path to sqldeployment.yaml file>
312+
```
313+
Once the deployment completes when you run the kubectl get all command you should see result as shown below:
314+
315+
```bash
316+
C:\>kubectl get all
317+
NAME READY STATUS RESTARTS AGE
318+
pod/mssql-pri-0 1/1 Running 0 33h
319+
pod/mssql-sec-0 1/1 Running 0 33h
320+
pod/mssql-sec-1 1/1 Running 0 33h
321+
322+
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
323+
service/kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 33h
324+
service/mssql-pri-0 LoadBalancer 10.0.134.183 20.204.22.235 1433:30678/TCP,7979:31136/TCP 33h
325+
service/mssql-sec-0 LoadBalancer 10.0.74.50 20.204.23.32 1433:31009/TCP,7979:30114/TCP 33h
326+
service/mssql-sec-1 LoadBalancer 10.0.63.62 20.204.74.9 1433:31616/TCP,7979:32190/TCP 33h
327+
328+
NAME READY AGE
329+
statefulset.apps/mssql-pri 1/1 33h
330+
statefulset.apps/mssql-sec 2/2 33h
331+
```
332+
As you can see, we have three SQL Server instances each using its own storage with three loadbalancer services exposing port 1433(SQL) and 7979 (DxEnterprise Cluster) so you can connect to each of the SQL Server instance using the External-IP address and the SA_PASSWORD is the same password that you provided when creating the mssql secret in the preceding steps.
333+
334+
## Configure the DxEnterprise Cluster on the Containers deployed
335+
336+
DxEnterprise is high availability clustering software from DH2i that supports SQL Server availability groups, including in containers. A fully featured [developer](https://dh2i.com/dxenterprise-dxodyssey-developer-edition) edition is available for non-production use. To configure the DxEnterprise cluster in containers, follow the steps in the sections "Configure the Primary and Create the Availability Group", "Join the Remaining Containers to the DxEnterprise Cluster" and "Configure the Availability Group Database(s)" respectively from this [DH2i guide](https://dh2i.com/wp-content/uploads/DxEnterprise-v21.0-SQL-Server-for-Kubernetes-StatefulSet-on-Azure-Quick-Start-Guide.pdf)
337+
338+
With this, you should have an Always On availability group created and database(s) added to the group supporting high availability.
339+
340+
## Steps to configure Read/write connection redirection: (Optional)
341+
342+
Once, the availability group is created you can enable the Read/write connection redirection from secondary to primary using the below steps. Refer, Read_write_routing_URL to know more. This fulfils the listener requirements
343+
344+
```bash
345+
USE [master]
346+
GO
347+
ALTER AVAILABILITY GROUP [ag_name]
348+
MODIFY REPLICA ON N'<name of the primary replica>' WITH (SECONDARY_ROLE(ALLOW_CONNECTIONS = ALL))
349+
GO
350+
USE [master]
351+
GO
352+
ALTER AVAILABILITY GROUP [AGS1]
353+
MODIFY REPLICA ON N'<name of the secondary-0 replica>' WITH (SECONDARY_ROLE(ALLOW_CONNECTIONS = ALL))
354+
GO
355+
USE [master]
356+
GO
357+
ALTER AVAILABILITY GROUP [AGS1]
358+
MODIFY REPLICA ON N'<name of the secondary-1 replica>' WITH (SECONDARY_ROLE(ALLOW_CONNECTIONS = ALL))
359+
GO
360+
USE [master]
361+
GO
362+
ALTER AVAILABILITY GROUP AGS1 MODIFY REPLICA ON N'<name of the primary replica>' WITH  
363+
(PRIMARY_ROLE (READ_WRITE_ROUTING_URL = 'TCP://<External IP address of primary -0>:1433'))
364+
GO
365+
USE [master]
366+
GO
367+
ALTER AVAILABILITY GROUP AGS1 MODIFY REPLICA ON N'<name of the secondary-0 replica>' WITH  
368+
(PRIMARY_ROLE (READ_WRITE_ROUTING_URL = 'TCP://<External IP address of secondary -0>:1433'))
369+
GO
370+
USE [master]
371+
GO
372+
ALTER AVAILABILITY GROUP AGS1 MODIFY REPLICA ON N'<name of the secondary-1 replica>' WITH  
373+
(PRIMARY_ROLE (READ_WRITE_ROUTING_URL = 'TCP://<External IP address of secondary -1>:1433'))
374+
GO
375+
```
376+
377+
## Next Steps
378+
379+
1. [Deploy SQL Server containers on Azure Kubernetes Service](https://docs.microsoft.com/en-us/sql/linux/tutorial-sql-server-containers-kubernetes?view=sql-server-ver15)
380+
2. [Deploy SQL Server Read Scale AG on SQL Server Linux based containers deployed on kubernetes](https://techcommunity.microsoft.com/t5/sql-server/configure-sql-server-ag-read-scale-for-sql-containers-deployed/ba-p/2224742)

0 commit comments

Comments
 (0)