About Me

My photo
I am MCSE in Data Management and Analytics with specialization in MS SQL Server and MCP in Azure. I have over 13+ years of experience in IT industry with expertise in data management, Azure Cloud, Data-Canter Migration, Infrastructure Architecture planning and Virtualization and automation. Contact me if you are looking for any sort of guidance in getting your Infrastructure provisioning automated through Terraform. I sometime write for a place to store my own experiences for future search and read by own blog but can hopefully help others along the way. Thanks.

How to Enlarge AdventureWorks DB & Pause-Resuming-Abort online index rebuilds in SQL server 2017

How to  Enlarge AdventureWorks DB & SQL Server 2017 Resumable Online Index Rebuilds


Problem:-

Rebuilding indexes on large tables can sometimes take quite a while to complete, and if an index rebuild fails for any reason, such as a failover occurring or a disk runs out of available space, then the entire process would normally need to start all over again at the beginning.

SQL Server 2017 fixes this problem by enabling an option to pause, and then later resume, online index rebuilds without losing any progress.


Managing indexes is a critical component of database maintenance but we often don’t think about the indicators behind the index maintenance operations. SQL Server 2017 introduces a very useful index feature, to mitigate the administration overhead of index maintenance which we’ll review and discuss in this article.

Here we will Discuss:-

  1. How to Enlarge AdventureWorks database
  2. Index maintenance options available in SQL 2017
  3. How to use index_resumable_operations


lets Start

First download AdventureWorks2014 database from below link:-


https://github.com/Microsoft/sql-server-samples/releases/tag/adventureworks



Restore adventureworks2014  database
Change compatibility level to 140.

and then execute script provided below, this will Enlarge AdventureWorks database..

USE AdventureWorks2014;
GO



IF OBJECT_ID('Sales.SalesOrderHeaderEnlarged') IS NOT NULL

 DROP TABLE Sales.SalesOrderHeaderEnlarged;

GO



CREATE TABLE Sales.SalesOrderHeaderEnlarged
 (
 SalesOrderID int NOT NULL IDENTITY (1, 1) NOT FOR REPLICATION,
 RevisionNumber tinyint NOT NULL,
 OrderDate datetime NOT NULL,
 DueDate datetime NOT NULL,
 ShipDate datetime NULL,
 Status tinyint NOT NULL,
 OnlineOrderFlag dbo.Flag NOT NULL,
 SalesOrderNumber  AS (isnull(N'SO'+CONVERT([nvarchar](23),[SalesOrderID],0),N'*** ERROR ***')),
 PurchaseOrderNumber dbo.OrderNumber NULL,
 AccountNumber dbo.AccountNumber NULL,
 CustomerID int NOT NULL,
 SalesPersonID int NULL,
 TerritoryID int NULL,
 BillToAddressID int NOT NULL,
 ShipToAddressID int NOT NULL,
 ShipMethodID int NOT NULL,
 CreditCardID int NULL,
 CreditCardApprovalCode varchar(15) NULL,
 CurrencyRateID int NULL,
 SubTotal money NOT NULL,
 TaxAmt money NOT NULL,
 Freight money NOT NULL,
 TotalDue  AS (isnull(([SubTotal]+[TaxAmt])+[Freight],(0))),
 Comment nvarchar(128) NULL,
 rowguid uniqueidentifier NOT NULL ROWGUIDCOL,
 ModifiedDate datetime NOT NULL
 )  ON [PRIMARY]
GO

SET IDENTITY_INSERT Sales.SalesOrderHeaderEnlarged ON
GO

INSERT INTO Sales.SalesOrderHeaderEnlarged (SalesOrderID, RevisionNumber, OrderDate, DueDate, ShipDate, Status, OnlineOrderFlag, PurchaseOrderNumber, AccountNumber, CustomerID, SalesPersonID, TerritoryID, BillToAddressID, ShipToAddressID, ShipMethodID, CreditCardID, CreditCardApprovalCode, CurrencyRateID, SubTotal, TaxAmt, Freight, Comment, rowguid, ModifiedDate)

SELECT SalesOrderID, RevisionNumber, OrderDate, DueDate, ShipDate, Status, OnlineOrderFlag, PurchaseOrderNumber, AccountNumber, CustomerID, SalesPersonID, TerritoryID, BillToAddressID, ShipToAddressID, ShipMethodID, CreditCardID, CreditCardApprovalCode, CurrencyRateID, SubTotal, TaxAmt, Freight, Comment, rowguid, ModifiedDate
FROM Sales.SalesOrderHeader WITH (HOLDLOCK TABLOCKX)
GO
SET IDENTITY_INSERT Sales.SalesOrderHeaderEnlarged OFF
GO
ALTER TABLE Sales.SalesOrderHeaderEnlarged ADD CONSTRAINT
 PK_SalesOrderHeaderEnlarged_SalesOrderID PRIMARY KEY CLUSTERED
 (
 SalesOrderID
 ) WITH( STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
CREATE UNIQUE NONCLUSTERED INDEX AK_SalesOrderHeaderEnlarged_rowguid ON Sales.SalesOrderHeaderEnlarged
 (
 rowguid
 ) WITH( STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
CREATE UNIQUE NONCLUSTERED INDEX AK_SalesOrderHeaderEnlarged_SalesOrderNumber ON Sales.SalesOrderHeaderEnlarged
 (
 SalesOrderNumber
 ) WITH( STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX IX_SalesOrderHeaderEnlarged_CustomerID ON Sales.SalesOrderHeaderEnlarged
 (
 CustomerID
 ) WITH( STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX IX_SalesOrderHeaderEnlarged_SalesPersonID ON Sales.SalesOrderHeaderEnlarged
 (
 SalesPersonID
 ) WITH( STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
IF OBJECT_ID('Sales.SalesOrderDetailEnlarged') IS NOT NULL
 DROP TABLE Sales.SalesOrderDetailEnlarged;
GO
CREATE TABLE Sales.SalesOrderDetailEnlarged
 (
 SalesOrderID int NOT NULL,
 SalesOrderDetailID int NOT NULL IDENTITY (1, 1),
 CarrierTrackingNumber nvarchar(25) NULL,
 OrderQty smallint NOT NULL,
 ProductID int NOT NULL,
 SpecialOfferID int NOT NULL,
 UnitPrice money NOT NULL,
 UnitPriceDiscount money NOT NULL,
 LineTotal  AS (isnull(([UnitPrice]*((1.0)-[UnitPriceDiscount]))*[OrderQty],(0.0))),
 rowguid uniqueidentifier NOT NULL ROWGUIDCOL,
 ModifiedDate datetime NOT NULL
 )  ON [PRIMARY]
GO
SET IDENTITY_INSERT Sales.SalesOrderDetailEnlarged ON
GO
INSERT INTO Sales.SalesOrderDetailEnlarged (SalesOrderID, SalesOrderDetailID, CarrierTrackingNumber, OrderQty, ProductID, SpecialOfferID, UnitPrice, UnitPriceDiscount, rowguid, ModifiedDate)
SELECT SalesOrderID, SalesOrderDetailID, CarrierTrackingNumber, OrderQty, ProductID, SpecialOfferID, UnitPrice, UnitPriceDiscount, rowguid, ModifiedDate
FROM Sales.SalesOrderDetail WITH (HOLDLOCK TABLOCKX)
GO
SET IDENTITY_INSERT Sales.SalesOrderDetailEnlarged OFF
GO
ALTER TABLE Sales.SalesOrderDetailEnlarged ADD CONSTRAINT
 PK_SalesOrderDetailEnlarged_SalesOrderID_SalesOrderDetailID PRIMARY KEY CLUSTERED
 (
 SalesOrderID,
 SalesOrderDetailID
 ) WITH( STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
CREATE UNIQUE NONCLUSTERED INDEX AK_SalesOrderDetailEnlarged_rowguid ON Sales.SalesOrderDetailEnlarged
 (
 rowguid
 ) WITH( STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
CREATE NONCLUSTERED INDEX IX_SalesOrderDetailEnlarged_ProductID ON Sales.SalesOrderDetailEnlarged
 (
 ProductID
 ) WITH( STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
GO
BEGIN TRANSACTION
DECLARE @TableVar TABLE
(OrigSalesOrderID int, NewSalesOrderID int)
INSERT INTO Sales.SalesOrderHeaderEnlarged
 (RevisionNumber, OrderDate, DueDate, ShipDate, Status, OnlineOrderFlag,
  PurchaseOrderNumber, AccountNumber, CustomerID, SalesPersonID, TerritoryID,
  BillToAddressID, ShipToAddressID, ShipMethodID, CreditCardID,
  CreditCardApprovalCode, CurrencyRateID, SubTotal, TaxAmt, Freight, Comment,
  rowguid, ModifiedDate)
OUTPUT inserted.Comment, inserted.SalesOrderID
 INTO @TableVar
SELECT RevisionNumber, DATEADD(dd, number, OrderDate) AS OrderDate,
  DATEADD(dd, number, DueDate),  DATEADD(dd, number, ShipDate),
 Status, OnlineOrderFlag,
  PurchaseOrderNumber,
  AccountNumber,
  CustomerID, SalesPersonID, TerritoryID, BillToAddressID,
  ShipToAddressID, ShipMethodID, CreditCardID, CreditCardApprovalCode,
  CurrencyRateID, SubTotal, TaxAmt, Freight, SalesOrderID,
  NEWID(), DATEADD(dd, number, ModifiedDate)
FROM Sales.SalesOrderHeader AS soh WITH (HOLDLOCK TABLOCKX)
CROSS JOIN (
  SELECT number
  FROM ( SELECT TOP 10 number
    FROM master.dbo.spt_values
    WHERE type = N'P'
      AND number < 1000
    ORDER BY NEWID() DESC
   UNION
    SELECT TOP 10 number
    FROM master.dbo.spt_values
    WHERE type = N'P'
      AND number < 1000
    ORDER BY NEWID() DESC
   UNION
    SELECT TOP 10 number
    FROM master.dbo.spt_values
    WHERE type = N'P'
      AND number < 1000
    ORDER BY NEWID() DESC
   UNION
    SELECT TOP 10 number
    FROM master.dbo.spt_values
    WHERE type = N'P'
      AND number < 1000
    ORDER BY NEWID() DESC
    ) AS tab
) AS Randomizer
ORDER BY OrderDate, number
INSERT INTO Sales.SalesOrderDetailEnlarged
 (SalesOrderID, CarrierTrackingNumber, OrderQty, ProductID,
  SpecialOfferID, UnitPrice, UnitPriceDiscount, rowguid, ModifiedDate)
SELECT
 tv.NewSalesOrderID, CarrierTrackingNumber, OrderQty, ProductID,
 SpecialOfferID, UnitPrice, UnitPriceDiscount, NEWID(), ModifiedDate
FROM Sales.SalesOrderDetail AS sod
JOIN @TableVar AS tv
 ON sod.SalesOrderID = tv.OrigSalesOrderID
ORDER BY sod.SalesOrderDetailID
COMMIT

/* it will take 8~10 minutes

output would be like:-


(31465 row(s) affected)

(121317 row(s) affected)

(1195670 row(s) affected)

(4610046 row(s) affected)


Resuming online index rebuilds


============================

USE [AdventureWorks2014]
GO
ALTER INDEX [PK_SalesOrderHeaderEnlarged_SalesOrderID] ON [Sales].[SalesOrderHeaderEnlarged] REBUILD PARTITION = ALL
WITH
 (PAD_INDEX = OFF,
  STATISTICS_NORECOMPUTE = OFF,
   SORT_IN_TEMPDB = OFF,
  ONLINE = ON,RESUMABLE=ON,
  ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
GO

if you want to pause


on another tab we can execute below query..



USE [AdventureWorks2014]

GO

ALTER INDEX [PK_SalesOrderHeaderEnlarged_SalesOrderID] ON [Sales].[SalesOrderHeaderEnlarged] pause



select * from sys.index_resumable_operations



concentrate on 3 column

1. State_description
2. %complete
3. sql_text



If you want to resume it again



go to another tab and execute the command



USE [AdventureWorks2014]

GO

ALTER INDEX [PK_SalesOrderHeaderEnlarged_SalesOrderID] ON [Sales].[SalesOrderHeaderEnlarged] resume



 If you want to abort


USE [AdventureWorks2014]
GO
ALTER INDEX [PK_SalesOrderHeaderEnlarged_SalesOrderID] ON [Sales].[SalesOrderHeaderEnlarged] abort

We have another option here max_duration while rebuilding index


Coming to MAX_DURATION

USE [AdventureWorks2014]
GO

ALTER INDEX [PK_SalesOrderHeaderEnlarged_SalesOrderID] ON [Sales].[SalesOrderHeaderEnlarged] REBUILD PARTITION = ALL
WITH
 (PAD_INDEX = OFF,
  STATISTICS_NORECOMPUTE = OFF,
   SORT_IN_TEMPDB = OFF,
 ONLINE = ON,RESUMABLE=ON,max_duration = 10 minutes,
  ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON)
GO


Thanks for Reading






Which sessions & SQL Text cosuming most of space on TempDB database of SQL Server

As far as TempDB database is concerned, regularly we get cases with below update in the case summary.

Could not allocate a new page for database \\'tempdb\\' because of insufficient disk space
Log Alert: Could not allocate space for object 'dbo.SORT temporary run storage: 140835581198336'

In order to resolve this issue, we generally shrink data/log file of tempdb.

it looks tome it is a  wrong approach to shrink the data/log file of tempdb.

First, we need to know which sessions consumed and any badly-written queries.

In order to find this, we need to execute below query...

SELECT
              SUM(unallocated_extent_page_count) AS [free pages],
              (SUM(unallocated_extent_page_count)*1.0/128) AS [free space in MB],
              SUM(version_store_reserved_page_count) AS [version store pages used],
             (SUM(version_store_reserved_page_count)*1.0/128) AS [version store space in MB],
             SUM(internal_object_reserved_page_count) AS [internal object pages used],
              (SUM(internal_object_reserved_page_count)*1.0/128) AS [internal object space in [MB],
             SUM(user_object_reserved_page_count) AS [user object pages used],
            (SUM(user_object_reserved_page_count)*1.0/128) AS [user object space in MB]
           FROM sys.dm_db_file_space_usage;
            Go

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

SELECT
              SUM(unallocated_extent_page_count) AS [free pages],
              (SUM(unallocated_extent_page_count)*1.0/128) AS [free space in MB],
              SUM(version_store_reserved_page_count) AS [version store pages used],
             (SUM(version_store_reserved_page_count)*1.0/128) AS [version store space in MB],
             SUM(internal_object_reserved_page_count) AS [internal object pages used],
              (SUM(internal_object_reserved_page_count)*1.0/128) AS [internal object space in [MB],
             SUM(user_object_reserved_page_count) AS [user object pages used],
            (SUM(user_object_reserved_page_count)*1.0/128) AS [user object space in MB]
           FROM sys.dm_db_file_space_usage;
            Go
SELECT R1.session_id, R1.request_id, R1.Task_request_internal_objects_alloc_page_count, R1.Task_request_internal_objects_dealloc_page_count,
R1.Task_request_user_objects_alloc_page_count,R1.Task_request_user_objects_dealloc_page_count,R3.Session_request_internal_objects_alloc_page_count ,
R3.Session_request_internal_objects_dealloc_page_count,R3.Session_request_user_objects_alloc_page_count,R3.Session_request_user_objects_dealloc_page_count,
R2.sql_handle, RL2.text as SQLText, R2.statement_start_offset, R2.statement_end_offset, R2.plan_handle FROM (SELECT session_id, request_id,
SUM(internal_objects_alloc_page_count) AS Task_request_internal_objects_alloc_page_count, SUM(internal_objects_dealloc_page_count)AS
Task_request_internal_objects_dealloc_page_count,SUM(user_objects_alloc_page_count) AS Task_request_user_objects_alloc_page_count,
SUM(user_objects_dealloc_page_count)AS Task_request_user_objects_dealloc_page_count FROM sys.dm_db_task_space_usage
GROUP BY session_id, request_id) R1 INNER JOIN (SELECT session_id, SUM(internal_objects_alloc_page_count) AS Session_request_internal_objects_alloc_page_count,
SUM(internal_objects_dealloc_page_count)AS Session_request_internal_objects_dealloc_page_count,SUM(user_objects_alloc_page_count) AS Session_request_user_objects_alloc_page_count,
SUM(user_objects_dealloc_page_count)AS Session_request_user_objects_dealloc_page_count FROM sys.dm_db_Session_space_usage
GROUP BY session_id) R3 on R1.session_id = R3.session_id
left outer JOIN sys.dm_exec_requests R2 ON R1.session_id = R2.session_id and R1.request_id = R2.request_id
OUTER APPLY sys.dm_exec_sql_text(R2.sql_handle) AS RL2
Where
Task_request_internal_objects_alloc_page_count >0 or
Task_request_internal_objects_dealloc_page_count>0 or
Task_request_user_objects_alloc_page_count >0 or
Task_request_user_objects_dealloc_page_count >0 or
Session_request_internal_objects_alloc_page_count >0 or
Session_request_internal_objects_dealloc_page_count >0 or
Session_request_user_objects_alloc_page_count >0 or
Session_request_user_objects_dealloc_page_count >0


look for the SQL Text column here, it will give an idea.

or if you want to do using extended events.. follow this link

https://www.brentozar.com/archive/2015/12/tracking-tempdb-growth-using-extended-events/


Thanks for Reading..




How to check which is a closest Azure data center from your premise

How to check which Azure Region is close to your premise.

 this particular site's pretty cool.

You get a live running list of latency measures, and it gives you your closest datacenters in a millisecond time.

The Site is:-

http://azurespeed.com/



Thanks for Reading. 

Extended Events query to get DeadLocks/ blocked_process/SQL_QueryIOStats/Error_reported in case severity greater than 10 & then Alter the Event.


  1. Extended Events query to get DeadLocks
  2. Extended Events query to get blocked_process
  3. Extended Events query to get LongRunningQuery
  4. Extended Events query to get SQL_QueryIOStats
  5. Extended Events query to get error_trap
  6. Use the system_health Session
  7. Viewing the Session Data
  8. Restoring the system_health Session 
  9. Find the Objects That Have the Most Locks Taken on Them
  10. Determine Which Queries Are Holding Locks


Extended Events query to get  DeadLocks
====================================

How to check the Running status of SQL Server SQL * services in one shot.

How to check the Running status of SQL Server  SQL * services in one shot.


Cannot connect to XXXXXlistener.database.windows.net

Issue:-

While connecting with Azure Failover groups listeners, I get following error in SQL Server 2014 SSMS.

Cannot connect to XXXXXlistener.database.windows.net.

------------------------------
ADDITIONAL INFORMATION:

Cannot open server 'sqlsouthasia' requested by the login. Client with IP address '61.62.123.64' is not allowed to access the server.  To enable access, use the Windows Azure Management Portal or run sp_set_firewall_rule on the master database to create a firewall rule for this IP address or address range.  It may take up to five minutes for this change to take effect. (Microsoft SQL Server, Error: 40615)


Resolution:-

As per suggestion, we have to execute sp_set_firewall_rule sp from master databases of the server or from Azure portal will have to add 61.62.123.64  IP address.

The query is 


exec sp_set_firewall_rule N'office', '61.62.123.64', '61.62.123.64';  

 or using Azure portal 




The issue will resolve

Thanks for reading..