Tuesday, November 12, 2013

In-Memory OLTP and the Identity Column

Over the past month I've been playing around with the new In-Memory OLTP (code name: "Hekaton") features within SQL Server 2014 CTP2. My organization is all about low latency applications, and this is one feature of SQL Server that I need to get familiar with ASAP.

To do this, I started my own little project that takes an existing database and converts parts of it into in-memory tables.  Once that step is complete, I could work on rewriting the TSQL code.

It might seem fairly simple, but with every new feature of SQL Server there are usually limitations. And one of the first ones I noticed was the use of an IDENTITY column. They are prohibited in Hekaton tables which means I had to find an alternative. This is where the new SEQUENCE object comes into play.

The CREATE SEQUENCE command allows you to create a user-defined numerical value that can be ascending or descending. This gives it much more flexibility than an IDENTITY column, and it's fully supported for use within an in-memory table.

Looking at the example below, we have a table with an IDENTITY value used for the OrderID column.

CREATE TABLE dbo.Orders (
     OrderID INT IDENTITY(1,1) NOT NULL
    ,OrderDate DATETIME NOT NULL
    ,CustomerID INT NOT NULL
    ,NetAmount MONEY NOT NULL
    ,Tax MONEY NOT NULL
    ,TotalAmount MONEY NOT NULL
);
GO

And we have a typical insert statement to insert a new order. Notice the IDENTITY column is not specified because it's value is automatically generated during at runtime.

INSERT INTO dbo.Orders (OrderDate,CustomerID,NetAmount,Tax,TotalAmount)
VALUES (GETDATE(),16,9.99,0.80,10.79);
GO

So how would this need to be rewritten to be turned into an in-memory table?  First we just need to create the table without the IDENTITY value.

CREATE TABLE dbo.Orders (
     OrderID INT NOT NULL
    ,OrderDate DATETIME NOT NULL
    ,CustomerID INT NOT NULL
    ,NetAmount MONEY NOT NULL
    ,Tax MONEY NOT NULL
    ,TotalAmount MONEY NOT NULL
);
GO

Then we'll need to create a SEQUENCE that produces the same order of values as the IDENTITY. In our example, it starts and 1 and increments by one.

CREATE SEQUENCE dbo.CountBy1 AS INT
    START WITH 1
    INCREMENT BY 1;
GO

The insert statement will look a little different, because we'll need to call the NEXT VALUE FOR function for the SEQUENCE we just created.

INSERT INTO dbo.Orders (OrderID,OrderDate,CustomerID,NetAmount,Tax,TotalAmount)
VALUES (NEXT VALUE FOR dbo.CountBy1,GETDATE(),16,9.99,0.80,10.79);
GO

You could also generate the next sequence number ahead of time and then insert the value in a later statement.

DECLARE @NextValue INT = NEXT VALUE FOR dbo.CountBy1;

-- Do some other stuff here then insert --

INSERT INTO dbo.Orders (OrderID,OrderDate,CustomerID,NetAmount,Tax,TotalAmount)
VALUES (@NextValue,GETDATE(),16,9.99,0.80,10.79);
GO

So far, I think Microsoft has done a great job with the new Hekaton feature. They are definitely marketing it as a feature to implement with little to no changes in code, but I think that really depends on the existing code.  This is very basic rewrite, but one that only took a few minutes to implement.

Check out Books Online for more detailed information about both Hekaton and Sequence Numbers.

Tuesday, October 8, 2013

TSQL Tuesday #47 - Your Best SQL Server SWAG

The host for T-SQL Tuesday #47 is Kendal Van Dyke (blog|twitter), and his topic of choice is about the best SQL Server SWAG we ever received at a conference; specifically, the “good stuff”.
I’ve been doing a lot of work with SQL Server over the years, but I’ve only had the opportunity to attend the big conferences a few times. As a matter of fact, next week will be my first time attending the SQL PASS Summit. We're supposed to talk about the “good stuff” and not any of the “cheap tchotchkes” that are given away by the vendors, but I feel that I really have to include both.


First, I’d like to talk about a piece of swag that I received while at the SQL Server Connection conference in Las Vegas in November 2007. This wasn't my first trip to Las Vegas, but it was my first conference in there. And to make it better, one of my best friends from college was attending the same conference. So you could only imagine the fun I had while in Las Vegas with “college buddy”. In the vendor area, one of the representatives from Microsoft was handing out koozies with SQL Server 2008 printed on the side. These were not your normal koozies. They were slap koozies!


I actually own two other slap koozies, but this one was definitely going to be my new favorite. Like I said, it's cheap, but I love it, and it's great conversation starter.


Now let’s talk about the good stuff. 
The date was November 7, 2005.  The location was the Moscone Center in San Francisco, CA. The event was the launch party for SQL Server 2005, Visual Studio 2005, .NET Framework 2.0, and BizTalk Server 2006. Microsoft had just spent years rewriting SQL Server, and now they were throwing this elaborate party to celebrate the release. Unlike a week-long conference, this one-day event was completely free. I was living in San Francisco at the time, so it made it really easy to get to this event. All I had to do was hop on my scooter and head downtown. Microsoft didn’t disappoint for their launch party. The event boasted some big headliners. Microsoft CEO Steve Ballmer gave the keynote speech.


The live entertainment was also exciting.  The headliner band was Cheap Trick.  Although not at the height of their popularity, they are a talented rock band in any day.


There was also another all girl cover band that played nothing but AC/DC music.  They were called AC/DShe. Quite a catchy name.


The other highlight of the night was the presence of Paul Teutul, Sr. from the Orange County Choppers show on the Discovery Channel. Not only was Paul Sr. there hanging out in the crowd taking pictures with the attendees, but his team build a chopper for Microsoft with the SQL Server logo on it.



So finally to the swag. Each attendee was given a free copy of SQL Server 2005 Standard Edition and Visual Studio 2005 Professional Edition. Most software given away at conferences are evaluation or time-bombed copy, but these were fully licensed copies. In 2005, this was probably $1000 worth of software that was now mine.


It may sound anticlimactic, but for a guy on a shoe-string budget, living in one of the most expensive cities in the country, this was definitely the best swag I’ve ever received.

Tuesday, September 24, 2013

One Year Later

Wow!  It’s been one year since I launched my blog, and my how things have changed.

Accomplishments Over the Past Year
I’ve had a chance to interact with a lot of people relating to many of the posts on my blog, and even run into a few people that said “Hey I know you through your blog”. I’ve gotten much more involved in the #sqlfamily through Twitter, Stackexchange, as well as through my local SQL Server user group. Although I’ve attended meetings at my local group off and on over the past several years, I am now making a specific point to attend every meeting for both the Charlotte SQL Server User Group and the Charlotte BI Group. I’ve attended SQL Saturday’s. I’ve moved into a new job at my company, where I am now responsible for platform engineering of SQL Server for Wells Fargo Securities. I’ve gone from outdated MCP certifications to the more current MCITP: Database Administrator 2008. And most importantly, the Atlanta Braves won their first division title since 2005.

The Roadmap for the Upcoming Year
I plan to keep writing about SQL Server through my blog, as well as continue learning about SQL Server through reading other blogs. That’s one thing I learned quickly about blogging. The more I wrote about SQL Server, the more I have read. My wife keeps telling me “For someone who hates to read, you sure do read a lot."

I had hoped to eventually get to an MCM certification, but Microsoft derailed that recently by terminating the program. So for now, I’ll continue on with the development exams for SQL Server 2008 and then move to upgrade them to the SQL Server 2012 MCSE: Data Platform. For my new job, I’m not required to have certifications, but I do need to have a more holistic view of SQL Server, rather than have a more narrow view on just the database engine. Studying for the certifications has helped in those areas that I’m less familiar with, such as Analysis Services.

In just a few more weeks I’ll be attending my first SQL PASS Summit. I have been so excited about this ever since I found out it will be hosted in my town, Charlotte, NC. The Charlotte Convention Center is right next door to where I work, and I’m obviously familiar with the surrounding area. I’ve been to the DEV Connections conference in Las Vegas before, but this will be my first PASS Summit.

I also hope to start speaking at local events. I already do this within my company, so now I want to venture out and do it in a more public arena. I might start with my local user group and move up to SQL Saturdays and beyond.

I also want to make sure I set aside plenty of time for my own family. My wife has been incredibly supportive in my blogging, attending user group meetings, and studying for certifications. I want her to know how much I’m indebted to her.

Thanks to all who have read my blog, and I hope I can continue to provide quality information.


Go Braves!
</ <| <\ <| </ <| <\ <| </ <| <\ <|
(That’s the tomahawk chop)

Tuesday, September 3, 2013

The Case of the NULL Query_Plan

As a DBA, we're often asked to troubleshoot performance issues for stored procedures.  One of the most common tools at our disposal is the query execution plan cached in memory by SQL Server. Once we have the query plan, we can dissect what SQL Server is doing and hopefully find some places for improvement.

Grabbing the actual XML query plan for a stored procedure from the cache is fairly easy using the following query.

USE AdventureWorks2012;
GO
SELECT qp.query_plan FROM sys.dm_exec_procedure_stats ps
    JOIN sys.objects o ON ps.object_id = o.object_id
    JOIN sys.schemas s ON o.schema_id = s.schema_id
    CROSS APPLY sys.dm_exec_query_plan(ps.plan_handle) qp
WHERE ps.database_id = DB_ID()
    AND s.name = 'dbo'
    AND o.name = 'usp_MonsterStoredProcedure';
GO


From this point, we can open the XML query plan in Management Studio or Plan Explorer to start our investigation. But what happens if SQL Server returns NULL for the query plan?


Let's back up a little bit.  We were pretty sure the query plan is still in cache, right?  Let's verify it.

USE AdventureWorks2012;
GO
SELECT * FROM sys.dm_exec_procedure_stats ps
    JOIN sys.objects o on ps.object_id = o.object_id
WHERE o.name = 'usp_MonsterStoredProcedure';
GO

Sure enough.  The query plan is still cached in memory, and we even can even see the plan_handle.


So why did our first query not return the XML plan?  Let's copy the plan_handle and manually run it through the sys.dm_exec_query_plan function.

SELECT * FROM sys.dm_exec_query_plan(0x05000500DD93100430BFF0750100000001000000000000000000000000000000000000000000000000000000);
GO

Why are we getting NULL returned for the XML query plan when we know is in the cache?  In this case, because the query plan is so large and complex, we're hitting an XML limitation within SQL Server.  "XML datatype instance has too many levels of nested nodes. Maximum allowed depth is 128 levels".  

Let's try to pull the text version of query plan.
SELECT * FROM sys.dm_exec_text_query_plan(0x05000500DD93100430BFF0750100000001000000000000000000000000000000000000000000000000000000,DEFAULT,DEFAULT);
GO


It looks as though we have solved the issue; however, we didn't.  Management Studio has a 65535 character limit in grid results and 8192 character limit in text results.  Our query plan has been truncated far from the end.  Now it seems we are back to square one.  

We still know the query plan is in cache, but we just need a tool other than Management Studio to retrieve it.  This is where Powershell enters the picture.

With Powershell, we can create a simple script to execute the sys.dm_exec_text_query_plan function and then output the data to a file.  All we need is to pass two variables.  The first is the SQL Server name where the plan is cached, and the second is the plan_handle. 

param (    
    [Parameter(Mandatory=$true)]
    [ValidateNotNullOrEmpty()]
    [string]
    $SqlInstance

   ,[Parameter(Mandatory=$true)]
    [ValidateNotNullOrEmpty()]
    [string]
    $PlanHandle
)

The script will simply execute a TSQL script and capture the output into a string variable.
$SqlCommand = "SELECT query_plan FROM sys.dm_exec_text_query_plan(" 
    + $PlanHandle + ",DEFAULT,DEFAULT);"
$QueryPlanText = $cmd.ExecuteScalar()

The final step will use System.IO.StreamWriter() to output the data to a file.
$stream = New-Object System.IO.StreamWriter($FileName)
$stream.WriteLine($QueryPlanText)


The Powershell script will save the entire XML query plan to a file named output.sqlplan.  As you can see below, the actual plan was over 5MB.


Finally we're able to view the entire query plan in our favorite tool and see the complexity of the stored procedure.


This is just another example of why  DBAs need to set aside some time to learn Powershell.  The entire script is posted below.  Feel free to modify it as needed to fit your environment.

######################################################################################
#
#   File Name:    Get-QueryPlan.ps1
#
#   Applies to:   SQL Server 2008
#                 SQL Server 2008 R2
#                 SQL Server 2012
#
#   Purpose:      Used to retrieve an XML query plan from cache.
#
#   Prerequisite: Powershell must be installed.
#                 SQL Server components must be installed.
#
#   Parameters:   [string]$SqlInstance - SQL Server name (Ex: SERVER\INSTANCE)
#                 [string]$PlanHandle - Binary query handle
#
#   Author:       Patrick Keisler
#
#   Version:      1.0.0
#
#   Date:         08/30/2013
#
#   Help:         http://www.patrickkeisler.com/
#
######################################################################################

#Define input parameters
param ( 
  [Parameter(Mandatory=$true)]
  [ValidateNotNullOrEmpty()]
  [string]
  $SqlInstance
  
  ,[Parameter(Mandatory=$true)]
  [ValidateNotNullOrEmpty()]
  [string]
  $PlanHandle
  )

Write-Host "Script starting."

#Grab the path where the Powershell script was executed from.
$path = Split-Path $MyInvocation.MyCommand.Path

#Build the SQL Server connection objects
$conn = New-Object System.Data.SqlClient.SqlConnection
$builder = New-Object System.Data.SqlClient.SqlConnectionStringBuilder
$cmd = New-Object System.Data.SqlClient.SqlCommand

#Build the TSQL statement & connection string
$SqlCommand = "SELECT query_plan FROM sys.dm_exec_text_query_plan(" + $PlanHandle + ",DEFAULT,DEFAULT);"
$builder.psBase.DataSource = $SqlInstance
$builder.psBase.InitialCatalog = "master"
$builder.psBase.IntegratedSecurity = $true
$builder.psBase.ApplicationName = "Get-QueryPlan"
$builder.psBase.Pooling = $true
$builder.psBase.ConnectTimeout = 15
$conn.ConnectionString = $builder.ConnectionString
$cmd.Connection = $conn
$cmd.CommandText = $SqlCommand

try
{
 if ($conn.State -eq "Closed")
 {
  #Open a connection to SQL Server
  $conn.Open()
 }
 
 #Execute the TSQL statement
 [string]$QueryPlanText = $cmd.ExecuteScalar()

 #Write the output to a file
 $FileName = $path + "\output.sqlplan"
 $stream = New-Object System.IO.StreamWriter($FileName)
 $stream.WriteLine($QueryPlanText)

 if ($stream.BaseStream -ne $null)
 {
  #Close the stream object
  $stream.close()
 }

 if ($conn.State -eq "Open")
 {
  #Close the SQL Server connection
  $conn.Close()
 }
 
 Write-Host "Script completed successfully."
}
catch
{
 #Capture errors if needed
 if ($_.Exception.InnerException)
 {
  $Host.UI.WriteErrorLine("ERROR: " + $_.Exception.InnerException.Message)
  if ($_.Exception.InnerException.InnerException)
  {
   $Host.UI.WriteErrorLine("ERROR: " + $_.Exception.InnerException.InnerException.Message)
  }
 }
 else
 {
  $Host.UI.WriteErrorLine("ERROR: " + $_.Exception.Message)
 }
 
 Write-Host .
 Write-Host "ERROR: Script failed."
}

Tuesday, August 27, 2013

How to Tell If Your Users are Connecting to the Availability Group Listener

You've spent a lot of time planning and building out a new SQL Server 2012 environment complete with Availability Group Listeners, but how can you be sure the end users are connecting to the listener and not directly to the SQL Server instance?

So why would we care about this?  To begin with, if the users are not connecting to the listener, then upon a failover to another replica, those users would have to connect to a different SQL Server instance name.  Having a single point of connection is crucial for the high availability process to work correctly.

In a previous blog post, we setup an Availability Group Listener, AdventureWorks.mcp.com, with two IP addresses:   192.168.1.55 & 192.168.2.55.  We'll use this one for our example.


The DMV, sys.dm_exec_connections, contains information about each connection to a SQL Server instance, and can be used to answer our question.

Open a TSQL connection to either the Availability Group listener, and execute the following command.

SELECT
     session_id
    ,local_net_address
    ,local_tcp_port
FROM sys.dm_exec_connections;
GO


The local_net_address and local_tcp_port columns will display the IP address and port number of the client's connection target.  This will be the connection string the users entered to connect to the SQL Server instance.

If the IP address and port number match the Availability Group IP, then you're in good shape.  If they do not match, then some users are likely connecting directly to the SQL Server instance, and that will need to be changed.

By joining the sys.dm_exec_sessions DMV, you'll also be able to get the hostname and program name of each connection.

SELECT
     ec.session_id
    ,es.host_name
    ,es.program_name
    ,local_net_address
    ,local_tcp_port
FROM sys.dm_exec_connections ec
    JOIN sys.dm_exec_sessions es ON ec.session_id = es.session_id;
GO


As you can see in this picture, we have one connection on session_id 62 that is connecting directly to the SQL Server instance and not the to the Availability Group Listener.  At this point, I would track down that user, and have them use the correct connection string.

Using this DMV will allow you to verify the users are connecting to SQL Server using the correct connection strings, and help prevent unneeded outages during a failover between replicas.