Login or Sign Up to become a member!
LessThanDot Sit Logo

LessThanDot

Data Management

Less Than Dot is a community of passionate IT professionals and enthusiasts dedicated to sharing technical knowledge, experience, and assistance. Inside you will find reference materials, interesting technical discussions, and expert tips and commentary. Once you register for an account you will have immediate access to the forums and all past articles and commentaries.

LTD Social Sitings

Lessthandot twitter Lessthandot Linkedin Lessthandot friendfeed Lessthandot facebook Lessthandot rss

Note: Watch for social icons on posts by your favorite authors to follow their postings on these and other social sites.

Your profile

    Search

    XML Feeds

    Google Ads

    « Data Quality and Master Data Management with Microsoft SQL Server 2008 R2 book available as a free downloadBook Review: Securing SQL Server By Denny Cherry »
    comments

    In this blog post I want to discuss a problem I encounter frequently in SQL related forums - how to perform dynamic PIVOT involving multiple columns using pure T-SQL solution. In SQL Server Reporting Services this functionality can be easily achieved using Matrix template.

    With the introduction of PIVOT operator in SQL 2005, it became very easy to write queries that transform rows into columns. There are numerous articles on the Web as how to perform PIVOT and even dynamic PIVOT. However, most of these articles explain how to perform such queries for just one column. I want to expand the transformation for multiple columns.

    The idea of a solution is quite simple - you need to generate the SQL dynamically. Since we want to use pivot on multiple columns, we need to use CASE based pivot.

    You need to keep in mind, that resulting query should have less than 1024 columns. You may build the check for number of columns into the query.

    Now, every time I need to create a dynamic query, I need to know what is the final query I am going to arrive to. Having the idea in mind and printing the SQL command until I got it right helps to create the working query.

    Let's consider the first example based on the AdventureWorks SalesOrderHeader table.
    This code creates the table we're going to transform - it gives us summary of orders and total due per each quarter:

    1. USE AdventureWorks
    2.  
    3. SELECT   DATEPART(quarter,OrderDate) AS [Quarter],
    4.          DATEPART(YEAR,OrderDate)    AS [Year],
    5.          COUNT(SalesOrderID)         AS [Orders Count],
    6.          SUM(TotalDue)               AS [Total Due]
    7. INTO     #SalesSummary
    8. FROM     Sales.SalesOrderHeader
    9. GROUP BY DATEPART(quarter,OrderDate),
    10.          DATEPART(YEAR,OrderDate)
    11.  
    12. SELECT   *
    13. FROM     #SalesSummary
    14. ORDER BY [Year],
    15.          [Quarter]

    The #SalesSummary table lists count of orders and total due for each quarter and year.

    Quarterly Orders Summary
    QuarterYear Orders CountTotal Due
    32001 621 $5,850,932.9483
    4 2001 758 $8,476,619.278
    1 2002 741 $7,379,686.3091
    2 2002 825 $8,210,285.1655
    3 2002 1054 $13,458,206.13
    4 2002 1072 $10,827,327.4904
    1 2003 1091 $8,550,831.8702
    2 2003 1260 $10,749,269.374
    3 2003 4152 $18,220,131.5285
    4 2003 5940 $16,787,382.3141
    1 2004 6087 $14,170,982.5455
    2 2004 6888 $17,969,750.9487
    3 2004 976 $56,178.9223

    Now, suppose we want to see these results horizontally. The following dynamic SQL produces the desired output:

    1. DECLARE  @SQL  NVARCHAR(MAX),
    2.          @Cols NVARCHAR(MAX)
    3.          
    4. SELECT @Cols = STUFF((select ',
    5. SUM(CASE WHEN [Quarter]=' + CAST([Quarter] as varchar(3)) + ' AND [Year] = ' +
    6. CAST([Year] as char(4)) +
    7. ' THEN [Orders Count] ELSE 0 END) AS [' +
    8. CAST([Year] as char(4)) + '-' + CAST([Quarter] as varchar(3)) +
    9. ' Orders],
    10. SUM(CASE WHEN [Quarter]=' + CAST([Quarter] as varchar(3)) +
    11. ' AND [Year] = ' + CAST([Year] as char(4)) +
    12. ' THEN [Total Due] ELSE 0 END) AS [' + CAST([Year] as char(4)) + '-' +
    13. CAST([Quarter] as varchar(3)) + ' Sales]'
    14. FROM #SalesSummary
    15. ORDER BY [Year],[Quarter] FOR XML PATH(''),type).value('.','varchar(max)'),1,2,'')
    16.  
    17. SET @SQL = 'SELECT ' + @Cols + '  FROM #SalesSummary'
    18.  
    19. --print @SQL
    20. EXECUTE( @SQL)

    It produces the transformed result (I show only few first columns):

    Transformed result
    2001-3 Orders2001-3 Sales 2001-4 Orders2001-4 Sales 2002-1 Orders2002-1 Sales 2002-2 Orders2002-2 Sales
    621$5,850,932.9483758$8,476,619.278741 $7,379,686.3091825$8,210,285.1655

    You may want to un-comment PRINT @SQL statement if you want to see the generated command. I used XML PATH to concatenate rows into a string based on this blog post by Brad Schulz Making a list and checking it twice.

    Another interesting problem was presented in this MSDN Transact-SQL forum thread:

    For the claims table that has many integer columns, transform rows into columns. I demonstrate just the solution with the table creation script:

    1. USE tempdb
    2.  
    3. IF OBJECT_ID('Claims','U') IS NOT NULL
    4.   DROP TABLE Claims
    5.  
    6. GO
    7.  
    8. CREATE TABLE Claims (
    9.   Claim  INT,
    10.   HCPC   INT,
    11.   [mod]  INT,
    12.   charge INT,
    13.   paid   INT,
    14.   Qty    INT)
    15.  
    16. INSERT INTO Claims
    17. SELECT 12345,
    18.        99245,
    19.        90,
    20.        20,
    21.        10,
    22.        1
    23. UNION ALL
    24. SELECT 12345,
    25.        99112,
    26.        NULL,
    27.        30,
    28.        20,
    29.        1
    30. UNION ALL
    31. SELECT 12345,
    32.        99111,
    33.        80,
    34.        50,
    35.        25,
    36.        2
    37. UNION ALL
    38. SELECT 11112,
    39.        99911,
    40.        60,
    41.        50,
    42.        20,
    43.        1
    44. UNION ALL
    45. SELECT 12222,
    46.        99454,
    47.        NULL,
    48.        50,
    49.        20,
    50.        1
    51.  
    52. SELECT * FROM Claims

    The output is:

    Claims table
    ClaimHCPC modChargePaidQty
    12345 99245 90 20 10 1
    12345 99112 NULL 30 20 1
    12345 99111 80 50 25 2
    11112 99911 60 50 20 1
    12222 99454 NULL 50 20 1

    And the following code transforms the result (as you see, I am using a loop and ROW_NUMBER() approach):

    1. DECLARE  @SQL     NVARCHAR(MAX),
    2.          @Loop    INT,
    3.          @MaxRows INT
    4.  
    5. SET @Sql = ''
    6.  
    7. SELECT @MaxRows = MAX(MaxRow)
    8. FROM   (SELECT   COUNT(* ) AS MaxRow,
    9.                  Claim
    10.         FROM     Claims
    11.         GROUP BY Claim) X
    12.  
    13. SET @Loop = 1
    14.  
    15. WHILE @Loop <= @MaxRows
    16.   BEGIN
    17.     SELECT @SQL = @SQL + ',     SUM(CASE WHEN Row = ' + CAST(@Loop AS VARCHAR(10)) + ' THEN ' + QUOTENAME(Column_Name) + ' END) AS [' + COLUMN_NAME + CAST(@Loop AS VARCHAR(10)) + ']'
    18.     FROM   INFORMATION_SCHEMA.COLUMNS
    19.     WHERE  TABLE_Name = 'Claims'
    20.            AND COLUMN_NAME NOT IN ('Row','Claim')
    21.      
    22.     SET @Loop = @Loop + 1
    23.   END
    24.  
    25. --PRINT @SQL
    26. SET @SQL = 'SELECT Claim' + @SQL + ' FROM (select *,          row_number() over (partition by Claim ORDER BY Claim) as Row         FROM Claims) X GROUP BY Claim '
    27.  
    28. PRINT @SQL
    29.  
    30. EXECUTE( @SQL)
    Transformed Output
    Claim HCPC1mod1Charge1Paid1Qty1 HCPC2mod2Charge2Paid2Qty2 HCPC3mod3Charge3Paid3Qty3
    11112 99911 60 50 20 1NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL
    12222 99454 NULL 50 20 1 NULL NULL NULL NULL NULL NULL NULL NULL NULL NULL
    12345 99245 90 20 10 1 99112 NULL 30 20 1 99111 80 50 25 2

    I'll post another recent problem found in the following MSDN thread Basic Crosstab Query

    1. SET NOCOUNT ON;
    2. GO
    3. USE tempdb;
    4. GO
    5. CREATE TABLE #T (
    6. StoreID char(5) NOT NULL,
    7. CriteriaNo int NOT NULL,
    8. Result int NOT NULL,
    9. Position int NOT NULL,
    10. PRIMARY KEY (StoreID, CriteriaNo)
    11. );
    12. GO
    13. INSERT INTO #T(StoreID, CriteriaNo, Result, Position)
    14. SELECT '0001', 9, 10, 1 UNION ALL
    15. SELECT '0002', 9, 12, 2 UNION ALL
    16. SELECT '0001', 10, 5, 1 UNION ALL
    17. SELECT '0002', 10, 6, 2;
    18. GO
    19. -- dynamic
    20. DECLARE @sql nvarchar(MAX), @Cols nvarchar(max);
    21.  
    22. SELECT @Cols = (select ', ' + 'MAX(case when CriteriaNo = ' + CONVERT(varchar(20), CriteriaNo) +
    23. ' then Result else 0 end) AS CriteriaNo' + CONVERT(varchar(20), CriteriaNo) +
    24. ', MAX(case when CriteriaNo = ' + CONVERT(varchar(20), CriteriaNo) +
    25. ' then Position else 0 end) AS CriteriaPosition' + CONVERT(varchar(20), CriteriaNo)
    26. from (select distinct CriteriaNo from #T) X ORDER By CriteriaNo
    27. FOR XML PATH(''))
    28.  
    29. SET @sql = 'SELECT StoreID' + @Cols + ', SUM(Result) as PerformancePrint
    30. FROM #T
    31. GROUP BY StoreID'
    32.  
    33. EXECUTE(@sql)

    *** Remember, if you have a SQL related question, try our Microsoft SQL Server Programming forum or our Microsoft SQL Server Admin forum

    About the Author

    Naomi Nosonovsky, Senior Software Developer, has more than 15 years of enterprise experience in analysis, design, development and implementation of high-performance client/server data management solutions. She is a Personality of the Year for 2008, 2009, 2010 and 2011 at www.universalthread.com in .NET, SQL Server & VFP categories. She is also an All Star contributor/MCC Community Contributor at forums.asp.net and MSDN T-SQL forum.
    Social SitingsFacebookLinkedIndeliciousLTD RSS Feed
    12854 views
    InstapaperVote on HN

    13 comments

    Comment from: Boris [Visitor] Email
    Boris Why do you need to create dynamic sql if you already know the column names you are working with?
    05/29/11 @ 14:20
    Comment from: Naomi Nosonovsky [Member]
    Naomi Nosonovsky Good point! However, in the examples I shown I do not know the column names of the result in advance, hence I create them dynamically.
    05/29/11 @ 14:32
    Comment from: SQLDenis [Member] Email
    SQLDenis I do the same kind of thing..it would be nice if SQL Server had a dynamic crosstab query built in

    BTW a couple o weeks I built a dynamic pivot query that had over a 1000 columns for a report....fun stuff
    05/29/11 @ 17:12
    Comment from: Boris [Visitor] Email
    Boris So the first two bits of code are not at all related to the third? I found it very confusing because the first two do not need to use dynamic sql and the third is for a different query and is not explained at all.
    05/30/11 @ 01:46
    Comment from: Naomi Nosonovsky [Member]
    Naomi Nosonovsky The first code is to create a table we're going to work with. We can use either temp table or CTE or derived table. I found that Temp table performs quicker than CTE or derived tables under certain circumstances, so the first piece of code creates a table which we want to transform. The actual transformation is done in the second code. The third code is another variation of the theme, but it's slightly different than the first. In the third variation of the code we don't know the field names we're transforming, so it makes it even more complicated than the first.

    I also made some changes in the blog to make it more transparent.
    05/30/11 @ 09:39
    Comment from: Boris [Visitor] Email
    Boris So the first 2 are from SalesOrderHeader in AdventureWorks and the 3rd is a different query on a different table from a random post on MSDN? The first two dont need dynamic sql cos u know the field names but u use it for some reason, and the 1 from the random post does need it cos u dont know the names? No wonder im confused lol...
    05/30/11 @ 10:14
    Comment from: Naomi Nosonovsky [Member]
    Naomi Nosonovsky In the first problem, we don't know in advance all the quarters and years - hence dynamic query.

    We have this table

    Quarter   Year   OrdersCount     TotalDue
    Q3 2001 10 $100
    Q4 2001 50 $2332
    Q1 2002 74 $4567

    etc.

    We don't know the exact quarters and years and we want to transform all of them into a row, e.g.


    Q3-2001 Cnt Q3-2001 Sales Q4-2001 Cnt Q4-2001 Sales
    10 $100 50 $2332

    Similar to what you produce in SSRS, but I wanted to show pure T-SQL solution.

    The second problem is not related to the first, but IMHO, it demonstrates another interesting twist in constructing dynamic query. I could have used another AW example, but I already had the solution for the thread saved, so I just used the existing thread and a solution as an example.
    05/30/11 @ 10:26
    Comment from: mike [Visitor]
    mike Hi,

    The "claims" dynamic code only produces the first column for me. Can you check it and see if it is correct?? I am a beginner with dynamic sql, and not sure where to look for the error!

    thanks
    Mike
    10/26/12 @ 09:59
    Comment from: Naomi Nosonovsky [Member]
    Naomi Nosonovsky Sorry, I didn't see your comment earlier. I just re-run the code from the article as is in SQL Server 2012 and I got many columns as output. Have you copied the code exactly?
    10/30/12 @ 12:14
    Comment from: RoMe [Visitor] Email
    RoMe Hi,
    first of all thanks! As a SQL "beginner" this was of great help.
    I'm experiencing the same problem as Mike: "The "claims" dynamic code only produces the first column for me."
    I'm using sql express 2005 (by any chance could this be the problem?)
    Thanks!
    12/06/12 @ 04:16
    Comment from: Naomi Nosonovsky [Member]
    Naomi Nosonovsky Very strange. What do you see as printed SQL and did you try to execute it by itself?

    Also, what is the compatibility level of your TempDB?

    I don't have SQL 2005 to test, so I just tested this code in SQL 2012 and got 3 rows, although first two rows had NULLs in most of the columns.
    12/06/12 @ 05:40
    Comment from: krishna [Visitor]
    krishna i was wondering if we have multiple columns and have to convert them to rows how to we do that.

    eg: product1 product1 product3
    1 2 3
    now if we have to have all the columns into 1 column and their respective values into another column, how do we do it



    01/15/13 @ 08:14
    Comment from: Naomi Nosonovsky [Member]
    Naomi Nosonovsky I think in your case you just need dynamic UNPIVOT, right?

    E.g. do you want to get

    ProductName Quantity
    Product1 1
    Product2 2
    Product3 3

    but you don't know the names of products columns in advance?

    If so, that's dynamic UNPIVOT - slightly different topic than the topic of this blog post.

    I suggest you to post your question either in the SQL forum here or in MSDN Transact-SQL forum with more information and someone will be able to help you better.
    01/15/13 @ 08:28

    Leave a comment


    Your email address will not be revealed on this site.

    To mislead the spambots.

    Your URL will be displayed.
    (Line breaks become <br />)
    (Name, email & website)
    (Allow users to contact you through a message form (your email will not be revealed.)