Coalesce in SQL Server
Content
The Many Uses of Coalesce in SQL Server
Problem Many times people come across the Coalesce function and think that it is just a more powerful form of ISNULL. In actuality, I have found it to be one of the most useful functions with the least documentation. In this tip, I will show you the basic use of Coalesce and also some features you probably never new existed. Solution Let's start with the documented use of coalesce. According to MSDN, coalesce returns the first non-null expression among its arguments. For example,
SELECT COALESCE(NULL, NULL, NULL, GETDATE())
will return the current date. It bypasses the first NULL values and returns the first non-null value. Using Coalesce to Pivot If you run the following statement against the AdventureWorks database
SELECT Name FROM HumanResources.Department WHERE (GroupName = 'Executive General and Administration')
you will come up with a standard result set such as this.
If you want to pivot the data you could run the following command.
DECLARE @DepartmentName VARCHAR(1000) SELECT @DepartmentName = COALESCE(@DepartmentName,'') + Name + ';' FROM HumanResources.Department WHERE (GroupName = 'Executive General and Administration') SELECT @DepartmentName AS DepartmentNames
and get the following result set.
Using Coalesce to Execute Multiple SQL Statements Once you can pivot data using the coalesce statement, it is now possible to run multiple SQL statements by pivoting the data and using a semicolon to separate the operations. Let's say you want to find the values for any column in the Person schema that has the column name “Name”. If you execute the following script it will give you just that.
DECLARE @SQL VARCHAR(MAX) CREATE TABLE #TMP (Clmn VARCHAR(500), Val VARCHAR(50)) SELECT @SQL=COALESCE(@SQL,'')+CAST('INSERT INTO #TMP Select ''' + TABLE_SCHEMA + '.' + TABLE_NAME + '.' + COLUMN_NAME + ''' AS Clmn, Name FROM ' + TABLE_SCHEMA + '.[' + TABLE_NAME + '];' AS VARCHAR(MAX)) FROM INFORMATION_SCHEMA.COLUMNS JOIN sysobjects B ON INFORMATION_SCHEMA.COLUMNS.TABLE_NAME = B.NAME WHERE COLUMN_NAME = 'Name' AND xtype = 'U' AND TABLE_SCHEMA = 'Person' PRINT @SQL EXEC(@SQL) SELECT * FROM #TMP DROP TABLE #TMP
here is the result set.
My personal favorite is being able to kill all the transactions in a database using three lines of code. If you have ever tried to restore a database and could not obtain exclusive access, you know how useful this can be.
DECLARE @SQL VARCHAR(8000) SELECT @SQL=COALESCE(@SQL,'')+'Kill '+CAST(spid AS VARCHAR(10))+ '; ' FROM sys.sysprocesses WHERE DBID=DB_ID('AdventureWorks') PRINT @SQL --EXEC(@SQL) Replace the print statement with exec to execute
will give you a result set such as the following.
Next Steps
• •
Whenever I think I may need a cursor, I always try to find a solution using Coalesce first. I am sure I just scratched the surface on the many ways this function can be used. Go try and see what all you can come up with. A little innovative thinking can save several lines of code.
Coalesce returns the first non-null expression among its arguments. Lets say we have to return a non-null from more than one column, then we can use COALESCE function.
SELECT COALESCE(hourly_wage, salary, commission) AS 'Total Salary' FROM wages
In this case, If hourly_wage is not null and other two columns are null then hourly_wage will be returned. If hourly_wage, commission are null and salary is not null then salary will be returned. If commission is non-null and other two columns are null then commission will be returned.
Using COALESCE to Build Comma-Delimited String
Garth is back with another article. This one talks about building a comma-separated value string for use in HTML SELECT tags. It's also handy anytime you need to turn multiple records into a CSV field. It's a little longer and has some HTML but a good read.
I was reading the newsgroups a couple of days ago and came across a solution posted by Itzik Ben-Gan I thought was really smart. In order for you to understand why I like it so much I have to give you a little background on the type of
applications I work on. Most of my projects for the past 2.5 years have focused on developing browser-based applications that access data stored in SQL Server. On almost all the projects I have worked on there has been at least one Add/Edit screen that contained a multi-select list box.
For those of you with limited experience working with HTML, I need to explain that the values selected in a multi-select list box are concatenated in a comma-delimited string. The following HTML creates a multi-select list box that displays retail categories.
<SELECT name="RetailCategory" multiple> <OPTION value=1>Shoes <OPTION value=2>Sporting Goods <OPTION value=3>Restaurant <OPTION value=4>Women's Clothes <OPTION value=5>Toys </SELECT>
If a user selects more than one option the value associated with RetailCategory is a commadelimited string. For example, if the user selects Shoes, Women's Clothes and Toys, the value associate with RetailCategory is 1, 4, 5. When the user Submits their form I call a stored procedure to insert the data into the appropriate tables. The comma-delimited string is processed with a WHILE loop and the individual values are inserted into a dependent table. Now that we have covered the Add part, let's take a look at what happens when a user wants to Edit a row. When editing a row, you need to populate the form with the existing values--this includes making sure all the multi-select list box values that are associated with the row are shown as selected. To show an option as selected, you use the "selected" attribute. If we were editing the row associated with the previous example the final HTML would look like the code shown here.
<SELECT name="RetailCategory" multiple> <OPTION value=1 selected>Shoes <OPTION value=2>Sporting Goods <OPTION value=3>Restaurant <OPTION value=4 selected>Women's Clothes <OPTION value=5 selected>Toys </SELECT>
I say final, because the actual HTML is built on-the-fly using VBScript. To determine which options are shown as selected, you must return a comma-delimited string to IIS so you can manipulate it with VBScript. I use the Split function and a For loop to determine which options should be shown as selected. The following VBScript shows how this is done.
<% EmpArray = Split(rs("EmployeeList")) For Each i In EmpArray If rs2("Emp_UniqueID") = CInt(i) Then response.write "selected" End If Next
%>
The remainder of this article shows the inefficient way I used to build the string along with the new, efficient way I learned from the newsgroup posting.
The Old, Inefficient Approach
Let's create and populate some tables so we have some data to work with. Assume you have a sales effort management (SEM) system that allows you to track the number of sales calls made on a potential client. A sales call is not a phone call, but a get together such as lunch or another type of person-to-person meeting. One of the things the VP of Sales wants to know is how many of his sales personnel participate in a call. The following tables allow you to track this information.
CREATE TABLE Employees ( Emp_UniqueID smallint PRIMARY KEY, Emp_FName varchar(30) NOT NULL, Emp_LName varchar(30) NOT NULL, ) go CREATE TABLE SalesCalls ( SalCal_UniqueID smallint PRIMARY KEY, SalCal_Desc varchar(100) NOT NULL, SalCal_Date smalldatetime NOT NULL, ) go CREATE TABLE SalesCallsEmployees ( SalCal_UniqueID smallint NOT NULL, Emp_UniqueID smallint NOT NULL, ) go
A limited number of columns are used in order to make this article easier to digest. The SalesCallsEmployees table is a junction table (aka associative table) that relates the employees (sales personnel) to a particular sales call. Let's populate the tables with sample data using these INSERT statements.
INSERT INSERT INSERT INSERT Employees Employees Employees Employees VALUES VALUES VALUES VALUES (1,'Jeff','Bagwell') (2,'Jose','Lima') (3,'Chris','Truby') (4,'Craig','Biggio')
INSERT SalesCalls VALUES (1,'Lunch w/ John Smith','01/21/01') INSERT SalesCalls VALUES (2,'Golfing w/ Harry White','01/22/01') INSERT SalesCallsEmployees VALUES (1,1) INSERT SalesCallsEmployees VALUES (1,2)
INSERT SalesCallsEmployees VALUES (1,4) INSERT SalesCallsEmployees VALUES (2,2)
The first sales call (Lunch w/ John Smith) had three employees participate. Using the old approach, I used the code shown here (inside a stored procedure) to build the comma-delimited string. The resultset shows the output when the "Lunch w/ John Smith" sales call is edited.
DECLARE @Emp_UniqueID int, @EmployeeList varchar(100) SET @EmployeeList = '' DECLARE crs_Employees CURSOR FOR SELECT Emp_UniqueID FROM SalesCallsEmployees WHERE SalCal_UniqueID = 1 OPEN crs_Employees FETCH NEXT FROM crs_Employees INTO @Emp_UniqueID WHILE @@FETCH_STATUS = 0 BEGIN SELECT @EmployeeList = @EmployeeList+CAST(@Emp_UniqueID AS varchar(5))+ ', ' FETCH NEXT FROM crs_Employees INTO @Emp_UniqueID END SET @EmployeeList = SUBSTRING(@EmployeeList,1,DATALENGTH(@EmployeeList)-2) CLOSE crs_Employees DEALLOCATE crs_Employees SELECT @EmployeeList
--Results---------1, 2, 4
This code may look a little complicated, but all it's doing is creating a cursor that holds the Emp_UniqueID values associated with the sales call and processing it with a WHILE to build the string. The important thing for you to note is that this approach takes several lines of code and uses a cursor. In general, cursors are considered evil and should only be used as a last resort.
The New and Improved Approach
The new and improved approach can create the same resultset with a single SELECT statement. The following code shows how it's done.
DECLARE @EmployeeList varchar(100) SELECT @EmployeeList = COALESCE(@EmployeeList + ', ', '') + CAST(Emp_UniqueID AS varchar(5))
FROM SalesCallsEmployees WHERE SalCal_UniqueID = 1 SELECT @EmployeeList
--Results---------1, 2, 4
Should I use COALESCE() or ISNULL()?
As with many technology questions involving roughly equivalent choices, it depends. There are a variety of minor differences between COALESCE() and ISNULL(): COALESCE() is ANSI standard, so that is an advantage for the purists out there.
Many consider ISNULL()'s readability and common sense naming to be an advantage. While I will agree that it easier to spell and pronounce, I disagree that its naming is intuitive. In other languages such as VB/VBA/VBScript, ISNULL() accepts a single input and returns a single boolean output. ISNULL() accepts exactly two parameters. If you want to take the first non-NULL among more than two values, you will need to nest your ISNULL() statements. COALESCE(), on the other hand, can take multiple inputs: SELECT ISNULL(NULL, NULL, 'foo') -- yields: Server: Msg 174, Level 15, State 1, Line 1 The isnull function requires 2 arguments. SELECT COALESCE(NULL, NULL, 'foo') -- yields: ---foo
In order to make this work with ISNULL(), you would have to say: SELECT ISNULL(NULL, ISNULL(NULL, 'foo'))
The result of ISNULL() always takes on the datatype of the first parameter (regardless of whether it is NULL or NOT NULL). COALESCE works more like a CASE expression, which returns a single datatype depending on precendence and accommodating all possible outcomes. For example:
DECLARE @foo VARCHAR(5) SET @foo = NULL SELECT ISNULL(@foo, '123456789') -- yields: ----12345 SELECT COALESCE(@foo, '123456789') -- yields: --------123456789
This gets more complicated if you start mixing incompatible datatypes, e.g.: DECLARE @foo VARCHAR(5), @bar INT SET @foo = 'foo' SET @bar = NULL SELECT ISNULL(@foo, @bar) SELECT COALESCE(@foo, @bar) -- yields: ----foo Server: Msg 245, Level 16, State 1, Line 6 Syntax error converting the varchar value 'foo' to a column of data type int.
A relatively scarce difference is the ability to apply constraints to computed columns that use COALESCE() or ISNULL(). SQL Server views a column created by COALESCE() as nullable, whereas one using ISNULL() is not. So: CREATE TABLE dbo.Try ( col1 INT, col2 AS COALESCE(col1, 0) PRIMARY KEY ) GO -- yields: Server: Msg 8111, Level 16, State 2, Line 1 Cannot define PRIMARY KEY constraint on nullable column in table
'Try'. Server: Msg 1750, Level 16, State 1, Line 1 Could not create constraint. See previous errors.
Whereas the following works successfully: CREATE TABLE dbo.Try ( col1 INT, col2 AS ISNULL(col1, 0) PRIMARY KEY ) GO
If you are using COALESCE() and or ISNULL() as a method of allowing optional parameters into your WHERE clause, please see Article #2348 for some useful information (the most common techniques will use a scan, but the article shows methods that will force a more efficient seek). Finally, COALESCE() can generate a less efficient plan in some cases, for example when it is used against a subquery. Take the following example in Pubs and compare the execution plans: USE PUBS GO SET SHOWPLAN_TEXT ON GO SELECT COALESCE ( (SELECT a2.au_id FROM pubs..authors a2 WHERE a2.au_id = a1.au_id), '' ) FROM authors a1 SELECT ISNULL ( (SELECT a2.au_id FROM pubs..authors a2 WHERE a2.au_id = a1.au_id), '' ) FROM authors a1 GO
SET SHOWPLAN_TEXT OFF GO
Notice the extra work that COALESCE() has to do? This may not be a big deal against this tiny table in Pubs, but in a bigger environment this can bring servers to their knees. And no, this hasn't been made any more efficient in SQL Server 2005, you can reproduce the same kind of plan difference in AdventureWorks: USE AdventureWorks GO SET SHOWPLAN_TEXT ON GO SELECT COALESCE ( (SELECT MAX(Name) FROM Sales.Store s2 WHERE s2.name = s1.name), '' ) FROM Sales.Store s1 SELECT ISNULL ( (SELECT MAX(Name) FROM Sales.Store s2 WHERE s2.name = s1.name), '' ) FROM Sales.Store s1 GO SET SHOWPLAN_TEXT OFF GO
How do I find a stored procedure containing <text>?
I see this question at least once a week. Usually, people are trying to find all the stored procedures that reference a specific object. While I think that the best place to do this kind of searching is through your source control tool (you do keep your database objects in source control, don't you?), there are certainly some ways to do this in the database. Let's say you are searching for 'foobar' in all your stored procedures. You can do this using the
INFORMATION_SCHEMA.ROUTINES view, or syscomments: SELECT ROUTINE_NAME, ROUTINE_DEFINITION FROM INFORMATION_SCHEMA.ROUTINES WHERE ROUTINE_DEFINITION LIKE '%foobar%' AND ROUTINE_TYPE='PROCEDURE' If you want to present these results in ASP, you can use the following code, which also highlights the searched string in the body (the hW function is based on Article #2344): <% set conn = CreateObject("ADODB.Connection") conn.Open = "<connection string>" ' String we're looking for: str = "foobar" sql = "SELECT ROUTINE_NAME, ROUTINE_DEFINITION " & _ "FROM INFORMATION_SCHEMA.ROUTINES " & _ "WHERE ROUTINE_DEFINITION LIKE '%" & str & "%' " & _ " AND ROUTINE_TYPE='PROCEDURE'" set rs = conn.execute(sql) if not rs.eof then do while not rs.eof s = hW(str, server.htmlEncode(rs(1))) s = replace(s, vbTab, " ") s = replace(s, vbCrLf, "<br>") response.write "<b>" & rs(0) & "</b><p>" & s & "<hr>" rs.movenext loop else response.write "No procedures found." end if function hW(strR, tStr) w = len(strR) do while instr(lcase(tStr), lcase(strR)) > 0 cPos = instr(lcase(tStr), lcase(strR)) nStr = nStr & _ left(tStr, cPos - 1) & _ "<b>" & mid(tStr, cPos, w) & "</b>" tStr = right(tStr, len(tStr) - cPos - w + 1) loop hW = nStr & tStr end function rs.close: set rs = nothing
conn.close: set conn = nothing %> Another way to perform a search is through the system table syscomments: SELECT OBJECT_NAME(id) FROM syscomments WHERE [text] LIKE '%foobar%' AND OBJECTPROPERTY(id, 'IsProcedure') = 1 GROUP BY OBJECT_NAME(id) Now, why did I use GROUP BY? Well, there is a curious distribution of the procedure text in system tables if the procedure is greater than 8KB. So, the above makes sure that any procedure name is only returned once, even if multiple rows in or syscomments draw a match. But, that begs the question, what happens when the text you are looking for crosses the boundary between rows? Here is a method to create a simple stored procedure that will do this, by placing the search term (in this case, 'foobar') at around character 7997 in the procedure. This will force the procedure to span more than one row in syscomments, and will break the word 'foobar' up across rows. Run the following query in Query Analyzer, with results to text (CTRL+T): SET NOCOUNT ON SELECT 'SELECT '''+REPLICATE('x', 7936)+'foobar' SELECT REPLICATE('x', 500)+'''' This will yield two results. Copy them and inject them here: CREATE PROCEDURE dbo.x AS BEGIN SET NOCOUNT ON << put both results on this line >> END GO Now, try and find this stored procedure in INFORMATION_SCHEMA.ROUTINES or syscomments using the same search filter as above. The former will be useless, since only the first 8000 characters are stored here. The latter will be a little more useful, but initially, will return 0 results because the word 'foobar' is broken up across rows, and does not appear in a way that LIKE can easily find it. So, we will have to take a slightly more aggressive approach to make sure we find this procedure. Your need to do this, by the way, will depend partly on your desire for thoroughness, but more so on the ratio of stored procedures you have that are greater than 8KB. In all the systems that I manage, I don't have more than a handful that approach this size, so this isn't something I reach for very often. Maybe for you it will be more useful.
First off, to demonstrate a little better (e.g. by having more than one procedure that exceeds 8KB), let's create a second procedure just like above. Let's call it dbo.y, but this time remove the word 'foobar' from the middle of the SELECT line. CREATE PROCEDURE dbo.y AS BEGIN SET NOCOUNT ON << put both results on this line, but replace 'foobar' with something else >> END GO Basically, what we're going to do next is loop through a cursor, for all procedures that exceed 8KB. We can get this list as follows: SELECT OBJECT_NAME(id) FROM syscomments WHERE OBJECTPROPERTY(id, 'IsProcedure') = 1 GROUP BY OBJECT_NAME(id) HAVING COUNT(*) > 1 We'll need to create a work table to hold the results as we loop through the procedure, and we'll need to use UPDATETEXT to append each row with the new 8000-or-less chunk of the stored procedure code. -- create temp table CREATE TABLE #temp ( Proc_id INT, Proc_Name SYSNAME, Definition NTEXT ) -- get the names of the procedures that meet our criteria INSERT #temp(Proc_id, Proc_Name) SELECT id, OBJECT_NAME(id) FROM syscomments WHERE OBJECTPROPERTY(id, 'IsProcedure') = 1 GROUP BY id, OBJECT_NAME(id) HAVING COUNT(*) > 1 -- initialize the NTEXT column so there is a pointer UPDATE #temp SET Definition = '' -- declare local variables DECLARE
@txtPval binary(16), @txtPidx INT, @curName SYSNAME, @curtext NVARCHAR(4000) -- set up a cursor, we need to be sure this is in the correct order -- from syscomments (which orders the 8KB chunks by colid) DECLARE c CURSOR LOCAL FORWARD_ONLY STATIC READ_ONLY FOR SELECT OBJECT_NAME(id), text FROM syscomments s INNER JOIN #temp t ON s.id = t.Proc_id ORDER BY id, colid OPEN c FETCH NEXT FROM c INTO @curName, @curtext -- start the loop WHILE (@@FETCH_STATUS = 0) BEGIN -- get the pointer for the current procedure name / colid SELECT @txtPval = TEXTPTR(Definition) FROM #temp WHERE Proc_Name = @curName -- find out where to append the #temp table's value SELECT @txtPidx = DATALENGTH(Definition)/2 FROM #temp WHERE Proc_Name = @curName -- apply the append of the current 8KB chunk UPDATETEXT #temp.definition @txtPval @txtPidx 0 @curtext FETCH NEXT FROM c INTO @curName, @curtext END -- check what was produced SELECT Proc_Name, Definition, DATALENGTH(Definition)/2 FROM #temp -- check our filter SELECT Proc_Name, Definition FROM #temp WHERE definition LIKE '%foobar%' -- clean up DROP TABLE #temp
CLOSE c DEALLOCATE c Adam Machanic had a slightly different approach to the issue of multiple rows in syscomments, though it doesn't solve the problem where a line exceeds 4000 characters *and* your search phrase teeters on the end of such a line. Anyway, it's an interesting function, check it out! SQL Server 2005 Luckily, SQL Server 2005 will get us out of this problem. There are new functions like OBJECT_DEFINITION, which returns the whole text of the procedure. Also, there is a new catalog view, sys.sql_modules, which also holds the entire text, and INFORMATION_SCHEMA.ROUTINES has been updated so that the ROUTINE_DEFINITION column also contains the full text of the procedure. So, any of the following queries will work to perform this search in SQL Server 2005: SELECT Name FROM sys.procedures WHERE OBJECT_DEFINITION(object_id) LIKE '%foobar%' SELECT OBJECT_NAME(object_id) FROM sys.sql_modules WHERE Definition LIKE '%foobar%' AND OBJECTPROPERTY(object_id, 'IsProcedure') = 1 SELECT ROUTINE_NAME FROM INFORMATION_SCHEMA.ROUTINES WHERE ROUTINE_DEFINITION LIKE '%foobar%' AND ROUTINE_TYPE = 'PROCEDURE'
Note that there is no good substitute for documentation around your application. The searching above can provide many irrelevant results if you search for a word that happens to only be included in comments in some procedures, that is part of a larger word that you use, or that should be ignored due to frequency (e.g. SELECT). It can also leave things out if, for example, you are searching for the table name 'Foo_Has_A_Really_Long_Name' and some wise developer has done this: EXEC('SELECT * FROM Foo_Has' +'_A_Really_Long_Name') Likewise, sp_depends will leave out any procedure like the above, in addition to any procedure that was created before the dependent objects exist. The latter scenario is allowed due to deferred name resolution —the parser allows you to create a procedure that references invalid objects, and doesn't check that they exist until you actually run the stored procedure. So, long story short, you can't be 100% certain that any kind of searching or in-built query is going to be the silver bullet that tracks down every reference to an object. But you can get pretty close.
Third-party products Whenever there is a void, someone is going to come up with a solution, right? I was alerted recently of this tool, which indexes all of your metadata to help you search for words...
Problem Many times people come across the Coalesce function and think that it is just a more powerful form of ISNULL. In actuality, I have found it to be one of the most useful functions with the least documentation. In this tip, I will show you the basic use of Coalesce and also some features you probably never new existed. Solution Let's start with the documented use of coalesce. According to MSDN, coalesce returns the first non-null expression among its arguments. For example,
SELECT COALESCE(NULL, NULL, NULL, GETDATE())
will return the current date. It bypasses the first NULL values and returns the first non-null value. Using Coalesce to Pivot If you run the following statement against the AdventureWorks database
SELECT Name FROM HumanResources.Department WHERE (GroupName = 'Executive General and Administration')
you will come up with a standard result set such as this.
If you want to pivot the data you could run the following command.
DECLARE @DepartmentName VARCHAR(1000) SELECT @DepartmentName = COALESCE(@DepartmentName,'') + Name + ';' FROM HumanResources.Department WHERE (GroupName = 'Executive General and Administration') SELECT @DepartmentName AS DepartmentNames
and get the following result set.
Using Coalesce to Execute Multiple SQL Statements Once you can pivot data using the coalesce statement, it is now possible to run multiple SQL statements by pivoting the data and using a semicolon to separate the operations. Let's say you want to find the values for any column in the Person schema that has the column name “Name”. If you execute the following script it will give you just that.
DECLARE @SQL VARCHAR(MAX) CREATE TABLE #TMP (Clmn VARCHAR(500), Val VARCHAR(50)) SELECT @SQL=COALESCE(@SQL,'')+CAST('INSERT INTO #TMP Select ''' + TABLE_SCHEMA + '.' + TABLE_NAME + '.' + COLUMN_NAME + ''' AS Clmn, Name FROM ' + TABLE_SCHEMA + '.[' + TABLE_NAME + '];' AS VARCHAR(MAX)) FROM INFORMATION_SCHEMA.COLUMNS JOIN sysobjects B ON INFORMATION_SCHEMA.COLUMNS.TABLE_NAME = B.NAME WHERE COLUMN_NAME = 'Name' AND xtype = 'U' AND TABLE_SCHEMA = 'Person' PRINT @SQL EXEC(@SQL) SELECT * FROM #TMP DROP TABLE #TMP
here is the result set.
My personal favorite is being able to kill all the transactions in a database using three lines of code. If you have ever tried to restore a database and could not obtain exclusive access, you know how useful this can be.
DECLARE @SQL VARCHAR(8000) SELECT @SQL=COALESCE(@SQL,'')+'Kill '+CAST(spid AS VARCHAR(10))+ '; ' FROM sys.sysprocesses WHERE DBID=DB_ID('AdventureWorks') PRINT @SQL --EXEC(@SQL) Replace the print statement with exec to execute
will give you a result set such as the following.
Next Steps
• •
Whenever I think I may need a cursor, I always try to find a solution using Coalesce first. I am sure I just scratched the surface on the many ways this function can be used. Go try and see what all you can come up with. A little innovative thinking can save several lines of code.
Coalesce returns the first non-null expression among its arguments. Lets say we have to return a non-null from more than one column, then we can use COALESCE function.
SELECT COALESCE(hourly_wage, salary, commission) AS 'Total Salary' FROM wages
In this case, If hourly_wage is not null and other two columns are null then hourly_wage will be returned. If hourly_wage, commission are null and salary is not null then salary will be returned. If commission is non-null and other two columns are null then commission will be returned.
Using COALESCE to Build Comma-Delimited String
Garth is back with another article. This one talks about building a comma-separated value string for use in HTML SELECT tags. It's also handy anytime you need to turn multiple records into a CSV field. It's a little longer and has some HTML but a good read.
I was reading the newsgroups a couple of days ago and came across a solution posted by Itzik Ben-Gan I thought was really smart. In order for you to understand why I like it so much I have to give you a little background on the type of
applications I work on. Most of my projects for the past 2.5 years have focused on developing browser-based applications that access data stored in SQL Server. On almost all the projects I have worked on there has been at least one Add/Edit screen that contained a multi-select list box.
For those of you with limited experience working with HTML, I need to explain that the values selected in a multi-select list box are concatenated in a comma-delimited string. The following HTML creates a multi-select list box that displays retail categories.
<SELECT name="RetailCategory" multiple> <OPTION value=1>Shoes <OPTION value=2>Sporting Goods <OPTION value=3>Restaurant <OPTION value=4>Women's Clothes <OPTION value=5>Toys </SELECT>
If a user selects more than one option the value associated with RetailCategory is a commadelimited string. For example, if the user selects Shoes, Women's Clothes and Toys, the value associate with RetailCategory is 1, 4, 5. When the user Submits their form I call a stored procedure to insert the data into the appropriate tables. The comma-delimited string is processed with a WHILE loop and the individual values are inserted into a dependent table. Now that we have covered the Add part, let's take a look at what happens when a user wants to Edit a row. When editing a row, you need to populate the form with the existing values--this includes making sure all the multi-select list box values that are associated with the row are shown as selected. To show an option as selected, you use the "selected" attribute. If we were editing the row associated with the previous example the final HTML would look like the code shown here.
<SELECT name="RetailCategory" multiple> <OPTION value=1 selected>Shoes <OPTION value=2>Sporting Goods <OPTION value=3>Restaurant <OPTION value=4 selected>Women's Clothes <OPTION value=5 selected>Toys </SELECT>
I say final, because the actual HTML is built on-the-fly using VBScript. To determine which options are shown as selected, you must return a comma-delimited string to IIS so you can manipulate it with VBScript. I use the Split function and a For loop to determine which options should be shown as selected. The following VBScript shows how this is done.
<% EmpArray = Split(rs("EmployeeList")) For Each i In EmpArray If rs2("Emp_UniqueID") = CInt(i) Then response.write "selected" End If Next
%>
The remainder of this article shows the inefficient way I used to build the string along with the new, efficient way I learned from the newsgroup posting.
The Old, Inefficient Approach
Let's create and populate some tables so we have some data to work with. Assume you have a sales effort management (SEM) system that allows you to track the number of sales calls made on a potential client. A sales call is not a phone call, but a get together such as lunch or another type of person-to-person meeting. One of the things the VP of Sales wants to know is how many of his sales personnel participate in a call. The following tables allow you to track this information.
CREATE TABLE Employees ( Emp_UniqueID smallint PRIMARY KEY, Emp_FName varchar(30) NOT NULL, Emp_LName varchar(30) NOT NULL, ) go CREATE TABLE SalesCalls ( SalCal_UniqueID smallint PRIMARY KEY, SalCal_Desc varchar(100) NOT NULL, SalCal_Date smalldatetime NOT NULL, ) go CREATE TABLE SalesCallsEmployees ( SalCal_UniqueID smallint NOT NULL, Emp_UniqueID smallint NOT NULL, ) go
A limited number of columns are used in order to make this article easier to digest. The SalesCallsEmployees table is a junction table (aka associative table) that relates the employees (sales personnel) to a particular sales call. Let's populate the tables with sample data using these INSERT statements.
INSERT INSERT INSERT INSERT Employees Employees Employees Employees VALUES VALUES VALUES VALUES (1,'Jeff','Bagwell') (2,'Jose','Lima') (3,'Chris','Truby') (4,'Craig','Biggio')
INSERT SalesCalls VALUES (1,'Lunch w/ John Smith','01/21/01') INSERT SalesCalls VALUES (2,'Golfing w/ Harry White','01/22/01') INSERT SalesCallsEmployees VALUES (1,1) INSERT SalesCallsEmployees VALUES (1,2)
INSERT SalesCallsEmployees VALUES (1,4) INSERT SalesCallsEmployees VALUES (2,2)
The first sales call (Lunch w/ John Smith) had three employees participate. Using the old approach, I used the code shown here (inside a stored procedure) to build the comma-delimited string. The resultset shows the output when the "Lunch w/ John Smith" sales call is edited.
DECLARE @Emp_UniqueID int, @EmployeeList varchar(100) SET @EmployeeList = '' DECLARE crs_Employees CURSOR FOR SELECT Emp_UniqueID FROM SalesCallsEmployees WHERE SalCal_UniqueID = 1 OPEN crs_Employees FETCH NEXT FROM crs_Employees INTO @Emp_UniqueID WHILE @@FETCH_STATUS = 0 BEGIN SELECT @EmployeeList = @EmployeeList+CAST(@Emp_UniqueID AS varchar(5))+ ', ' FETCH NEXT FROM crs_Employees INTO @Emp_UniqueID END SET @EmployeeList = SUBSTRING(@EmployeeList,1,DATALENGTH(@EmployeeList)-2) CLOSE crs_Employees DEALLOCATE crs_Employees SELECT @EmployeeList
--Results---------1, 2, 4
This code may look a little complicated, but all it's doing is creating a cursor that holds the Emp_UniqueID values associated with the sales call and processing it with a WHILE to build the string. The important thing for you to note is that this approach takes several lines of code and uses a cursor. In general, cursors are considered evil and should only be used as a last resort.
The New and Improved Approach
The new and improved approach can create the same resultset with a single SELECT statement. The following code shows how it's done.
DECLARE @EmployeeList varchar(100) SELECT @EmployeeList = COALESCE(@EmployeeList + ', ', '') + CAST(Emp_UniqueID AS varchar(5))
FROM SalesCallsEmployees WHERE SalCal_UniqueID = 1 SELECT @EmployeeList
--Results---------1, 2, 4
Should I use COALESCE() or ISNULL()?
As with many technology questions involving roughly equivalent choices, it depends. There are a variety of minor differences between COALESCE() and ISNULL(): COALESCE() is ANSI standard, so that is an advantage for the purists out there.
Many consider ISNULL()'s readability and common sense naming to be an advantage. While I will agree that it easier to spell and pronounce, I disagree that its naming is intuitive. In other languages such as VB/VBA/VBScript, ISNULL() accepts a single input and returns a single boolean output. ISNULL() accepts exactly two parameters. If you want to take the first non-NULL among more than two values, you will need to nest your ISNULL() statements. COALESCE(), on the other hand, can take multiple inputs: SELECT ISNULL(NULL, NULL, 'foo') -- yields: Server: Msg 174, Level 15, State 1, Line 1 The isnull function requires 2 arguments. SELECT COALESCE(NULL, NULL, 'foo') -- yields: ---foo
In order to make this work with ISNULL(), you would have to say: SELECT ISNULL(NULL, ISNULL(NULL, 'foo'))
The result of ISNULL() always takes on the datatype of the first parameter (regardless of whether it is NULL or NOT NULL). COALESCE works more like a CASE expression, which returns a single datatype depending on precendence and accommodating all possible outcomes. For example:
DECLARE @foo VARCHAR(5) SET @foo = NULL SELECT ISNULL(@foo, '123456789') -- yields: ----12345 SELECT COALESCE(@foo, '123456789') -- yields: --------123456789
This gets more complicated if you start mixing incompatible datatypes, e.g.: DECLARE @foo VARCHAR(5), @bar INT SET @foo = 'foo' SET @bar = NULL SELECT ISNULL(@foo, @bar) SELECT COALESCE(@foo, @bar) -- yields: ----foo Server: Msg 245, Level 16, State 1, Line 6 Syntax error converting the varchar value 'foo' to a column of data type int.
A relatively scarce difference is the ability to apply constraints to computed columns that use COALESCE() or ISNULL(). SQL Server views a column created by COALESCE() as nullable, whereas one using ISNULL() is not. So: CREATE TABLE dbo.Try ( col1 INT, col2 AS COALESCE(col1, 0) PRIMARY KEY ) GO -- yields: Server: Msg 8111, Level 16, State 2, Line 1 Cannot define PRIMARY KEY constraint on nullable column in table
'Try'. Server: Msg 1750, Level 16, State 1, Line 1 Could not create constraint. See previous errors.
Whereas the following works successfully: CREATE TABLE dbo.Try ( col1 INT, col2 AS ISNULL(col1, 0) PRIMARY KEY ) GO
If you are using COALESCE() and or ISNULL() as a method of allowing optional parameters into your WHERE clause, please see Article #2348 for some useful information (the most common techniques will use a scan, but the article shows methods that will force a more efficient seek). Finally, COALESCE() can generate a less efficient plan in some cases, for example when it is used against a subquery. Take the following example in Pubs and compare the execution plans: USE PUBS GO SET SHOWPLAN_TEXT ON GO SELECT COALESCE ( (SELECT a2.au_id FROM pubs..authors a2 WHERE a2.au_id = a1.au_id), '' ) FROM authors a1 SELECT ISNULL ( (SELECT a2.au_id FROM pubs..authors a2 WHERE a2.au_id = a1.au_id), '' ) FROM authors a1 GO
SET SHOWPLAN_TEXT OFF GO
Notice the extra work that COALESCE() has to do? This may not be a big deal against this tiny table in Pubs, but in a bigger environment this can bring servers to their knees. And no, this hasn't been made any more efficient in SQL Server 2005, you can reproduce the same kind of plan difference in AdventureWorks: USE AdventureWorks GO SET SHOWPLAN_TEXT ON GO SELECT COALESCE ( (SELECT MAX(Name) FROM Sales.Store s2 WHERE s2.name = s1.name), '' ) FROM Sales.Store s1 SELECT ISNULL ( (SELECT MAX(Name) FROM Sales.Store s2 WHERE s2.name = s1.name), '' ) FROM Sales.Store s1 GO SET SHOWPLAN_TEXT OFF GO
How do I find a stored procedure containing <text>?
I see this question at least once a week. Usually, people are trying to find all the stored procedures that reference a specific object. While I think that the best place to do this kind of searching is through your source control tool (you do keep your database objects in source control, don't you?), there are certainly some ways to do this in the database. Let's say you are searching for 'foobar' in all your stored procedures. You can do this using the
INFORMATION_SCHEMA.ROUTINES view, or syscomments: SELECT ROUTINE_NAME, ROUTINE_DEFINITION FROM INFORMATION_SCHEMA.ROUTINES WHERE ROUTINE_DEFINITION LIKE '%foobar%' AND ROUTINE_TYPE='PROCEDURE' If you want to present these results in ASP, you can use the following code, which also highlights the searched string in the body (the hW function is based on Article #2344): <% set conn = CreateObject("ADODB.Connection") conn.Open = "<connection string>" ' String we're looking for: str = "foobar" sql = "SELECT ROUTINE_NAME, ROUTINE_DEFINITION " & _ "FROM INFORMATION_SCHEMA.ROUTINES " & _ "WHERE ROUTINE_DEFINITION LIKE '%" & str & "%' " & _ " AND ROUTINE_TYPE='PROCEDURE'" set rs = conn.execute(sql) if not rs.eof then do while not rs.eof s = hW(str, server.htmlEncode(rs(1))) s = replace(s, vbTab, " ") s = replace(s, vbCrLf, "<br>") response.write "<b>" & rs(0) & "</b><p>" & s & "<hr>" rs.movenext loop else response.write "No procedures found." end if function hW(strR, tStr) w = len(strR) do while instr(lcase(tStr), lcase(strR)) > 0 cPos = instr(lcase(tStr), lcase(strR)) nStr = nStr & _ left(tStr, cPos - 1) & _ "<b>" & mid(tStr, cPos, w) & "</b>" tStr = right(tStr, len(tStr) - cPos - w + 1) loop hW = nStr & tStr end function rs.close: set rs = nothing
conn.close: set conn = nothing %> Another way to perform a search is through the system table syscomments: SELECT OBJECT_NAME(id) FROM syscomments WHERE [text] LIKE '%foobar%' AND OBJECTPROPERTY(id, 'IsProcedure') = 1 GROUP BY OBJECT_NAME(id) Now, why did I use GROUP BY? Well, there is a curious distribution of the procedure text in system tables if the procedure is greater than 8KB. So, the above makes sure that any procedure name is only returned once, even if multiple rows in or syscomments draw a match. But, that begs the question, what happens when the text you are looking for crosses the boundary between rows? Here is a method to create a simple stored procedure that will do this, by placing the search term (in this case, 'foobar') at around character 7997 in the procedure. This will force the procedure to span more than one row in syscomments, and will break the word 'foobar' up across rows. Run the following query in Query Analyzer, with results to text (CTRL+T): SET NOCOUNT ON SELECT 'SELECT '''+REPLICATE('x', 7936)+'foobar' SELECT REPLICATE('x', 500)+'''' This will yield two results. Copy them and inject them here: CREATE PROCEDURE dbo.x AS BEGIN SET NOCOUNT ON << put both results on this line >> END GO Now, try and find this stored procedure in INFORMATION_SCHEMA.ROUTINES or syscomments using the same search filter as above. The former will be useless, since only the first 8000 characters are stored here. The latter will be a little more useful, but initially, will return 0 results because the word 'foobar' is broken up across rows, and does not appear in a way that LIKE can easily find it. So, we will have to take a slightly more aggressive approach to make sure we find this procedure. Your need to do this, by the way, will depend partly on your desire for thoroughness, but more so on the ratio of stored procedures you have that are greater than 8KB. In all the systems that I manage, I don't have more than a handful that approach this size, so this isn't something I reach for very often. Maybe for you it will be more useful.
First off, to demonstrate a little better (e.g. by having more than one procedure that exceeds 8KB), let's create a second procedure just like above. Let's call it dbo.y, but this time remove the word 'foobar' from the middle of the SELECT line. CREATE PROCEDURE dbo.y AS BEGIN SET NOCOUNT ON << put both results on this line, but replace 'foobar' with something else >> END GO Basically, what we're going to do next is loop through a cursor, for all procedures that exceed 8KB. We can get this list as follows: SELECT OBJECT_NAME(id) FROM syscomments WHERE OBJECTPROPERTY(id, 'IsProcedure') = 1 GROUP BY OBJECT_NAME(id) HAVING COUNT(*) > 1 We'll need to create a work table to hold the results as we loop through the procedure, and we'll need to use UPDATETEXT to append each row with the new 8000-or-less chunk of the stored procedure code. -- create temp table CREATE TABLE #temp ( Proc_id INT, Proc_Name SYSNAME, Definition NTEXT ) -- get the names of the procedures that meet our criteria INSERT #temp(Proc_id, Proc_Name) SELECT id, OBJECT_NAME(id) FROM syscomments WHERE OBJECTPROPERTY(id, 'IsProcedure') = 1 GROUP BY id, OBJECT_NAME(id) HAVING COUNT(*) > 1 -- initialize the NTEXT column so there is a pointer UPDATE #temp SET Definition = '' -- declare local variables DECLARE
@txtPval binary(16), @txtPidx INT, @curName SYSNAME, @curtext NVARCHAR(4000) -- set up a cursor, we need to be sure this is in the correct order -- from syscomments (which orders the 8KB chunks by colid) DECLARE c CURSOR LOCAL FORWARD_ONLY STATIC READ_ONLY FOR SELECT OBJECT_NAME(id), text FROM syscomments s INNER JOIN #temp t ON s.id = t.Proc_id ORDER BY id, colid OPEN c FETCH NEXT FROM c INTO @curName, @curtext -- start the loop WHILE (@@FETCH_STATUS = 0) BEGIN -- get the pointer for the current procedure name / colid SELECT @txtPval = TEXTPTR(Definition) FROM #temp WHERE Proc_Name = @curName -- find out where to append the #temp table's value SELECT @txtPidx = DATALENGTH(Definition)/2 FROM #temp WHERE Proc_Name = @curName -- apply the append of the current 8KB chunk UPDATETEXT #temp.definition @txtPval @txtPidx 0 @curtext FETCH NEXT FROM c INTO @curName, @curtext END -- check what was produced SELECT Proc_Name, Definition, DATALENGTH(Definition)/2 FROM #temp -- check our filter SELECT Proc_Name, Definition FROM #temp WHERE definition LIKE '%foobar%' -- clean up DROP TABLE #temp
CLOSE c DEALLOCATE c Adam Machanic had a slightly different approach to the issue of multiple rows in syscomments, though it doesn't solve the problem where a line exceeds 4000 characters *and* your search phrase teeters on the end of such a line. Anyway, it's an interesting function, check it out! SQL Server 2005 Luckily, SQL Server 2005 will get us out of this problem. There are new functions like OBJECT_DEFINITION, which returns the whole text of the procedure. Also, there is a new catalog view, sys.sql_modules, which also holds the entire text, and INFORMATION_SCHEMA.ROUTINES has been updated so that the ROUTINE_DEFINITION column also contains the full text of the procedure. So, any of the following queries will work to perform this search in SQL Server 2005: SELECT Name FROM sys.procedures WHERE OBJECT_DEFINITION(object_id) LIKE '%foobar%' SELECT OBJECT_NAME(object_id) FROM sys.sql_modules WHERE Definition LIKE '%foobar%' AND OBJECTPROPERTY(object_id, 'IsProcedure') = 1 SELECT ROUTINE_NAME FROM INFORMATION_SCHEMA.ROUTINES WHERE ROUTINE_DEFINITION LIKE '%foobar%' AND ROUTINE_TYPE = 'PROCEDURE'
Note that there is no good substitute for documentation around your application. The searching above can provide many irrelevant results if you search for a word that happens to only be included in comments in some procedures, that is part of a larger word that you use, or that should be ignored due to frequency (e.g. SELECT). It can also leave things out if, for example, you are searching for the table name 'Foo_Has_A_Really_Long_Name' and some wise developer has done this: EXEC('SELECT * FROM Foo_Has' +'_A_Really_Long_Name') Likewise, sp_depends will leave out any procedure like the above, in addition to any procedure that was created before the dependent objects exist. The latter scenario is allowed due to deferred name resolution —the parser allows you to create a procedure that references invalid objects, and doesn't check that they exist until you actually run the stored procedure. So, long story short, you can't be 100% certain that any kind of searching or in-built query is going to be the silver bullet that tracks down every reference to an object. But you can get pretty close.
Third-party products Whenever there is a void, someone is going to come up with a solution, right? I was alerted recently of this tool, which indexes all of your metadata to help you search for words...
