Tuesday, December 7, 2010

[MSSQL][Library] How to handle a Delimited String similar to handle an Array() of Values

Often there is the desire to pass data to T-SQL as an array() of values, yet T-SQL does not provide an array() datatype.

The following presents one method to simulate handling an array() using T-SQL. The client application creates a single-variate array(), and then passes that array() as a VARCHAR() parameter to a SQL Stored Procedure. The Stored Procedure, in turn, passes that VARCHAR(), or string, value to a User Defined Function that parses the string into a temporary table, and provides that temporary table back to the Stored Procedure to use as though it were a real table.


This 'method' can be used to separate a List of Values into Discrete values, and then use that List of Discrete Values in most operations like a Table.
It can be used in a SELECT statement, a multi-table JOIN, and in WHERE clause criteria.


The User may Specify any character as Delimiters.

     Examples:

          '1,2,35,101'                      (Comma Delimited)
          'Act1|Act2|Act5'                (Vertical Bar Delimited)
          '12345 12432 3234 452'     (Single Space Delimited)

Note: For handling very large amounts of data in a 'simulated' Array(), or for rapid multiple calls to the User Defined Function, it may be more efficient to consider creating a CLR Function using C#/VB and Regular Expressions. For small simulated arrays(), there may be little, if any, performance difference.


USE []
GO
IF OBJECT_ID(N'udf__Split') IS NOT NULL
  DROP FUNCTION udf__Split
GO
SET ANSI_NULLS OFF
GO
SET QUOTED_IDENTIFIER OFF
GO
-- *****************************************************************************
-- Name         : udf__Split
-- Author       : Chandra Gunawan
-- Date         : 06-Dec-2010
-- Description  : Convert comma seperated list of item to table
-- Parameters   :
--     @pList       vchr(8000)  Delimited list of item to split
--                                format:  item1[<dlm>[item2]...]
--                                example: a,b,c
--     @pDelimiter  vchr(1)     Delimiter (dlm)
--                                example: ,
--
-- Usage Sample :
--   -------------------------------------------------
--   SELECT * FROM dbo.udf__Split('a,b,c', ',')
--   -------------------------------------------------
--
-- Maintenance Log
-- =============================================================================
-- Date        ID     Description
-- -----------------------------------------------------------------------------
-- 02-Jun-09   chg    Initial Version
-- *****************************************************************************
CREATE FUNCTION dbo.udf__Split
(
    @pList      varchar(8000),
    @pDelimiter varchar(1) = ','
)
RETURNS @pTable TABLE
(
    id   smallint IDENTITY(1,1),
    item varchar(1000)
)
BEGIN 
  DECLARE @iSpot smallint,
          @cStr  varchar(8000),
          @cSql  varchar(8000) 
    
  IF ISNULL(@pDelimiter, '') = '' SET @pDelimiter = ','

  WHILE @pList <> '' 
  BEGIN 
    SET @iSpot = CHARINDEX(@pDelimiter, @pList) 
    IF @iSpot >
    BEGIN 
      SET @cStr = LEFT(@pList, @iSpot - 1) 
      SET @pList = RIGHT(@pList, len(@pList) - @iSpot) 
    END 
    ELSE BEGIN 
      SET @cStr = @pList 
      SET @pList = '' 
    END 
    INSERT @pTable SELECT @cStr
  END 
  RETURN
END
GO
GRANT SELECT ON udf__Split TO public
GO

Usage Sample
The string values are parsed into individual 'rows' and then sorted.

Usage with the String Containing Numeric values.
(Note the use of CAST() in the ORDER BY clause to cause the Values to sort numerically.)
  

SELECT *
  FROM dbo.udf__Split('11,23,3,14', ',') AS s
  ORDER BY cast(s.[item] AS int)

id 
item 
3
3
1
11
4
14
2
23


Usage with the String Containing Character values.

SELECT *
  FROM dbo.udf__Split( 'Bob|Jane|Mary|Li|Hsiao|Lubor', '|' ) AS s
  ORDER BY s.[item]
 
id 
item 
1
Bob
5
Hsiao
2
Jane
4
Li
6
Lubor
3
Mary


To get the second entry of the list.

DECLARE @cItem varchar(100)

SELECT @cItem = item
  FROM dbo.udf__Split( 'Bob|Jane|Mary|Li|Hsiao|Lubor', '|' )
  WHERE id = 2

SELECT @cItem

(No column name)
Jane


See also:
Convert comma separated list of item into single column table (udf__ListToSingleColumn)

 

Thursday, November 25, 2010

[mysql] Selecting Random Selection Of Rows From A Database Table

The begining of this story is a PHPNuke module. The module has a block that picks a random item from a database table and displays it. The content of the table never changes. Here is how the programmer of the block does it:

He knows that there are 6899 rows because the last id in the table has a value of 6899. So with the "rand" PHP function, he picks a random number from 1 to 6899 and then queries the database table with a WHERE clause that checks if the value of the id is equal to the picked number:

srand(time());
$random = (rand(1,6899));
$res = mysql_query ("select * from my_table where id=$random");


The problem here is that because of some deleted rows (he has forgotten he deleted them) in the table, the id is not a continuous sequence from 1 to 6899 and that's why some times the picked number (as a value of the id) not exists in the table and the PHPNuke block displays an empty box.

The RAND function in MySQL is precisely there to prevent this kind of problem. Instead of picking a random id with a PHP code, let the server do it for you:

SELECT * FROM my_table ORDER BY RAND() LIMIT 1;

This query returns a random row from the table my_table, no matter how many rows there are and no matter if the id is a continuous sequence of numbers or not. With this query you are sure you get a random row every single time.

Conclusion:
Never reinvent the wheel.

Wednesday, November 24, 2010

[mssql] Selecting Random Selection Of Rows From A Database Table

I found a nice tip for selecting random rows from within a SQL Server 2000 database. Well actually, pseudorandom. Since my undergraduate thesis was on the topic of pseudorandom number generation, I might as well be precise. For some reason, my non-geek friends find it awfully funny when I mention pseudorandom numbers.

I digress. In order to select 10 records from some table at random, try this:
SELECT TOP 10 * FROM someTable ORDER By NEWID()
Now for my homework, I should find out just how random this is. There's a whole slew of statistical tests I can run to gauge the randomness of pseudorandom number generator such as the Chi-square Test, Serial Correlation Coefficient, and 2-D Random Walk Test to name a few.

IMPORTANT: Please note that this will NOT work in SQL 7 on NT4 because the NEWID() function there generates sequential results

Tuesday, November 2, 2010

[mssql] Compare the records of two tables in SQL Server

When you have two tables (or resultsets from SELECT statements) that you wish to compare, and you want to see any changes in ANY columns, as well as to see which rows exist in 1 table but not the other (in either direction) I have found that the UNION operator works quite well.

UNION allows you to compare all columns very quickly, and also handles comparing NULL values to other NULLs successfully, which a join clause or a WHERE condition doesn't normally do.  It also allows you to very quickly see which rows are missing in either table, which only a FULL OUTER JOIN will do, but of course we all know to avoid those at all costs (right?) -- a full outer join is about as “unrelational” as you can get.  (every column returned is potentially Null and must be wrapped in a COALESCE function).  Best of all, the UNION is quick and easy and short.

The basic idea is: if we GROUP the union of two tables on all columns, then if the two tables are identical all groups will result in a COUNT(*) of 2.  But for any rows that are not completely matched on any column in the GROUP BY clause, the COUNT(*) will be 1 -- and those are the ones we want.  We also need to add a column to each part of the UNION to indicate which table each row comes from, otherwise there is no way to distinguish between which row comes from which table.

So, here's an example, assuming we are comparing tables A and B, and the primary key of both tables is ID:


SELECT MIN(TableName) as TableName, KeyColumn, Column2 [, ...]
  FROM
  (SELECT '<TableName1>' as TableName, KeyColumn, Column2 [, ...]
     FROM <TableName1>
     UNION ALL
   SELECT '<TableName2>' as TableName, KeyColumn, Column2 [, ...]
     FROM <TableName2>
  ) tmp
  GROUP BY KeyColumn, Column2 [, ...]
  HAVING COUNT(*) = 1   -- Alternative: COUNT(*) <> 2
  ORDER BY KeyColumn

The above returns all rows in either table that do not completely match all columns in the other.  In addition, it returns all rows in either table that do not exist in the other table.  It handles nulls as well, since GROUP BY normally consolidates NULL  values together in the same group.  If both tables match completely, no rows are returned at all.

The MIN() aggregate function used on the TableName column is just arbitrary -- it has no effect since we are only returning groups of rows in which there has been no consolidation with the GROUP BY (note the HAVING clause).

Tuesday, September 7, 2010

[mssql] Database in Recovery Status

Q: I have a database currently showing to be in recovery.  I have not been able to find a GUI method to monitor the progress (to determine when it might complete) so is there a command line T-SQL method to monitor the recovery process?

A: If this is 2005, this might be what you need:

SELECT session_id, command, status, percent_complete, *
  FROM sys.dm_exec_requests
 
It works for other types of commands that have known progress indicators.  If not, then I really don't think it tells you.  You might also check the error log to see how long it took the previous time...

Tuesday, June 15, 2010

[mssql] How do I format money/decimal with commas?

Sometimes you want to have your money fields properly formatted with commas like this: 13,243,543.57
You can use the CONVERT function and give a value between 0 and 2 to the style and the format will be displayed based on that

Below is an example:



DECLARE @v MONEY
SELECT @v = 1322323.6666

SELECT CONVERT(VARCHAR,@v,0)    --1322323.67    
/* Rounded but no formatting */

SELECT CONVERT(VARCHAR,@v,1)    --1,322,323.67    
/* Formatted with commas */

SELECT CONVERT(VARCHAR,@v,2)    --1322323.6666 
/* No formatting */
 

If you have a decimal field it doesn't work with the convert function
The work around is to convert it to money



DECLARE @v2 DECIMAL (36,10)
SELECT @v2 = 13243543.56565656

SELECT CONVERT(VARCHAR,CONVERT(MONEY,@v2),1) --13,243,543.57
/* Formatted with commas */

[mssql] How To Get Output Into SQL Server Table

Have you every needed to get the information returned from a stored procedure (SP) into a SQL Server table? Or have you wondered how you might get the contents of an operating system (OS) file into a SQL Server table? How about placing the output of some Windows executable into a SQL Server table so you can manipulate it with T-SQL in some way? So, what methods can you use to accomplish these different, but similar tasks? This article will show you how to insert rows into a SQL Server table from various sources.

Getting Output from a Stored Procedure into a Table

The most commonly asked question I get is "How can I get the output of a SP into a SQL Server table." Now, if you did not know how to do this you might be inclined to just copy the code from the stored procedure and paste it into your T-SQL code. This way your code can insert the record set into a table instead of trying to get it from an existing SP. However, there are alternatives to this cut and paste method. You can use the "execute_statement" option on an "INSERT" statement, to get the output of a stored procedure into a table easily. Here is an example of getting the output of the "sp_who" SP into a table:

set nocount on
create table #sp_who (
  spid      smallint,
  ecid      smallint,
  status    nchar(30),
  loginame  nchar(128),
  hostname  nchar(128),
  blk       char(5),
  dbname    nchar(128),
  cmd       nchar(16))
insert into #sp_who execute sp_who
  select * from #sp_who
drop table #sp_who

As you can see, this is a very simple process. I first created a table that contains a column for each column returned from the "sp_who" SP. In my example, each column of my temporary table has the same name, data type and length as those returned from the "sp_who" SP. After the table is created, all it takes to get the SP's output into a table is to issue an INSERT statement that executes the "sp_who" stored procedure. Once this INSERT statement completes SQL Server will place all the records displayed by the "sp_who" SP in the temporary table #sp_who. Using this method allows me then to manipulate the "sp_who" output however I would like. If a store procedure outputs multiple record sets then the above method, to get the output of the stored procedure into a table, will not work. In fact, if you try it you will receive an error.

Getting the Content of an OS File into a Table

Now suppose that you want to get the contents of an OS file into a T-SQL table.
I am not talking about the raw, comma delimited text file here, but instead I am referring to just a plan unformatted text file. Do you have any ideas how this might be accomplished? It is quite easy to accomplish this with "xp_cmdshell" and a one-column table. Let's go through an example of how this would be done. For my example, let's get the current SQL Server ERRORLOG file into a temporary table. Here is the code to do that:


create
table #errorlog(line varchar(2000))

insert into #errorlog
  execute xp_cmdshell 'type "C:\Program Files\Microsoft SQL Server\MSSQL\LOG\ERRORLOG" '
select line from #errorlog
drop table #errorlog

As you can see, I have basically used the same method as getting a stored procedure's output into a table. Except this time I used the xp_cmdshell extended stored procedure to execute the DOS "type" command. The DOS "type" command just displays the content of the file specified. Therefore by using the execute clause on the INSERT statement in conjunction with the "type" command of DOS I was able to get the contents of the ERRORLOG OS file into a SQL Server table.
Now I can use T-SQL to manipulate the data from an OS file by using this process. The only drawback here is each line in the OS file is stored in a single column within a row. Therefore, if you want to search and extract stuff you will need to use the CHARINDEX, PATINDEX, and/or SUBSTRING functions.

Getting Multiple Record Sets into a Table

Remember earlier when I said if a store procedure returns multiple record sets that you get an error when trying to use the EXECUTE clause of an INSERT statement. Now let me show you a way to get around this issue. The trick to making this work is to first route the output from the SP into an OS file, then use the method I showed you above to get the OS file into a SQL Server table. Let me go through an example.
Say I want to get the output of the "sp_spaceused" SP, into a SQL Server table. Because it produces multiple record sets, I am not able to use the first method I discussed above. Here is some code that will allow me to get the output of the "sp_spaceused" SP into a SQL Server table.


create table #errorlog(line varchar(2000))
execute master.dbo.xp_cmdshell 'osql -SYourSQLMachine
  -E -Q"execute sp_spaceused" -o"c:\temp\sp_out.txt" -s"" '
insert into #errorlog
  execute master.dbo.xp_cmdshell 'type "c:\temp\sp_out.txt" '
select line from #errorlog
drop table #errorlog

Here you can see I used xp_cmdshell to execute the OSQL utility. By using this utility I was able to direct the output of the "sp_spaceused" SP directly into an OS file using the -o option. I then used the procedure described in the preceding section to get the OS file created by the OSQL utility into my temporary table named #errorlog. Once again, using this method causes each line of output from the "sp_spaceused" SP to be stored in a single column record in my temporary table.

Output From an Executable Into a Table

For my last example, I will show you how to get the output of an ".exe" into a SQL Server table. Suppose I want to get my server's IP configuration information into a SQL Server table so I can manipulate it. To display this information you would run the "ipconfig.exe" executable. So here is the code I would use to get the IP configure information into a table:


create table #ipconfig(line varchar(2000))
insert into #ipconfig
  execute xp_cmdshell 'ipconfig.exe'
select line from #ipconfig
drop table #ipconfig

Here the code is similar to the code I used to get the contents of an OS file into a table. The only difference is instead of running a DOS command with xp_cmdshell , I ran the "ipconfig.exe" executable.

Conclusion

With the use of an INSERT statement, a SQL Server table, and the occasional execution of the xp_cmdshell extended stored procedure it is possible to get almost anything into a SQL Server table. I hope that the next time you need to get the output of a stored procedure, physical file, and/or the output of an ".exe" into a SQL Server table you will consider using one of the processes I described in this article.