Author name: Russ

Russ Loski is a data warehouse developer focusing on SSIS and SQL Server development. He has worked with SQL Server since 1998. He has used various technologies to move data from one source to others. He has worked for well-known companies such as the Bank of America, the Dallas Cowboys Football Club, T-Mobile and Coca Cola Enterprises. Currently he is developing a Data Warehouse for a large health care organization.

SSIS Removing Duplicate Rows Using Conditions

Sometime your data source has duplicates. SSIS has an out of the box tool to remove these duplicates. The Sort transformation has a “Remove rows with duplicate sort values” property (https://www.mssqltips.com/sqlservertip/3036/removing-duplicates-rows-with-ssis-sort-transformation/). But it has a problem. It is very random as to the values that it removes. How do you specify which row to remove? […]

SSIS Removing Duplicate Rows Using Conditions Read More »

Removing quotes from CSV created by PowerShell

In a previous post, I demonstrate how to reshape a comma delimited file (CSV) using PowerShell. I noted that the results put all of the column values in double quotes: “ID”,”Name”,”State” “5”,”Stephanie”,”Arizona” “4”,”Melanie”,”Oregon” “2”,”Katie”,”Texas” I used this code to produce this result: cd c:\MyFolder Import-Csv -Path SampleData.csv|Where {$_.Country -eq “USA”} ` | SELECT ID, Name,

Removing quotes from CSV created by PowerShell Read More »

Using PowerShell to shape comma delimited file

PowerShell has power to handle many ETL tasks (PowerShell as an ETL tool). In this post I illustrate how you can use PowerShell to shape a comma delimited file. That is, how to remove columns, filter rows and sort the file output. Let’s start with a simple file (SampleData.csv): ID,Name,State,Country 1,Russ,Sandys,Bermuda 2,Katie,Texas,USA 3,Gail,Sandys,Bermuda 4,Melanie,Oregon,USA 5,Stephanie,Arizona,USA

Using PowerShell to shape comma delimited file Read More »

SSIS Token Function

Microsoft quietly added two functions to their SSIS expression language: Token and TokenCount. I think that these functions will find their way into any serious SSIS developer’s tool belt. Here I show how you can use this to solve a very common problem, interpreting dates. When importing data, frequently one column of data will need

SSIS Token Function Read More »