SSIS Conditional split transformation




This article was created to achieve very specific task; for easier to understand tutorial (covering different scenarios in more details) please visit SSIS Conditional Split Tutorial

SSIS Conditional split transformation allows you to split source rows (query) into multiple groups in data flow which you can direct to different destination paths. In this tutorial I will use SSIS 2008 R2 and my own real-life example. The screenshot and explanation should be fairly similar in other versions of SQL Server for instance SSIS 2005 and SSIS 2012.

SSIS Conditional split transformation is very useful for usually applying business rules where certain rows need to have slightly different transformation applied (data cleansing) or need to be re-directed to different destination for user input. Another example is to split source into equal groups which is less common example but this is something I need to do in order to achieve parallel processing. Althought I used for parallel processing in control flow you could use this method to achieve better performance inside data flow as well.

In my example I will want to get a list of ProxyIPAddresses (100 rows) and split them into 10 groups and put them into 10 different recordset destinations. After extracting my ProxyIPAddresses (100 rows) I perform validation with 30s timeout and it used to take about 20-30 minutes. Once I performed split into 10 groups and run the same packages 10 times in parallel in control flow. The time decreased to 2-3 minutes and it didn't use much more resources as most of it was wait.

Not that we covered purpose of conditional split let's move on to the actual example.

Let's start with SQL query I use. I have 100 IP addresses and I use SQL function to create second column that splits it into 10 groups from 1 to 10.

 

Are you a paid member?

Log in or Buy Membership 1 month unlimied use access pass for $25
or $10/month (yearly subscription)





Did you find this page useful? +4  |  -0
(4 Votes)