SSIS Conditional split transformation

My Learning: Log in Learning Report [Log in]


Remove Ads or Log in
Remove Ads or Log in
+0  |  -0

This article was created to achieve very specific task; for easier to understand tutorial (covering different scenarios in more details) please visit SSIS Conditional Split Tutorial

SSIS Conditional split transformation allows you to split source rows (query) into multiple groups in data flow which you can direct to different destination paths. In this tutorial I will use SSIS 2008 R2 and my own real-life example. The screenshot and explanation should be fairly similar in other versions of SQL Server for instance SSIS 2005 and SSIS 2012.

SSIS Conditional split transformation is very useful for usually applying business rules where certain rows need to have slightly different transformation applied (data cleansing) or need to be re-directed to different destination for user input. Another example is to split source into equal groups which is less common example but this is something I need to do in order to achieve parallel processing. Althought I used for parallel processing in control flow you could use this method to achieve better performance inside data flow as well.

In my example I will want to get a list of ProxyIPAddresses (100 rows) and split them into 10 groups and put them into 10 different recordset destinations. After extracting my ProxyIPAddresses (100 rows) I perform validation with 30s timeout and it used to take about 20-30 minutes. Once I performed split into 10 groups and run the same packages 10 times in parallel in control flow. The time decreased to 2-3 minutes and it didn't use much more resources as most of it was wait.

Not that we covered purpose of conditional split let's move on to the actual example.

In my example I started with adding OLEDB Source and conditional split. See below screenshot.

SSIS Conditional Split

In the OLEDB source I need to provide logic for conditional split so I added extra column GroupNumber I used NTILE function to group rows into 10 more or less equal groups. In my case I have 100 rows so I will get 10 groups with 10 rows each. See below preview of data between OLEDB source and conditional split which shows GroupNumber that I will use for the split.

SSIS Conditional Split OLEDB source preview

It is time to configure our conditional split transformation. Drag green arrow (data path) from OLEDB source to conditional split transformation and double click the transformation. You should get empty window that needs filling. See below screenshot that show how I used Source GroupNumber column and specified 10 split (+1 default). NOTE: Conditional split transformation and Default output name. You can rename default output name and use it in on of the destination. The way it works is very simple. If you have rows that don't meet any of the specified conditions then it goes to default output. If you have rows that don't meet any condition and you don't have default output specified than these rows will 'disappear' and the package will run with success, so in most cases this behaviour is not desirable and you should bare this in mind.

SSIS Conditional Split Transformation

Now we just need to click conditional split transformation and drag the green arrow to our destination which in my case is recorset. You will get a pop up where you can select one of your splits. See below screenshot with example of setting group 1.

SSIS Conditional split destination configuration

Below is the final result after the package has been run.

SSIS Conditional split final result
 


Remove Ads or Log in

KEBI Academy is open!

You can now join our KEBI Academy and just learn more.

How much? It's voluntary payment so you decide how much you want to contribute to what you get out of it.

Join KEBI Academy

Visitor Voting

Visitor comment (we like):
"I like you both, you read Dale Carnagie, you want what most of us want and you are trying to do something useful for all! You are already successful! Blessings!"
Add comment (no email required).



Add Comment
No data was returned.
  Join KEBI Academy (Voluntary Payment)