SSIS Conditional split transformation

My Learning: Log in Learning Report [Log in]


Remove Ads or Log in
Remove Ads or Log in
+0  |  -0

This article was created to achieve very specific task; for easier to understand tutorial (covering different scenarios in more details) please visit SSIS Conditional Split Tutorial

SSIS Conditional split transformation allows you to split source rows (query) into multiple groups in data flow which you can direct to different destination paths. In this tutorial I will use SSIS 2008 R2 and my own real-life example. The screenshot and explanation should be fairly similar in other versions of SQL Server for instance SSIS 2005 and SSIS 2012.

SSIS Conditional split transformation is very useful for usually applying business rules where certain rows need to have slightly different transformation applied (data cleansing) or need to be re-directed to different destination for user input. Another example is to split source into equal groups which is less common example but this is something I need to do in order to achieve parallel processing. Althought I used for parallel processing in control flow you could use this method to achieve better performance inside data flow as well.

In my example I will want to get a list of ProxyIPAddresses (100 rows) and split them into 10 groups and put them into 10 different recordset destinations. After extracting my ProxyIPAddresses (100 rows) I perform validation with 30s timeout and it used to take about 20-30 minutes. Once I performed split into 10 groups and run the same packages 10 times in parallel in control flow. The time decreased to 2-3 minutes and it didn't use much more resources as most of it was wait.

Not that we covered purpose of conditional split let's move on to the actual example.

In my example I started with adding OLEDB Source and conditional split. See below screenshot.

SSIS Conditional Split

In the OLEDB source I need to provide logic for conditional split so I added extra column GroupNumber I used NTILE function to group rows into 10 more or less equal groups. In my case I have 100 rows so I will get 10 groups with 10 rows each. See below preview of data between OLEDB source and conditional split which shows GroupNumber that I will use for the split.

SSIS Conditional Split OLEDB source preview

It is time to configure our conditional split transformation. Drag green arrow (data path) from OLEDB source to conditional split transformation and double click the transformation. You should get empty window that needs filling. See below screenshot that show how I used Source GroupNumber column and specified 10 split (+1 default). NOTE: Conditional split transformation and Default output name. You can rename default output name and use it in on of the destination. The way it works is very simple. If you have rows that don't meet any of the specified conditions then it goes to default output. If you have rows that don't meet any condition and you don't have default output specified than these rows will 'disappear' and the package will run with success, so in most cases this behaviour is not desirable and you should bare this in mind.

SSIS Conditional Split Transformation

Now we just need to click conditional split transformation and drag the green arrow to our destination which in my case is recorset. You will get a pop up where you can select one of your splits. See below screenshot with example of setting group 1.

SSIS Conditional split destination configuration

Below is the final result after the package has been run.

SSIS Conditional split final result
 


Remove Ads or Log in

We need you!

By becoming a proud member we can:
  • Switch off annoying ads
  • Create more articles and videos
  • Develop more useful features supporting your learning
  • Provide FREE webinars (for members)
  • Feel our hard work since 2011 is appreciated
Become a proud member and get exclusive features.

If 2% of our 64,943 visitors (Last 30 days) become a proud member than we can quit our jobs and work full time on this website. We started membership in May 2014 and so far 0.07% of our visitors are members and we would like to say THANK YOU to those who become a proud member and support our efforts.

Thanks to members:
  • Education is for FREE and we create more FREE articles and videos
  • We provide new features (members) that can speed up learning.
  • 53% of our page views (82,000 each month) do not have page ads.
  • We switched video ads from 14% of our videos (50 out of 345).
We have invested 4 years of our time in this website and we estimate it will take about 5 years before we can quit our jobs and dedicate all our time to develop considerably our website which should help to truly achieve our vision of free education for everyone; all thanks to support of our members only. Become a proud member

Visitor Voting

Did you know we (Katie & Emil) immigrated to UK in 2005 without knowing English, having any IT skills (or degree) and we set up a website in 2008 to learn by teaching?
Visitor Voting: Yes +13  |  No -6
Add comment (no email required).



Add Comment
No data was returned.


Are you a member?

If yes than please log in

Learning Map is for members with Bronze, Silver and Gold Membership.
For more information visit membership options.


Close this window , log in or learn more about membership benefits
  Become a Proud Member