What is the difference between a Transform and a Routine that is defined of type Transform Function? When should I use routines and when should I use transforms?
Thanks.
Transforms and Routines
Routines and Transforms are both used to capture "re-usable" transformation logic. Historically, Transforms used to have more relevance, as the DataStage Designer specifically reacted to their meta data and performed specific validations based upon the "Data Element" (a notable part of the dialogue used to create Transforms) selected for the Transform. However, our customers preferred that we make Routines and Transforms more equally accessible in a Transformer Stage, so those differences today are not as apparent.
A DataStage Routine is certainly the best place to go when you have some
"detailed" logic that you want to re-use or preserve. While you can write
very complicated and long streams of code in a routine, routines get used most often when you simply want to capture the 8 to 10 lines of code for a common transformation (ie...the edit of a factory code that requires left justification, substring, and upcase for instance --- or a static IF...THEN...ELSE sequence).
For simpler sequences of code, a formal Transform is more efficient in the way that it is loaded and utilized by the DataStage engine. No big deal if you are processing only 10,000 rows, but significant if you are processing
many millions. If a sequence of lower level granular functions can be
represented in a single line (right hand side of an equation), then make it into a Transform. It will be far more efficient. This can be seen in the
"sdk" category that comes installed with the product. Check thru the
examples and you will see logic that could certainly be represented as a set of iterative steps in a Routine, but is far more efficient when placed into a single line.
I generally like to use Routines initially, especially so that I can include comments in the code and address any necessary complexity. However, if I find that the routine is simple enough to be reduced to a Transform, then all the better because the performance gain is worthwhile.
-----Original Message-----
From: Roosa, Mike [mailto:Mike.Roosa@getronics.com]
Sent: Friday, December 15, 2000 4:12 PM
To: informix-datastage@oliver.com
Subject: Transforms and Routines
What is the difference between a Transform and a Routine that is defined of type Transform Function? When should I use routines and when should I use transforms?
Thanks.
A DataStage Routine is certainly the best place to go when you have some
"detailed" logic that you want to re-use or preserve. While you can write
very complicated and long streams of code in a routine, routines get used most often when you simply want to capture the 8 to 10 lines of code for a common transformation (ie...the edit of a factory code that requires left justification, substring, and upcase for instance --- or a static IF...THEN...ELSE sequence).
For simpler sequences of code, a formal Transform is more efficient in the way that it is loaded and utilized by the DataStage engine. No big deal if you are processing only 10,000 rows, but significant if you are processing
many millions. If a sequence of lower level granular functions can be
represented in a single line (right hand side of an equation), then make it into a Transform. It will be far more efficient. This can be seen in the
"sdk" category that comes installed with the product. Check thru the
examples and you will see logic that could certainly be represented as a set of iterative steps in a Routine, but is far more efficient when placed into a single line.
I generally like to use Routines initially, especially so that I can include comments in the code and address any necessary complexity. However, if I find that the routine is simple enough to be reduced to a Transform, then all the better because the performance gain is worthwhile.
-----Original Message-----
From: Roosa, Mike [mailto:Mike.Roosa@getronics.com]
Sent: Friday, December 15, 2000 4:12 PM
To: informix-datastage@oliver.com
Subject: Transforms and Routines
What is the difference between a Transform and a Routine that is defined of type Transform Function? When should I use routines and when should I use transforms?
Thanks.
Cant do much better than Ernies answer except, perhaps, to add to it a bit. I do like Transforms.
Once upon a time, you could only invoke a transform function (routine) through a Transform. There was an esoteric engineering reason for this, but Ive forgotten what it was. Something to do with insulating the code.
Transforms, to my mind, continue to offer two major benefits. First, they allow you to create reusable components for frequently used transformations. While this is true also for routines, at run time a routine must be called (and therefore separately loaded). The definition expression of a Transform becomes in-line code at compile time so, as Ernie stated, is slightly more efficient. For example, I often implement a Transform called ReplaceNullWithZero, because this is a commonly performed task. Second, they save me having to remember conversion constants, and re-invent the wheel over and over. This is more relevant when you are obtaining data from more than one country, and have to convert, say, miles to kilometres, pounds to kilograms or gallons to litres; things like that. A lot of these already exist in the SDK.
And, like Ernie, where the processing is a little more complex, or obscure, I prefer a Routine, where the code can be made more easily maintainable and internally documented. (That said, I do create comprehensive documentation in the long description of Transforms, too, to make the life of whoever has to maintain it down the years that much easier. Remember that the DW is going to be there for a long time!)
Once upon a time, you could only invoke a transform function (routine) through a Transform. There was an esoteric engineering reason for this, but Ive forgotten what it was. Something to do with insulating the code.
Transforms, to my mind, continue to offer two major benefits. First, they allow you to create reusable components for frequently used transformations. While this is true also for routines, at run time a routine must be called (and therefore separately loaded). The definition expression of a Transform becomes in-line code at compile time so, as Ernie stated, is slightly more efficient. For example, I often implement a Transform called ReplaceNullWithZero, because this is a commonly performed task. Second, they save me having to remember conversion constants, and re-invent the wheel over and over. This is more relevant when you are obtaining data from more than one country, and have to convert, say, miles to kilometres, pounds to kilograms or gallons to litres; things like that. A lot of these already exist in the SDK.
And, like Ernie, where the processing is a little more complex, or obscure, I prefer a Routine, where the code can be made more easily maintainable and internally documented. (That said, I do create comprehensive documentation in the long description of Transforms, too, to make the life of whoever has to maintain it down the years that much easier. Remember that the DW is going to be there for a long time!)