HOW CAN I USE BUILDOPS IN PARALLEL???

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
MARKO970
Participant
Posts: 20
Joined: Thu Nov 13, 2003 5:46 am

HOW CAN I USE BUILDOPS IN PARALLEL???

Post by MARKO970 »

HI ALL,

IN MY PRECEDENT TOPIC I ASK AS I CAN USE DS FUNCTION BUT TEEJ
TOLD ME THAT I HAVE TO USE THE BUILDOPS ... WHAT ARE AND HOW CAN I USE THEM ???
I 'VE WORKED WITH DATASTAGE SERVER AND I DON'T KNOW THEM...
THANK'S FOR HELP!
bigpoppa
Participant
Posts: 190
Joined: Fri Feb 28, 2003 11:39 am

HOW CAN I USE BUILDOPS IN PARALLEL???

Post by bigpoppa »

A PX buildop is a user-defined c program, written in a PX proprietary style, that does record processing and data transformation.

To write one, you will need need to use the PX "New Parallel Stage" option in the Tools stage.

The PX documentation should have some good examples of how to build simple buildops.

I recommend that you build a simple buildop before you try to build one with complex transformation logic. Just get the hang of supplying inputs and outputs to the buildop and get a handle on the record-processing macros before you jump into a complex buildop.

-BP
Teej
Participant
Posts: 677
Joined: Fri Aug 08, 2003 9:26 am
Location: USA

Post by Teej »

Here's a little handbook on BuildOPS based on my experience. Wish I could go there and actually guide you through the whole concept.

BuildOPS and CustomOPS are primarly C++ code. BuildOPS's source and compiled code can be found under your Project folder's 'buildop' folder on the UNIX platform (dunno about Windows, but I believe PX is not yet supported there). The source code for your CustomOPS are usually within your Project folder. Both paths can be redefined by Administrator.

BuildOPS are compiled by DataStage itself. You are responsible for compiling CustomOPS yourself.

BuildOPS are limited to a specific table definition for input and output. CustomOPS can be vey flexible.

BuildOPS are purely C++ with no separate functions. CustomOPS can use anything as long as the landing functions are defined in C++. Pro*C, Fortran, Pascal, whatever.

BuildOPS are generally used for a specific purpose. CustomOPS are generally used like any normal stages.

Both requires at a minimum basic understanding of C/C++.

* * *

Detailed documentations for both can be found on your client installation CD entitled "Parallel Job Developer's Guide" (parjdev.pdf), Chapter 49 - "Specifying Custom Parallel Stage". For BuildOPS, scroll down to page 49-7.

The power behind the BuildOPS is its ability to execute specific code BEFORE and AFTER the stage get records. So if you want to report specific count of records only once -- easy doing. If you want to dynamically set the default value of a parameter befor the job -- easy doing.

You can also have multiple inputs and outputs. Need to do a double lookup comparing in upper-case for both? No problem.

The disadvantage of doing a buildop -- performance costs. Yes, it's compiled and all, but unlike the stages Ascential provided, there is very little you can really do in way of performance tuning without basically converting the code to a real CustomOPS stage. So if there is a way to do it that is provided by Ascential, stick to it unless you can conclusively prove that your buildop will perform better.

I am here for any specific questions you may have while you experiment with BuildOPS.

Good Luck.

-T.J.
Developer of DataStage Parallel Engine (Orchestrate).
Post Reply