Use of C++ compiler in promotion process

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
bullt9s
Participant
Posts: 1
Joined: Thu Nov 20, 2003 9:30 am

Use of C++ compiler in promotion process

Post by bullt9s »

I'm curious to learn how others are promoting DataStage code to production. We've been told that we must recompile our applications for every environment we promote our code to if we do transforms. We want to compile the code once and then promote the objects to our pre-production and then production environments. We really don't want to have a C++ compiler active in our production environment for various reasons. What do others do? Are we dreaming?

We are using DataStage XE & Parallel Extender on HP-UX platform. Thanks for your thoughts.

Bullt9s
vasu
Premium Member
Premium Member
Posts: 18
Joined: Tue Dec 31, 2002 3:04 am

Re: Use of C++ compiler in promotion process

Post by vasu »

[Vasu]
As per my testing PX jobs needed re-compilation in Production environment. For re-compilation C++ compiler also needed.

My Platform on AIX 5.2
I am getting fatal error during my testing.
Event Type: Fatal
0509-022 Cannot load module
0509-103 The module has an invalid magic number.


bullt9s wrote:I'm curious to learn how others are promoting DataStage code to production. We've been told that we must recompile our applications for every environment we promote our code to if we do transforms. We want to compile the code once and then promote the objects to our pre-production and then production environments. We really don't want to have a C++ compiler active in our production environment for various reasons. What do others do? Are we dreaming?

We are using DataStage XE & Parallel Extender on HP-UX platform. Thanks for your thoughts.

Bullt9s
Thanks
Vasu
Eric
Participant
Posts: 254
Joined: Mon Sep 29, 2003 4:35 am

Post by Eric »

If the pre-production and production environments are the same (Same version and patch level of OS) then you won't need to recompile in in the production environment.
Teej
Participant
Posts: 677
Joined: Fri Aug 08, 2003 9:26 am
Location: USA

Re: Use of C++ compiler in promotion process

Post by Teej »

bullt9s wrote:We really don't want to have a C++ compiler active in our production environment for various reasons.
We had the same belief here, but it was a pipe dream.

Buildop MUST be recompiled unless you want to manually move the binaries.

Jobs WILL be recompiled if there are any slight variations on each machines which WILL happens when you upgrade Developer box to 6.0.2 and leave Production at 6.0.1.

Jobs WILL have to be recompiled when you upgrade Production boxes during installation. It could be bypassed, but this is NOT advisable (there are distinct differences between versions that can bite you.)

This and other reasons led us to purchase a C++ compiler option, and lock down the production machine to a select few users. You could lock down your Production servers via Version Control, but Version Control WILL invoke compiling on the production box. (It only copy the source code.)

-T.J.
Developer of DataStage Parallel Engine (Orchestrate).
vasu
Premium Member
Premium Member
Posts: 18
Joined: Tue Dec 31, 2002 3:04 am

Post by vasu »

[vasu]

First of all on the same machine if you move PX job to one project to another project, PX job failed to execute with out compilation.

"If the pre-production and production environments are the same (Same version and patch level of OS) then you won't need to recompile in in the production environment."
Thanks
Vasu
Teej
Participant
Posts: 677
Joined: Fri Aug 08, 2003 9:26 am
Location: USA

Post by Teej »

vasu wrote:First of all on the same machine if you move PX job to one project to another project, PX job failed to execute with out compilation.
If you extract the DSX, just extract the Job Executables along with its source code. It should load just fine on the other project on the same server.

-T.J.
Developer of DataStage Parallel Engine (Orchestrate).
vasu
Premium Member
Premium Member
Posts: 18
Joined: Tue Dec 31, 2002 3:04 am

Post by vasu »

[vasu]
Hi TJ,
For server jobs point of your statement is correct. I have done once again PX job testing on the same machine(O.S: AIX5.2, Export: Job Designs, Job Executable, Program source options selected). These errors I am getting.
0509-022 Cannot load module
0509-103 The module has an invalid magic number
Can I know your environment O.S on PX
Teej wrote:
vasu wrote:First of all on the same machine if you move PX job to one project to another project, PX job failed to execute with out compilation.
If you extract the DSX, just extract the Job Executables along with its source code. It should load just fine on the other project on the same server.

-T.J.
Thanks
Vasu
clshore
Charter Member
Charter Member
Posts: 115
Joined: Tue Oct 21, 2003 11:45 am

Post by clshore »

Just an observation that the PX 'compile' does not automatically result in a call to the C++ compiler. It causes the appropriate osh code to be created, which in turn *may* involve a C++ compiler.
Teej
Participant
Posts: 677
Joined: Fri Aug 08, 2003 9:26 am
Location: USA

Post by Teej »

vasu wrote:0509-022 Cannot load module
0509-103 The module has an invalid magic number
Can I know your environment O.S on PX
Hmm, never saw that one before, even on a server without a C++ compiler (always get a different message particularly complaining that there is no available C++ compiler.)

We uses Tru64 here.

-T.J.
Developer of DataStage Parallel Engine (Orchestrate).
vasu
Premium Member
Premium Member
Posts: 18
Joined: Tue Dec 31, 2002 3:04 am

Post by vasu »

Hi TJ,

Is this visible "PX Compiled jobs can run without c++ compiler if I have the same operating system and patch level. The only machine difference is in terms of Processor, RAM, Physical Memory."

Vasu

Hmm, never saw that one before, even on a server without a C++ compiler (always get a different message particularly complaining that there is no available C++ compiler.)

We uses Tru64 here.

-T.J.[/quote]
Last edited by vasu on Tue Jan 13, 2004 10:18 pm, edited 1 time in total.
Thanks
Vasu
Teej
Participant
Posts: 677
Joined: Fri Aug 08, 2003 9:26 am
Location: USA

Post by Teej »

vasu wrote:I mailed the exact error message to your forum Inbox mail. Is this visible "PX Compiled jobs can run without c++ compiler if I have the same operating system and patch level. The only machine difference is in terms of Processor, RAM, Physical Memory."
I took another look at it, and suddenly this gave me a clue:

Code: Select all

0509-022 Cannot load module /home/dsadm/Ascential/DataStage/Projects/test/RT_BP9.O/V0S1_compilertest_Transformer_1.o. 
0509-103 The module has an invalid magic number.. 
This somehow tells me that the actual Project folders were changed/migrated. Has anyone here ever seen the 'magic number' error before?

-T.J.
Developer of DataStage Parallel Engine (Orchestrate).
vasu
Premium Member
Premium Member
Posts: 18
Joined: Tue Dec 31, 2002 3:04 am

Post by vasu »

Hi TJ,

I hope this is not the reason"actual Project folders were changed/migrated". Because this type of test also failed.

I have developed a Px job on this project directory "Prj_test".
I exported this PX job as cTestz.dsx.
I deleted this "Prj_test" project on the DataStage server.
Then again created the "Prj_test" project on the DataStage server on the same location and same name.
I imported that Px job in newly created "Prj_test" project.
I executed the PX job
Same error I got.
Note: On the Same machine I have done this test.
JOB Number and file location"V0S1_compilertest_Transformer_1.o" are same.

Vasu

I took another look at it, and suddenly this gave me a clue:

Code: Select all

0509-022 Cannot load module /home/dsadm/Ascential/DataStage/Projects/test/RT_BP9.O/V0S1_compilertest_Transformer_1.o. 
0509-103 The module has an invalid magic number.. 
This somehow tells me that the actual Project folders were changed/migrated. Has anyone here ever seen the 'magic number' error before?

-T.J.[/quote]
Last edited by vasu on Tue Jan 13, 2004 11:34 pm, edited 1 time in total.
Thanks
Vasu
kcbland
Participant
Posts: 5208
Joined: Wed Jan 15, 2003 8:56 am
Location: Lutz, FL
Contact:

Post by kcbland »

42
Kenneth Bland

Rank: Sempai
Belt: First degree black
Fight name: Captain Hook
Signature knockout: right upper cut followed by left hook
Signature submission: Crucifix combined with leg triangle
bigpoppa
Participant
Posts: 190
Joined: Fri Feb 28, 2003 11:39 am

Use of C++ compiler in promotion process

Post by bigpoppa »

It seems like you have a link-loader problem. Using the environment vars you set for the bin and lib within PX, you should be able to compile a simple c program outside PX. If that doesn't work, then you have set up your env improperly within PX.

-BP
Teej
Participant
Posts: 677
Joined: Fri Aug 08, 2003 9:26 am
Location: USA

Post by Teej »

Ken. You are damn lucky I was not drinking at that time.

-T.J.
Developer of DataStage Parallel Engine (Orchestrate).
Post Reply