Search found 103 matches

by jgreve
Tue Oct 24, 2006 6:31 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: XMLInputPX stage - output columns: derivation purpose?
Replies: 2
Views: 1066

chulett wrote:Short answer - no. One of the joys of using a standard 'widget' for an interface, not all fields apply to all stages.
Outstanding!
Thank you.

I can live with the "hand-me-down" ui widgets.
It is not knowing which bits are relevant
that makes me crazy.
John G.
by jgreve
Mon Oct 23, 2006 7:14 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: XMLInputPX stage - output columns: derivation purpose?
Replies: 2
Views: 1066

XMLInputPX stage - output columns: derivation purpose?

Does "Derivation" actually do anything for XML Input PX stage? I ask because it isn't mentioned in the "XML PACK" (XMLPACK_20_Designer.pdf) docs and doesn't seem to have many posts in the PX-forum here. Anyway, XMLInput statge, Output tab, Columns has the following table: Column ...
by jgreve
Thu Oct 19, 2006 8:35 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Advice on generating OSH script
Replies: 10
Views: 4448

more background please Re: Advice on generating OSH script

How similar is "very similar"? Like, "Basically the same job w/hundreds different sources (non-matching schemas but a similar purpose) that map to a single target schema?" What drives your mappings, e.g. manually launched? Batch schedule? Real-time? It would be easier to specula...
by jgreve
Wed Oct 18, 2006 6:25 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Job failure: lookups on datasets
Replies: 6
Views: 1970

Chris, can your DS write to C:\TEMP\bogus.ds instead?

(a) you appear to have "=" rather than ":" after the drive letter (b) the pathname should be that of the control file; that is, end at ".ds" RW, that makes sense. While I would swear I saw path-names somewhere (like in KSH or OSH) using C=/whatever (instead of the norm...
by jgreve
Wed Oct 18, 2006 4:04 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Parallel job is being Aborted -- showing Not enough space
Replies: 14
Views: 8197

Maybe disable memory-mapping?

I wonder if setting these env-vars in your job would let it run? APT_BUFFERIO_NOMAP=1 APT_IO_NOMAP=1 The memory-map idea is a great performance booster, but if your nodes are very busy, there might not be that much memory available. Or, as Ultramundane suggested, changing your kernel settings to hav...
by jgreve
Wed Oct 18, 2006 9:15 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: XML Extraction from Sequential Stage ?
Replies: 8
Views: 5241

Re: XML Extraction from Sequential Stage ?

File Pattern : acential/datastage/folder/xml/data/*.xml Read Method: File Pattern Rajan.n had a good idea, try changing your file pattern to be absolute, e.g. start with a leading slash. Unix might be: /acential/datastage/folder/xml/data/*.xml Windows might be: C:/acential/datastage/folder/xml/data...
by jgreve
Fri Oct 13, 2006 7:59 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Can ChangeApply stage be useful?
Replies: 2
Views: 935

change-capture theory

I currently speccing an environment where the transformations run 24x7 on a DataStage grid but the database load should only occur overnight so you don't impact the users. One method for doing this quickly is to take a copy of the target table into a dataset overnight and use it for processing duri...
by jgreve
Fri Oct 13, 2006 11:42 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Can ChangeApply stage be useful?
Replies: 2
Views: 935

Can ChangeApply stage be useful?

Has anyone run into a real-world case where ChangeApply stage was helpful? I understand what it does, but I'm struggling to think of a case where I would want to use it. The only thing I've thought of so far would be a storage-optimization, so I can keep change-sets on hand instead of individual &qu...
by jgreve
Thu Oct 12, 2006 3:50 pm
Forum: General
Topic: Filter stage - documentation error?
Replies: 4
Views: 4052

It's correct. Logic is left to right where there are no parentheses. So the two AND operators will have been evaluated before it gets to the OR operator. (Optimized it may not even get to the OR operator if one of the preceding conditions is not true.) Ok, that makes sense. Thank you. *sigh* What w...
by jgreve
Wed Oct 11, 2006 4:36 pm
Forum: General
Topic: Filter stage - documentation error?
Replies: 4
Views: 4052

Filter stage - documentation error?

Here's an example from (parjdev.pdf) Parallel Job Developers Guide. I'm wondering if the docs are accurate? Filter Stage 29-5 ( p.682) -----begin excerpt----- Evaluating Input Columns You want to evaluate each input row to see if these conditions prevail: * EITHER all the following are true - Column...
by jgreve
Thu Oct 05, 2006 8:43 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Splitting a Large text file into small files
Replies: 13
Views: 5440

Well, good luck! My unix introduction was painful :) I trapped myself in the "cat" command (had to reboot the machine to get out of it!) I've learned a lot since then. If you're doing this on windows, go get a program call VIM (vim.org) - that is better than the "vi" includes. Sp...
by jgreve
Wed Oct 04, 2006 7:13 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Data stage Configuration File
Replies: 8
Views: 13251

ok, so how about posting some background info?

Currently, my cofiguration is set to 8 nodes on a 16 CPU system (SMP), it has a SAN and all my disk and scratch space is set to fs_stage_dataset/Dataset or fs_stage_dataset/Scratch. We have issues like SIGKILL, SIGBUS and these are due to resource constraints. One of the frequent errors that we hav...
by jgreve
Wed Oct 04, 2006 6:54 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Splitting a Large text file into small files
Replies: 13
Views: 5440

minor script, one-shot effort

I'd do a KSH script like this... thum's for file in * one looks fine, too. Depending on how I set the loop up, the files would expand (glob) at invocation time so it had an empty list to chew on - hence the find in mine. I like the date format %Y%m%d, which generates 20061004. "man date" w...