Search found 380 matches

by UCDI
Wed Nov 28, 2018 11:21 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Datastage -- Produces More Rows after each run
Replies: 2
Views: 2633

we have an outstanding ticket with IBM about the H-stage dropping a record here and there at random. We can run the same data 3 times and get {one record missed, all the data, different record missed} as the result. Whether you are seeing something similar or not, I can't say from your problem state...
by UCDI
Mon Nov 19, 2018 11:11 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Relationship between Nodes and Partitions...
Replies: 6
Views: 4438

If the data isn't suited to partitions "as is", run some of it through a Checksum stage and use that field as the partition key. It should give solid results.
by UCDI
Tue Oct 09, 2018 9:49 am
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: Calculating age using date
Replies: 9
Views: 26889

do you need some sort of logic to round up? That is doable but I have never heard of anyone doing that for your age in years. you can add a +1 if the # of days until next birthday is within N days, not hard to do...
by UCDI
Tue Aug 14, 2018 12:09 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Need DataStage to abort rather than loading 0 rows
Replies: 5
Views: 3527

Answering the subject and not the body... You can drop your extract to a dataset, count the # of records in the dataset with a routine, and use that to gracefully exit or proceed depending on having data or not. This is nice for efficiency as well for complex jobs that do a whole lot of nothing lik...
by UCDI
Thu Jul 26, 2018 2:04 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Can we copy Datasets from one server to another?
Replies: 11
Views: 7620

there is something that works for many small simple DS too: orchadmin dump -delim ',' dsname.ds outputname.csv but this is a quick fix for simple problems; there are times when it won't be sufficient, and its terrible for anything big. it will also lose the schema. But you can export a table def for...
by UCDI
Mon Jul 23, 2018 12:22 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Shifting and Truncating
Replies: 3
Views: 2635

a mix of substring operations should get you there, eg newvalue = left(oldvalue, 2):'S':right(oldvalue,4) newvalue2 = left(oldvalue, 2):'SS':right(oldvalue,3) ..etc do that in a stage variable is one way, eg if you wanted each value in a new column. use a loop if you wanted each one as a new record.
by UCDI
Mon Jul 23, 2018 12:18 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Can we copy Datasets from one server to another?
Replies: 11
Views: 7620

it is better if you do not do this. In an emergency, you can move the dataset AND its underlying files AND change the 'header' file (the one you interface to in datastage when you open or write a dataset) to point to the new locations. The header file is in some sort of unicodeish partial text parti...
by UCDI
Fri Jul 13, 2018 2:15 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Handling Junk Characters
Replies: 2
Views: 2759

to just remove the records entirely you can do it with a single convert, or possibly even more efficiently with 'alpha' if that fits the data need? A single convert, you convert all the valid chars to nothing and then check the length of the remainder. double convert fixes the data, which is a diffe...
by UCDI
Fri Jul 13, 2018 2:07 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Edit / Combine DSX with same target table?
Replies: 9
Views: 6819

if you do anything manually more than 2 or 3 times, its time to automate it. If none of the datastage ideas works for you, write something that will do the surgery for you. And its not always that much code. The one I did to transfer job params from one job to another took like 2 hours or less from ...
by UCDI
Mon Jul 02, 2018 9:19 am
Forum: General
Topic: Best way to handle password parameters
Replies: 9
Views: 6831

We do it that way. I can't comment if its the most secure way or not. I just checked 11.5 and you can extract the password with the simplest imaginable attack. Nothing even interesting, just pure datastage jobs with the right settings will print the password in the logs. So I would offer that encryp...
by UCDI
Wed Jun 27, 2018 10:38 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Edit / Combine DSX with same target table?
Replies: 9
Views: 6819

dsx hacking is really only useful for very simple things like copying job parameters from one job to another ( a feature that is sadly lacking), or the occasional overarching search and replace of 1 word or something. I don't understand enough of your design, but it seems to me you could use a share...
by UCDI
Fri Jun 08, 2018 1:00 pm
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Recursive Fold ! Tricky
Replies: 2
Views: 3094

do you need to handle leap years? that aside, transformer loops create a record per loop, and are handy but you already have all your records as you want them (right?) and are just grouping them so it may not be a clean solution (you can toss extra records, but its unnecessary work best avoided to g...
by UCDI
Thu Jun 07, 2018 11:01 am
Forum: General
Topic: Job seqencer killing of other process.
Replies: 3
Views: 4805

so you split to two, one of those goes into another seq, that one is set to any (?) and the outer one is set to all? This would mean that the any one can finish one thing, pass that to the all outer, and the all would be set to true for that side, so if the other side finishes as well, some of the '...
by UCDI
Thu Jun 07, 2018 8:17 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Convert Issue - Non printable characters ÿ and ›
Replies: 7
Views: 5817

Heh... about that... Look in your repository under stage types. There are a number of stages in there that are not in the palette. Its under Parallel/Processing/BasicTransformer Be careful with it. I know for sure that it cannot handle timestamps with microseconds, it loses the extended data. I don'...
by UCDI
Tue Jun 05, 2018 8:32 am
Forum: General
Topic: how to pass filename to sequential file stage in runtime
Replies: 9
Views: 5957

you can do it in a parallel job too if you use vb transformer or one of the others, I think there is a java one etc? most of the time this sounds like a parallel job (with a parameter) invoked from a sequence that has either a routine or user-variable stage to cook up the text. For file name that ju...