| Author |
Message |
|
Topic: ETL Modeling suggestions |
UCDI
Replies: 11
Views: 198
|
 Forum: General
Posted: Thu May 24, 2018 11:15 am Subject: ETL Modeling suggestions |
I shamefully do not know what tool generates our HTML models. I can find out. Its *probably* an IBM tool, since we are heavily invested there, but it could be something else entire.
Unrelated..
...
|
|
Topic: non-database sparse lookup alternative options |
UCDI
Replies: 16
Views: 136
|
Forum: General
Posted: Thu May 24, 2018 11:13 am Subject: non-database sparse lookup alternative options |
|
SSD was me thinking of dumping the file into ram as fast as possible. I think the lookup stage is doing something more beyond just a grab --- it may not be the disk read time, but the tool, if it too ...
|
|
Topic: Parameterized password in Stored Procedure stage |
UCDI
Replies: 12
Views: 135
|
Forum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Posted: Thu May 24, 2018 11:01 am Subject: Parameterized password in Stored Procedure stage |
NICE, I looked but did not find this list.
Looks like the one I was using had at least 5 offending characters in it..
Also seems like login to the tool does not matter what you have, but inside ...
|
|
Topic: non-database sparse lookup alternative options |
UCDI
Replies: 16
Views: 136
|
Forum: General
Posted: Wed May 23, 2018 2:25 pm Subject: non-database sparse lookup alternative options |
|
ok, that is what I understood also. If you are OK going outside of datastage and databases, sure, its very simple to make a C program do this, again looking at a memory mapped file, or eating up a bu ...
|
|
Topic: ETL Modeling suggestions |
UCDI
Replies: 11
Views: 198
|
Forum: General
Posted: Wed May 23, 2018 2:11 pm Subject: ETL Modeling suggestions |
|
We use FastTrack for mapping columns and the rules that define the mapping. Nothing else, but its been great for just that one task of keeping the mapping public, up to date, centralized, etc. The ...
|
|
Topic: Parameterized password in Stored Procedure stage |
UCDI
Replies: 12
Views: 135
|
Forum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Posted: Wed May 23, 2018 2:05 pm Subject: Parameterized password in Stored Procedure stage |
this could be TOTALLY unrelated and off the wall, but let me describe a recent issue I was having...
I was having my password constantly rejected for a teradata system. My password was rather leng ...
|
|
Topic: non-database sparse lookup alternative options |
UCDI
Replies: 16
Views: 136
|
Forum: General
Posted: Tue May 22, 2018 8:14 am Subject: non-database sparse lookup alternative options |
what does non-database mean?
you can do a lookup in a 'binary' fixed-width file with a hashing algorithm.
If it will fit, you can do that in memory even faster, or a memory mapped file.
but ...
|
|
Topic: set default value on look up failure |
UCDI
Replies: 4
Views: 178
|
Forum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Posted: Thu May 17, 2018 11:05 am Subject: set default value on look up failure |
|
if dealing with 'very' large volumes and few failed lookups you may find it more efficient to reject them and only transform the reject link, then funnel back. But generally, the above is the best wa ...
|
|
Topic: Convert space to null |
UCDI
Replies: 8
Views: 158
|
Forum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Posted: Wed May 16, 2018 1:40 pm Subject: Convert space to null |
I think that is all of them?
I only see a few also...
|
|
Topic: Array Size, Record Count, Commit Frequency, Bulk Insert |
UCDI
Replies: 8
Views: 393
|
Forum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Posted: Tue May 15, 2018 10:05 am Subject: Array Size, Record Count, Commit Frequency, Bulk Insert |
have you done stats and index on the target? Inserts can go long if the stats are a mess, or if the table isnt indexed well for the job.
bulk works well too, if you go that route. Hardware and ...
|
|
Topic: Reusable Job Design |
UCDI
Replies: 5
Views: 201
|
Forum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Posted: Tue May 15, 2018 9:53 am Subject: Reusable Job Design |
|
you can make the first part unique and the back end of the job reusable to protect your metadata. The front piece would do any manual conversions and corrections. Or you could make the RCP version, ...
|
|
Topic: MQ connector vs empty queue |
UCDI
Replies: 0
Views: 89
|
Forum: General
Posted: Thu May 10, 2018 10:37 am Subject: MQ connector vs empty queue |
DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Hey experts,
I am having a little trouble trying to find the magic combination of settings for the MQ connector stage that allows it to 'do nothing' if there are no messages on the queue.
What ...
|
|
Topic: validate the existence of a file |
UCDI
Replies: 9
Views: 208
|
Forum: General
Posted: Wed May 09, 2018 12:54 pm Subject: validate the existence of a file |
|
in a sequence, execute command and parse the results of a ls or similar check is pretty easy for a binary presence answer without waiting on it to suddenly appear.
|
|
Topic: Replace Strings in a text file |
UCDI
Replies: 14
Views: 739
|
Forum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Posted: Thu May 03, 2018 11:39 am Subject: Replace Strings in a text file |
Datastage is poor at string processing, but it has a couple of solid ways to do this task.
DataStage server jobs are particularly good at string processing.
For some reason that I am not privy to ...
|
|
Topic: Replace Strings in a text file |
UCDI
Replies: 14
Views: 739
|
Forum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Posted: Tue May 01, 2018 11:22 am Subject: Replace Strings in a text file |
a VB routine could do this without a lot of pain.
you may also find it handy to use the unix shell commands in a parallel job. You can do this with a 2 line C program (literally wrap the command sy ...
|
| |
