DSXchange: DataStage and IBM Websphere Data Integration Forum
Search found 321 matches
Author Message
 Topic: ETL Modeling suggestions
UCDI

Replies: 11
Views: 198

PostForum: General   Posted: Thu May 24, 2018 11:15 am   Subject: ETL Modeling suggestions
I shamefully do not know what tool generates our HTML models. I can find out. Its *probably* an IBM tool, since we are heavily invested there, but it could be something else entire.

Unrelated.. ...
 Topic: non-database sparse lookup alternative options
UCDI

Replies: 16
Views: 136

PostForum: General   Posted: Thu May 24, 2018 11:13 am   Subject: non-database sparse lookup alternative options
SSD was me thinking of dumping the file into ram as fast as possible. I think the lookup stage is doing something more beyond just a grab --- it may not be the disk read time, but the tool, if it too ...
 Topic: Parameterized password in Stored Procedure stage
UCDI

Replies: 12
Views: 135

PostForum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)   Posted: Thu May 24, 2018 11:01 am   Subject: Parameterized password in Stored Procedure stage
NICE, I looked but did not find this list.
Looks like the one I was using had at least 5 offending characters in it..

Also seems like login to the tool does not matter what you have, but inside ...
 Topic: non-database sparse lookup alternative options
UCDI

Replies: 16
Views: 136

PostForum: General   Posted: Wed May 23, 2018 2:25 pm   Subject: non-database sparse lookup alternative options
ok, that is what I understood also. If you are OK going outside of datastage and databases, sure, its very simple to make a C program do this, again looking at a memory mapped file, or eating up a bu ...
 Topic: ETL Modeling suggestions
UCDI

Replies: 11
Views: 198

PostForum: General   Posted: Wed May 23, 2018 2:11 pm   Subject: ETL Modeling suggestions
We use FastTrack for mapping columns and the rules that define the mapping. Nothing else, but its been great for just that one task of keeping the mapping public, up to date, centralized, etc. The ...
 Topic: Parameterized password in Stored Procedure stage
UCDI

Replies: 12
Views: 135

PostForum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)   Posted: Wed May 23, 2018 2:05 pm   Subject: Parameterized password in Stored Procedure stage
this could be TOTALLY unrelated and off the wall, but let me describe a recent issue I was having...

I was having my password constantly rejected for a teradata system. My password was rather leng ...
 Topic: non-database sparse lookup alternative options
UCDI

Replies: 16
Views: 136

PostForum: General   Posted: Tue May 22, 2018 8:14 am   Subject: non-database sparse lookup alternative options
what does non-database mean?

you can do a lookup in a 'binary' fixed-width file with a hashing algorithm.
If it will fit, you can do that in memory even faster, or a memory mapped file.

but ...
Resolved
 Topic: set default value on look up failure
UCDI

Replies: 4
Views: 178

PostForum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)   Posted: Thu May 17, 2018 11:05 am   Subject: set default value on look up failure
if dealing with 'very' large volumes and few failed lookups you may find it more efficient to reject them and only transform the reject link, then funnel back. But generally, the above is the best wa ...
Resolved
 Topic: Convert space to null
UCDI

Replies: 8
Views: 158

PostForum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)   Posted: Wed May 16, 2018 1:40 pm   Subject: Convert space to null
I think that is all of them?
I only see a few also...
 Topic: Array Size, Record Count, Commit Frequency, Bulk Insert
UCDI

Replies: 8
Views: 393

PostForum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)   Posted: Tue May 15, 2018 10:05 am   Subject: Array Size, Record Count, Commit Frequency, Bulk Insert
have you done stats and index on the target? Inserts can go long if the stats are a mess, or if the table isnt indexed well for the job.

bulk works well too, if you go that route. Hardware and ...
 Topic: Reusable Job Design
UCDI

Replies: 5
Views: 201

PostForum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)   Posted: Tue May 15, 2018 9:53 am   Subject: Reusable Job Design
you can make the first part unique and the back end of the job reusable to protect your metadata. The front piece would do any manual conversions and corrections. Or you could make the RCP version, ...
 Topic: MQ connector vs empty queue
UCDI

Replies: 0
Views: 89

PostForum: General   Posted: Thu May 10, 2018 10:37 am   Subject: MQ connector vs empty queue
DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Hey experts,

I am having a little trouble trying to find the magic combination of settings for the MQ connector stage that allows it to 'do nothing' if there are no messages on the queue.

What ...
Resolved
 Topic: validate the existence of a file
UCDI

Replies: 9
Views: 208

PostForum: General   Posted: Wed May 09, 2018 12:54 pm   Subject: validate the existence of a file
in a sequence, execute command and parse the results of a ls or similar check is pretty easy for a binary presence answer without waiting on it to suddenly appear.
Resolved
 Topic: Replace Strings in a text file
UCDI

Replies: 14
Views: 739

PostForum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)   Posted: Thu May 03, 2018 11:39 am   Subject: Replace Strings in a text file
Datastage is poor at string processing, but it has a couple of solid ways to do this task.
DataStage server jobs are particularly good at string processing.

For some reason that I am not privy to ...
Resolved
 Topic: Replace Strings in a text file
UCDI

Replies: 14
Views: 739

PostForum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)   Posted: Tue May 01, 2018 11:22 am   Subject: Replace Strings in a text file
a VB routine could do this without a lot of pain.
you may also find it handy to use the unix shell commands in a parallel job. You can do this with a 2 line C program (literally wrap the command sy ...
 

 Jump to:   



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours