DSXchange: DataStage and IBM Websphere Data Integration Forum
Search found 315 matches
Author Message
 Topic: non-database sparse lookup alternative options
UCDI

Replies: 2
Views: 20

PostForum: General   Posted: Tue May 22, 2018 8:14 am   Subject: non-database sparse lookup alternative options
what does non-database mean?

you can do a lookup in a 'binary' fixed-width file with a hashing algorithm.
If it will fit, you can do that in memory even faster, or a memory mapped file.

but ...
Resolved
 Topic: set default value on look up failure
UCDI

Replies: 4
Views: 151

PostForum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)   Posted: Thu May 17, 2018 11:05 am   Subject: set default value on look up failure
if dealing with 'very' large volumes and few failed lookups you may find it more efficient to reject them and only transform the reject link, then funnel back. But generally, the above is the best wa ...
Resolved
 Topic: Convert space to null
UCDI

Replies: 8
Views: 129

PostForum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)   Posted: Wed May 16, 2018 1:40 pm   Subject: Convert space to null
I think that is all of them?
I only see a few also...
 Topic: Array Size, Record Count, Commit Frequency, Bulk Insert
UCDI

Replies: 8
Views: 330

PostForum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)   Posted: Tue May 15, 2018 10:05 am   Subject: Array Size, Record Count, Commit Frequency, Bulk Insert
have you done stats and index on the target? Inserts can go long if the stats are a mess, or if the table isnt indexed well for the job.

bulk works well too, if you go that route. Hardware and ...
 Topic: Reusable Job Design
UCDI

Replies: 5
Views: 172

PostForum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)   Posted: Tue May 15, 2018 9:53 am   Subject: Reusable Job Design
you can make the first part unique and the back end of the job reusable to protect your metadata. The front piece would do any manual conversions and corrections. Or you could make the RCP version, ...
 Topic: MQ connector vs empty queue
UCDI

Replies: 0
Views: 72

PostForum: General   Posted: Thu May 10, 2018 10:37 am   Subject: MQ connector vs empty queue
DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Hey experts,

I am having a little trouble trying to find the magic combination of settings for the MQ connector stage that allows it to 'do nothing' if there are no messages on the queue.

What ...
Resolved
 Topic: validate the existence of a file
UCDI

Replies: 9
Views: 175

PostForum: General   Posted: Wed May 09, 2018 12:54 pm   Subject: validate the existence of a file
in a sequence, execute command and parse the results of a ls or similar check is pretty easy for a binary presence answer without waiting on it to suddenly appear.
Resolved
 Topic: Replace Strings in a text file
UCDI

Replies: 14
Views: 715

PostForum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)   Posted: Thu May 03, 2018 11:39 am   Subject: Replace Strings in a text file
Datastage is poor at string processing, but it has a couple of solid ways to do this task.
DataStage server jobs are particularly good at string processing.

For some reason that I am not privy to ...
Resolved
 Topic: Replace Strings in a text file
UCDI

Replies: 14
Views: 715

PostForum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)   Posted: Tue May 01, 2018 11:22 am   Subject: Replace Strings in a text file
a VB routine could do this without a lot of pain.
you may also find it handy to use the unix shell commands in a parallel job. You can do this with a 2 line C program (literally wrap the command sy ...
 Topic: Reading encrypted password into user variable
UCDI

Replies: 13
Views: 422

PostForum: General   Posted: Tue May 01, 2018 10:22 am   Subject: Reading encrypted password into user variable
In case someone else is wondering, I found a way to pull this off, albeit a little less elegantly than I was hoping. We have a routine that will run a DataStage job by name with parameters passed in ...
 Topic: Reading encrypted password into user variable
UCDI

Replies: 13
Views: 422

PostForum: General   Posted: Fri Apr 27, 2018 8:14 am   Subject: Reading encrypted password into user variable
Try using values files in a Parameter Set, and use logic to choose which values file to implement.
Thanks for the suggestion. I considered that option, but the issue is that I need both the ability ...
 Topic: Reading encrypted password into user variable
UCDI

Replies: 13
Views: 422

PostForum: General   Posted: Thu Apr 26, 2018 9:38 am   Subject: Reading encrypted password into user variable
First likely bad assumption: you're storing the variables in the project repository as User defined environment variables. If that's not true, my suggestion is not appropriate. I thought that was true ...
 Topic: Reading encrypted password into user variable
UCDI

Replies: 13
Views: 422

PostForum: General   Posted: Wed Apr 25, 2018 2:54 pm   Subject: Reading encrypted password into user variable
Name the variable in the stage for the password.

It depends on the stage, of course. If the password attribute prompts for two entries for the password, one types in the variable name.

Look for ...
 Topic: Reading encrypted password into user variable
UCDI

Replies: 13
Views: 422

PostForum: General   Posted: Wed Apr 25, 2018 12:06 pm   Subject: Reading encrypted password into user variable
DataStage® Release: 11x
Job Type: Parallel
OS: Unix
Additional info: Database connection failing
We store database connection info (DSN, user, password) in environment variables. I would like to develop a database-agnostic sequence that would dynamically read database connection info (through a ...
 Topic: File Pattern When File Name Contains Quotes
UCDI

Replies: 3
Views: 221

PostForum: IBM® DataStage Enterprise Edition (Formerly Parallel Extender/PX)   Posted: Thu Apr 19, 2018 7:45 am   Subject: File Pattern When File Name Contains Quotes
what about:

run command:
copy offendingfilename to newfilename
proceed.
end of job: run command delete newfilename

The run command thing does not really care. It will gladly attempt to invo ...
 

 Jump to:   



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours