Search found 251 matches

by rleishman
Thu Feb 22, 2007 7:45 pm
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: incremental loading
Replies: 6
Views: 3456

IMO this is better done at the extraction phase, not at load time. For delta-extraction, you can use one of the methods described above, or if extracting from an Oracle database you can look into Change Data Capture - it's a non-invasive way to capture all the changes on a table without timestamp co...
by rleishman
Thu Jan 18, 2007 12:54 am
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: Job Start TIme and Job End Time
Replies: 3
Views: 1479

A long time ago, I tried running the BuiltIn DSJobReport subroutine as the last activity of a Job Sequence, instead of from the After-Job subroutine of a server job, as is intended. From memory, it mostly worked, except the status showed as RUNNING, insteead of FINISHED. I chucked in the idea around...
by rleishman
Wed Jan 17, 2007 4:28 pm
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: How to assign value from table to environment variable?
Replies: 7
Views: 1939

Once again, I'll have a go at a free answer... If the second job is actually a Job Sequence, and the parameter you assign with $UserStatus in the second job is an Environment Variable (declared in the Enviroment section in Administrator), then every job subordinate to Job2 will automatically inherit...
by rleishman
Wed Jan 17, 2007 4:04 pm
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: Regarding FTP STAGE
Replies: 15
Views: 3890

If you are trying to PULL the file from your PC from the DS server, you will need an FTP listener running on your PC. This is not something you would need if you were PUSHing (FTP from the DOS prompt). Confirm that you are actually able to PULL a file by running FTP from the shell prompt on your DS ...
by rleishman
Wed Jan 17, 2007 5:34 am
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: How to assign value from table to environment variable?
Replies: 7
Views: 1939

I'm not a Premium Member either, so I can safely avoid plagiarism (at least directly - I'm sure it was Ray from whom I originally learned the same trick in the days before Premium Membership). Create a Job Sequence with two linked jobs: Job1 and Job2. Job1 has 3 stages: an OCI source, a Transformer,...
by rleishman
Wed Jan 17, 2007 5:19 am
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: Setting Job Variables in a stage and accessing it in another
Replies: 10
Views: 3716

oh! then there must be someway out, I mean if I have more than one such variables then I can not use DSSetUserStatus. Yes, and it was given to you above. You declare persistent variables in a routine using COMMON. You set the variables with one routine call from the first transformer, and fetch it ...
by rleishman
Wed Jan 17, 2007 1:06 am
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: Regarding FTP STAGE
Replies: 15
Views: 3890

My FTP stage runs on Unix, reads from Unix, and writes to a Sequential file. I used it because I had just one file to get, and was too lazy to code a shell script with all of the appropriate error handling and logging that I get for free with DS. General Tab Server Name: <Same as I use in command li...
by rleishman
Tue Jan 16, 2007 8:18 pm
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: Slow Loading Performance - Oracle 10g
Replies: 21
Views: 6841

I had EXACTLY the same problem when we first started building DS jobs. Getting <100rows/sec. We now get ~5000/sec - We did the following: - Use Bulk Loader, not OCI. - Ensure FKs disabled, triggers disabled, and PK/UK that are enforced by non-unique indexes are disabled. This allows the use of Direc...
by rleishman
Tue Jan 16, 2007 5:36 am
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: Performance problem using lookup
Replies: 23
Views: 5261

So you change something in the OCI and that dramatically alters the characteristic of the problem.

Suggest you return to my post above and run TK*Prof. Looks like Oracle is involved.
by rleishman
Mon Jan 15, 2007 5:36 am
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: Performance problem using lookup
Replies: 23
Views: 5261

It sounds as though you believe it is not a problem with Oracle, because the same job without the lookup is fast - fair enough. Prove it to yourself (and to us) by tracing the session and checking the elapsed time in TK*Prof. Add the following to the Before-SQL: alter session set sql_trace = true No...
by rleishman
Mon Jan 15, 2007 5:23 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Updating Sybase IQ database
Replies: 6
Views: 4142

I havent experimented enough with IQ to tell you for sure. I do know that updates and deletes are abysmally slow. Inserts are fine - bulk loads are better. We got ours running in an acceptable time frame, but the volumes were not crippling. In Oracle, it is generally faster to truncate (not delete!)...
by rleishman
Mon Jan 15, 2007 12:56 am
Forum: IBM<sup>®</sup> DataStage Enterprise Edition (Formerly Parallel Extender/PX)
Topic: Updating Sybase IQ database
Replies: 6
Views: 4142

I've used Sybase IQ, but with Informatica, not DataStage. The column-based architecture should not be a problem, it should be transparent to SQL. Just issue the same SQL Update statements that you would on any other database. Now performance WILL be a problem if you have significant volumes; updates...
by rleishman
Mon Jan 15, 2007 12:49 am
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: Performance problem using lookup
Replies: 23
Views: 5261

Also check that you have "Treat Warning Message as Fatal Error" checked in the OCI stage. If you have Unique/PK violations, ignoring these errors adds a huge overhead.
by rleishman
Thu Jan 11, 2007 8:06 pm
Forum: IBM<sup>®</sup> Infosphere DataStage Server Edition
Topic: To Load data into different tables using bulk loaders
Replies: 4
Views: 1485

If you are locking yourself out (multiple loads to the same table) then read on: Oracle won't let you perform multiple direct-path / parallel loads into the same table UNLESS each load is into a different partition, and the partition name is specified in the load spec. I suspect this may also work w...