DSXchange

IMO this is better done at the extraction phase, not at load time. For delta-extraction, you can use one of the methods described above, or if extracting from an Oracle database you can look into Change Data Capture - it's a non-invasive way to capture all the changes on a table without timestamp co...

Try some of these suggestions: viewtopic.php?t=95526

A long time ago, I tried running the BuiltIn DSJobReport subroutine as the last activity of a Job Sequence, instead of from the After-Job subroutine of a server job, as is intended. From memory, it mostly worked, except the status showed as RUNNING, insteead of FINISHED. I chucked in the idea around...

Once again, I'll have a go at a free answer... If the second job is actually a Job Sequence, and the parameter you assign with $UserStatus in the second job is an Environment Variable (declared in the Enviroment section in Administrator), then every job subordinate to Job2 will automatically inherit...

If you are trying to PULL the file from your PC from the DS server, you will need an FTP listener running on your PC. This is not something you would need if you were PUSHing (FTP from the DOS prompt). Confirm that you are actually able to PULL a file by running FTP from the shell prompt on your DS ...

I'm not a Premium Member either, so I can safely avoid plagiarism (at least directly - I'm sure it was Ray from whom I originally learned the same trick in the days before Premium Membership). Create a Job Sequence with two linked jobs: Job1 and Job2. Job1 has 3 stages: an OCI source, a Transformer,...

oh! then there must be someway out, I mean if I have more than one such variables then I can not use DSSetUserStatus. Yes, and it was given to you above. You declare persistent variables in a routine using COMMON. You set the variables with one routine call from the first transformer, and fetch it ...

My FTP stage runs on Unix, reads from Unix, and writes to a Sequential file. I used it because I had just one file to get, and was too lazy to code a shell script with all of the appropriate error handling and logging that I get for free with DS. General Tab Server Name: <Same as I use in command li...

I had EXACTLY the same problem when we first started building DS jobs. Getting <100rows/sec. We now get ~5000/sec - We did the following: - Use Bulk Loader, not OCI. - Ensure FKs disabled, triggers disabled, and PK/UK that are enforced by non-unique indexes are disabled. This allows the use of Direc...

So you change something in the OCI and that dramatically alters the characteristic of the problem.

Suggest you return to my post above and run TK*Prof. Looks like Oracle is involved.

It sounds as though you believe it is not a problem with Oracle, because the same job without the lookup is fast - fair enough. Prove it to yourself (and to us) by tracing the session and checking the elapsed time in TK*Prof. Add the following to the Before-SQL: alter session set sql_trace = true No...

I havent experimented enough with IQ to tell you for sure. I do know that updates and deletes are abysmally slow. Inserts are fine - bulk loads are better. We got ours running in an acceptable time frame, but the volumes were not crippling. In Oracle, it is generally faster to truncate (not delete!)...

I've used Sybase IQ, but with Informatica, not DataStage. The column-based architecture should not be a problem, it should be transparent to SQL. Just issue the same SQL Update statements that you would on any other database. Now performance WILL be a problem if you have significant volumes; updates...

Also check that you have "Treat Warning Message as Fatal Error" checked in the OCI stage. If you have Unique/PK violations, ignoring these errors adds a huge overhead.

If you are locking yourself out (multiple loads to the same table) then read on: Oracle won't let you perform multiple direct-path / parallel loads into the same table UNLESS each load is into a different partition, and the partition name is specified in the load spec. I suspect this may also work w...