Performance issue in Server job

justicevb · Post by **justicevb** » Fri Jan 23, 2009 5:39 am

Hi,

here in my scenario we are facing the performance problem because of Universe tables are used as look up tables.

In my scenario only we are using Transformer, look up Hash files, look up tables and sequential files.

Now it was loading the data with 20 rows/sec

need to load 900000 records..

Could some one help us..

Thanks,
vv

basu.ds · Post by **basu.ds** » Fri Jan 23, 2009 6:01 am

Change array size and transaction size while extracting and loading also it will give better performance and create index on target tables

chulett · Post by **chulett** » Fri Jan 23, 2009 8:23 am

What makes you think the lookups are the problem? And are they really "Universe" lookups or are they Hashed File lookups?

ray.wurlod · Post by **ray.wurlod** » Fri Jan 23, 2009 2:54 pm

Any SQL-based lookup will be slower than hashed file lookup.

Greater speed can be had if the columns mentioned in the WHERE clause are indexed; without that you are potentially performing a full table scan for each row processed.