Datastage Job stalled

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
Nagac
Premium Member
Premium Member
Posts: 127
Joined: Tue Mar 29, 2011 11:39 am
Location: India

Datastage Job stalled

Post by Nagac »

Hi

I have Datastage Job, Which reads the data from sql server and writes into Dataset. I am using ODBC Connector which uses the DSN(IBM native SQL Server Protocol)

Job works fine upto some time and after that it runs but i can't see any change in processed rows in DS Monitor even after several hours.

table which is reading is having nvarchar(max) and it has huge data in it. To find the problem i had updated this column with 0(all rows) and executed the DS Jobs, It was success.

Can someone please advise if there are any ENV variables are required to process large content in a column?

Thanks
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

How long does the source SQL with the max take to complete (not just start to return rows) outside of DataStage? That or you can add a transformer if you don't have one with an @FALSE constraint to the target dataset so no writes occur.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Nagac
Premium Member
Premium Member
Posts: 127
Joined: Tue Mar 29, 2011 11:39 am
Location: India

Post by Nagac »

It gives out of memory error. I have tested with SSIS it worked well with exporting to csv file
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Nagac wrote:It gives out of memory error.
Which does? And where?
-craig

"You can never have too many knives" -- Logan Nine Fingers
Nagac
Premium Member
Premium Member
Posts: 127
Joined: Tue Mar 29, 2011 11:39 am
Location: India

Post by Nagac »

Sorry, query(select * from table) has given memory error when i ran on SSMS first time,i ran the query again and it has given the results in SSMS in 12mins.
moalik
Participant
Posts: 39
Joined: Thu Sep 15, 2011 8:15 am
Location: Melbourne

Post by moalik »

Are you doing a select on the Table or on the View? Are you doing any joins or exists conditions with in the query? If so try, to achieve the joining capability in datastage rather than on SQL. Try increasing the Max Heap Size on the database.
Mohsin Khan
Datastage Consultant
Nagac
Premium Member
Premium Member
Posts: 127
Joined: Tue Mar 29, 2011 11:39 am
Location: India

Post by Nagac »

I am doing select on table and there are no joins as i am extracting from single table.

Good thing is that it is working after changing the Isolation level to Read Uncommitted instead of Read commited. I am not sure how it works. But it is doing my work.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Isolation levels are about the locks that are (or are not) taken during the session.
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply