Newbie question: resource disk and scratch disk management

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
philds
Participant
Posts: 8
Joined: Wed Sep 01, 2004 11:09 am

Newbie question: resource disk and scratch disk management

Post by philds »

Hi,

I'm just starting to work with DataStage PX (as a sys admin, not as a developer) and I haven't really seen the product working so far.
I have to answer some questionsand though I went through different documentations, some points remain obscure.

When configuring the PX Engine, one has to define a Configurations/config.apt file to define the available nodes and disk resources.

The disk resource are either scratch disk or disk.
The resource disk is meant to hold persistent data (how persistent ? during the execution of the project ? After the execution of the jobs of the project ?).
The scratch disk is for temporary files created by the PX engine (and so, unknown from the developper ?)

My concern is how to manage these resource directory in term of space, purge and backup.

I would think that the scratch disk probably doesn t need backup.
Does it need to be purged from time to time or does PX do this after the job completion ?

What about the resource disk directory ? It is called "Datasets" by default.
Does it handle only "Datasets" stage defined by the developpers in DS Designer or is there other kind of files stored there ?
Who is responsible for cleaning the files held in this directory ?
Does the developer have to code the deletion of these files ?
Are those file erased with a new content everytime the job is rerun ?

The above questions should help answering this following one:
does this resource disk directory need to be backuped ? How often ?

Concerning the size of these directory, how can it be determined ?
By the developer only or is there some PX Engine "overhead" ?


Thank your help !

Phil.
lgharis
Premium Member
Premium Member
Posts: 56
Joined: Wed May 26, 2004 10:08 am
Location: Dallas, TX

Post by lgharis »

Did you ever get an answer to your questions? I am looking at the Scratch and Datasets directories and wondering if it is possible to clean them up. I do not want to delete files and cause a corruption but there are files in the Datasets directory from 2002.
Leroy Gharis

Dallas, TX
pavankvk
Participant
Posts: 202
Joined: Thu Dec 04, 2003 7:54 am

Post by pavankvk »

u can use the orchadmin utility to manage those datasets.
the command takes options dump,delete,copy etc..so u do what ever u want to do..

lgharis wrote:Did you ever get an answer to your questions? I am looking at the Scratch and Datasets directories and wondering if it is possible to clean them up. I do not want to delete files and cause a corruption but there are files in the Datasets directory from 2002.
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Why does that text look vaguely familiar? Could it be because I've read the manual? :roll:

Please don't copy large slabs of the manuals into these fora; it wastes disk space. Everyone who has DataStage has the manuals. All you needed to do was to refer to the correct place in the manual.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
trokosz
Premium Member
Premium Member
Posts: 188
Joined: Thu Sep 16, 2004 6:38 pm
Contact:

Post by trokosz »

OK, read the manual then vs. asking the question
Post Reply