Deleting sequential files/dataset files

[Marley777]

How can I delete files created in the sequential file stage? How can i automate this process (deleting files). We are accumulating files and need to get rid of them and keep them from accumulating in the future. Obviously we don't want to delete what we need. I'm familiar with the
dataset manager, but this utility doesn't seem to be an automated process and is only used with operating system files (.ds suffix). I'm familiar with manually deleting the files in UNIX, but looking for a better way. Maybe there is a retention period of some kind that I can set.


Thanks for reading. Thank god for forums like this!!!!!

[ray.wurlod]

Most sites with which I've worked use a shell script started from cron. The script uses find to identify files (in particular directories) that haven't been touched for some time (7 days seems to be the most common) and removes them.

[richdhan]

Hi RStone,

We usually do this kind of work in the sequencer. After all the jobs for loading a dimension is completed we use the Routine activity to delete the intermediate files.

We use

orchadmin delete - to delete datasets
rm - to delete files created in sequential stages.

HTH
--Rich

Pride comes before a fall
Humility comes before honour

[mandyli]

Hi RStone,

Added rich stmt

We use

orchadmin delete - to delete datasets
rm - to delete files created in sequential stages.

Or

Use

After job_routine

-> ExecSh

->> rm /AD/final/AD.txt or /AD/final/AD.ds


Try above stmt.

Thanks
Man

[Marley777]

Hello to all that answered my original post. I think I have what I need for now. Thanks for your time and answers, it's greatly appreciated.



Thanks :D