Cleaning out Director log entries

[aartlett]

We are running our jobs with job instance that is an increasing number (EG: loadhash.1234, loadhash.1235 etc). The properties of the project is set to clear the log after 5 days, but the director has entries in it going back to day dot.

Apart from deleting via director, is there anyway to empty these entries that are > 5 days from the list? As there are thousands of entries it takes a while for the refresh.

I'm hoping for a simple list of files or Ubasic command or something :D

Hope someone out there has encountered this problem and come up wioth a solution.

Andrew

[ray.wurlod]

The best way is to enable auto-purge for the base job, with "jobs over 4 days old" as the setting.

You can do it with SQL, but then you may have to adjust some of the control record settings.

[trokosz]

Using the DataStage Administrator, select the project and push the COMMAND button to bring up the command window. After that, enter the following command to find the job number. Note, the job name is case sensitive.

LIST DS_JOBS WITH NAME="yourJobName"

Look at the output and note the job number. now issue the following
command to clear the log, where xxxx is the job number you just found.

CLEAR.FILE RT_LOGxxxx

Or

Instead of doing CLEAR.FILE, you can just drop the data portion of the file and recreate it using the following two commands. they should run instantly.

DELETE DATA RT_LOGxxxx
CREATE.FILE DATA RT_LOGxxxx

Hope it helps

[aartlett]

Ok, I've obviously not communicated properly.

The job LOGS are cleared out fine after 5 days. The job entries in director are building up and up. those over 5 days ago are empty. The base job (xxx) shows 5 days of log. The logs xxx.111, xxx.112 etc for 5 days show data. More than 5 days ago they are empty. So what we have are thousands of empty entries in director, slowing down the time to load the screen or refresh the screen.

What I need is a way (without recompiling the job which clears all entries) to delete the entries over 5 days old.

Any hwlp is appreciated.

Andrew

[xcb]

Not that this answers the OPs question but you should note that it isn't a recommended practice to simply clear your log file. If you search the forum you will find numerous posts by the gurus that frequent this site explaining against this as it also destroys the control records hidden within a log file.

LIST DS_JOBS WITH NAME="yourJobName"

Look at the output and note the job number. now issue the following
command to clear the log, where xxxx is the job number you just found.

CLEAR.FILE RT_LOGxxxx

Or

DELETE DATA RT_LOGxxxx
CREATE.FILE DATA RT_LOGxxxx

[aartlett]

Cameron,
I tries the clear logs in a dev environment that was soon destined for destruction (rather than assimilation) <<exterminate>>. It doesn't solve the issue of removing the (now empty) log entry in director. Yep it clears the log part but not the director entry.

Oh well ... if things aren't resolved by the new year when I'm back in full swing then I'll have to take the drastic step and contact (*gasp) Ascential.

This on top of planning to move to 7.5.

Andrew

[chulett]

You might want to read through this earlier thread where someone else had a similar dilemma. They came to the conclusion (after working with Ascential Support) that it simply wasn't possible to do what they wanted. I'm afraid you may be in the same boat.

[trokosz]

Hey, I just tied to help concerning overriding and clearing a large Log file which I thought was asked....and I actually do not agree that this is a bad practice....but its an idea ....best of luck

[chulett]

Cameron was just trying to point out the fact that if you do use CLEAR.FILE to empty a log file, you'll lose the control infomation that is there as well - notably any auto-purge settings you had for the job.

Not everyone realizes that and it's good to know if you go down that path.

[mak]

Craig,

I am also having an issue with these log files.When we created our jobs to test we didn't had AutoPurge option enabled.so log files are building.

Today we got FileSpace issue on out test box.i could not start our dspcr.
We have one project running with these same options on Production and i am worrying about this file space.

You are saying that CLEAR.FILE will remove control information from log file.How do we do CLEAR LOG FILES with out affecting this control information and getting this auto purge enabled on all these old jobs.

[chulett]

Kenneth Bland has posted some code in this thread that will run through a Project and set your Auto Purge options for you. Then your logs will start clearing themselves on a regular basis.

I've used it, it works great and has come in very handy.

[mak]

Thanks for you reply Craig,
Does that work for DS 7.1?

[chulett]

Yes... should be very version generic.

[mak]

Craig,

i got couple of questions.please correct me if i am getting anything wrong here.

We got PURGE options in DSParam file at project level and job level.
And using the routine you mentioned we are setting these purge options at job level. how different are they?

Suppose if have purge interval set to 4 at job level and Purgesetting/prevruns set to 5 at project level. how is it going decide?

Thanks for your reply.

[chulett]

What you set at the Project level only gets applied to new jobs when they are created. It becomes their default setting and changes at the project level have no effect on existing jobs. That's why you need a routine like this to force changes across existing jobs... the alternative is doing them one by one by hand. Been there, done that, wouldn't want to do it again.