Clearing a multiple input hash file

Post questions here relative to DataStage Server Edition for such areas as Server job design, DS Basic, Routines, Job Sequences, etc.

Moderators: chulett, rschirm, roy

Post Reply
dickfong
Participant
Posts: 68
Joined: Tue Apr 15, 2003 9:20 am

Clearing a multiple input hash file

Post by dickfong »

I have a simple job which has one input stream from a sequential file. A transformer which takes in the source and have mulitple lookup hash files attached to it. The logic is simple but it would output data to the same hash file when matching different lookups.

What I need to do is to clear the target hash file before writting any data into it. So shall I check any of the 'clear hash' checkbox in my target hash file to do so or do I need a seperate before-job routine or else?

Please advice. Thanks a lot.

DF
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Set your rule to "clear file before writing" on the Hashed File stage's Input link.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

Hmm... an airport lounge drive-by. :wink:

Technically, your answer is that you can check any one of the input links to the hash file to accomplish the task of clearing of it. When a job starts but before any rows start to flow through it, DataStage checks for conditions of that nature in the stages - truncate table, clear hash file, backup flat file, etc - and does it as those stages start. Or, to feel safe, you could mark all of the links.

As a caveat, I've seen times with the OCI stage where that didn't always happen properly. We fell back on a before-job routine that accomplished the truncate we needed and you can do the same thing with a hash file using CLEAR.FILE if the hash was created in the Project. Hash files explicity pathed (which were created by mkdbfile) can be 'cleared' at the O/S level by removing them (rm -rf) and the job will automatically recreated them, even if the Create option is not checked. Well, unless you are reading from the same hash before you are writing to it... :?

Whatever you end up doing, I'd strongly suggest a good bit o' Annotation be added to the job that states exactly what you are doing. Before/After Job routines or single link check boxes are not all that obvious to Those Who Come After and good annotations can help tremendously.
-craig

"You can never have too many knives" -- Logan Nine Fingers
tonystark622
Premium Member
Premium Member
Posts: 483
Joined: Thu Jun 12, 2003 4:47 pm
Location: St. Louis, Missouri USA

Post by tonystark622 »

I have seen jobs designed to read from the null device (/dev/null for Unix) and write to the hash file with Clear before writing checked. This is placed at the beginning of the job with other "dummy" activities to ensure that it happens before anything else.

Tony
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

Yes, I missed the "multiple inputs" part of your requirements. The problem now is that you do not know which link will fire first. If you do, check "clear file before writing on that link".
Otherwise, you need to use the CLEAR.FILE command (or its operating system equivalent for pathed hashed files) in a before-stage or before-job subroutine.
Don't overwrite the hashed file with the contents of /dev/null; doing this clears the control and sizing information too, which you definitely don't want!
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
tonystark622
Premium Member
Premium Member
Posts: 483
Joined: Thu Jun 12, 2003 4:47 pm
Location: St. Louis, Missouri USA

Post by tonystark622 »

Hi Ray!

I don't directly copy the contents of /dev/null to the hash file, I "read" it from a sequential file stage, through a transformer and output to the hash_file that needs to be initilialized. I have all the metadata defined in the transformer and hash file stages and the hash file is marked "Clear file before writing" and "Create File". This sequence of stages is either in a separate job, or at the beginning of a job so that it happens before anything else happens.

Tony
MounikaP
Premium Member
Premium Member
Posts: 6
Joined: Mon Nov 06, 2017 3:48 am

I could not see the full message

Post by MounikaP »

ray.wurlod wrote:Yes, I missed the "multiple inputs" part of your requirements. The problem now is that you do not know which link will fire first. If you do, check "clear file before writing on that link".
Otherwi ...
Even though i am the premium member, I could not see the full message. Could you please help me with the complete message posted.
Thanks and Regards,
Mounika P
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

If you just paid for a membership, you'll have to wait for that to be setup before you can see any premium content. If you look at the left-hand side of the forums where the names are displayed, you'll see that it hasn't been activated. Yet.
-craig

"You can never have too many knives" -- Logan Nine Fingers
MounikaP
Premium Member
Premium Member
Posts: 6
Joined: Mon Nov 06, 2017 3:48 am

Post by MounikaP »

Okay.. thanks... I shall wait for my status to get updated.
Thanks and Regards,
Mounika P
ray.wurlod
Participant
Posts: 54607
Joined: Wed Oct 23, 2002 10:52 pm
Location: Sydney, Australia
Contact:

Post by ray.wurlod »

tonystark622 wrote:I have seen jobs designed to read from the null device (/dev/null for Unix) and write to the hash file with Clear before writing checked.
FYI, under the covers the CLEAR.FILE command overwrites the hashed file from /dev/null, then writes a new header block.
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Post Reply