DSXchange

I have a simple job which has one input stream from a sequential file. A transformer which takes in the source and have mulitple lookup hash files attached to it. The logic is simple but it would output data to the same hash file when matching different lookups.

What I need to do is to clear the target hash file before writting any data into it. So shall I check any of the 'clear hash' checkbox in my target hash file to do so or do I need a seperate before-job routine or else?

Please advice. Thanks a lot.

DF

Set your rule to "clear file before writing" on the Hashed File stage's Input link.

Hmm... an airport lounge drive-by.

Technically, your answer is that you can check any one of the input links to the hash file to accomplish the task of clearing of it. When a job starts but before any rows start to flow through it, DataStage checks for conditions of that nature in the stages - truncate table, clear hash file, backup flat file, etc - and does it as those stages start. Or, to feel safe, you could mark all of the links.

As a caveat, I've seen times with the OCI stage where that didn't always happen properly. We fell back on a before-job routine that accomplished the truncate we needed and you can do the same thing with a hash file using CLEAR.FILE if the hash was created in the Project. Hash files explicity pathed (which were created by mkdbfile) can be 'cleared' at the O/S level by removing them (rm -rf) and the job will automatically recreated them, even if the Create option is not checked. Well, unless you are reading from the same hash before you are writing to it...

Whatever you end up doing, I'd strongly suggest a good bit o' Annotation be added to the job that states exactly what you are doing. Before/After Job routines or single link check boxes are not all that obvious to Those Who Come After and good annotations can help tremendously.

I have seen jobs designed to read from the null device (/dev/null for Unix) and write to the hash file with Clear before writing checked. This is placed at the beginning of the job with other "dummy" activities to ensure that it happens before anything else.

Tony

Yes, I missed the "multiple inputs" part of your requirements. The problem now is that you do not know which link will fire first. If you do, check "clear file before writing on that link".
Otherwise, you need to use the CLEAR.FILE command (or its operating system equivalent for pathed hashed files) in a before-stage or before-job subroutine.
Don't overwrite the hashed file with the contents of /dev/null; doing this clears the control and sizing information too, which you definitely don't want!

Hi Ray!

I don't directly copy the contents of /dev/null to the hash file, I "read" it from a sequential file stage, through a transformer and output to the hash_file that needs to be initilialized. I have all the metadata defined in the transformer and hash file stages and the hash file is marked "Clear file before writing" and "Create File". This sequence of stages is either in a separate job, or at the beginning of a job so that it happens before anything else happens.

Tony

ray.wurlod wrote:Yes, I missed the "multiple inputs" part of your requirements. The problem now is that you do not know which link will fire first. If you do, check "clear file before writing on that link".
Otherwi ...

Even though i am the premium member, I could not see the full message. Could you please help me with the complete message posted.

If you just paid for a membership, you'll have to wait for that to be setup before you can see any premium content. If you look at the left-hand side of the forums where the names are displayed, you'll see that it hasn't been activated. Yet.

Okay.. thanks... I shall wait for my status to get updated.

tonystark622 wrote:I have seen jobs designed to read from the null device (/dev/null for Unix) and write to the hash file with Clear before writing checked.

FYI, under the covers the CLEAR.FILE command overwrites the hashed file from /dev/null, then writes a new header block.