DSXchange: DataStage and IBM Websphere Data Integration Forum
View next topic
View previous topic
Add To Favorites
Author Message
admin
Site Admin



Joined: 12 Jan 2003
Posts: 8720

Points: 10

Post Posted: Thu Feb 26, 2004 10:39 am Reply with quote    Back to top    

I am constructing a DataStage "Change Data Capture" program that
compares the contents of today's daily sequential file with yesterday's
file in order to specify which records have been added, deleted or
changed.

The source file names are in the general format
"dailyfile_load_20040216_2042.dat", or
"dailyfile_load_yyyymmdd_hhmm.dat", and they will be in a directory
where each day's file will have a unique yyyymmdd_hhmm, as part of its
name.

I need to construct a DataStage routine that will capture today's and
yesterday's files and specify them as source file names.

My approach would be to construct a DataStage routine in Unix that
would capture today's date as yyyymmdd. Then I could copy today's file,
specifying the yyyymdd and using a wildcard for hhmm, to a new file with
a name in the format "dailyfile_load_20040216.dat". This new filename
could then become the DataStage sourcefile name using a DSSetParam in
the batch script.

Has anyone constructed such a routine? I am wondering if it would be
easier to find today's yyyymmdd value using BASIC in the batch script
and having it as an argument to the Unix routine, or would it be better
to find today's yyyymmdd value in Unix.

I would appreciate any and all suggestions.

Many thanks!
Kathy Armstrong
kathy.armstrong@us.fortis.com

****************************************************************
Please Note
The information in this E-mail message is legally privileged
and confidential information intended only for the use of the
individual(s) named above. If you, the reader of this message,
are not the intended recipient, you are hereby notified that
you should not further disseminate, distribute, or forward this
E-mail message. If you have received this E-mail in error,
please notify the sender. Thank you
*****************************************************************
_______________________________________________
datastage-users mailing list
datastage-users@oliver.com
http://www.oliver.com/mailman/listinfo/datastage-users

_________________
PLEASE READ
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
admin
Site Admin



Joined: 12 Jan 2003
Posts: 8720

Points: 10

Post Posted: Thu Feb 26, 2004 11:04 am Reply with quote    Back to top    

Kathy,

You may want to just create symbolic links to each of the two files at Unix.
Then your DataStage job can always refer to static names (substitute 'prev'
and 'curr' for the actual date) without having to set the job parameters
every time to match the dates.

Remember that you can use the CRC32 routine to generate a checksum which can
be used as the basis for your CDC comparison. There have been discussions
about CDC and the CRC32 routine on dsexchage.

Good luck.
Alan


_______________________________________________
datastage-users mailing list
datastage-users@oliver.com
http://www.oliver.com/mailman/listinfo/datastage-users

_________________
PLEASE READ
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
Rate this response:  
Not yet rated
admin
Site Admin



Joined: 12 Jan 2003
Posts: 8720

Points: 10

Post Posted: Thu Feb 26, 2004 2:09 pm Reply with quote    Back to top    

I'ts straightforward to do in DataStage BASIC or in a UNIX shell script.

Example in DataStage BASIC:

TheDate = Oconv(@DATE, "DYMD" : @VM : "MCN")
TheTime = Oconv(@TIME, "MTS:" : @VM : "MCN")
NewDir = OldFileName : "_" : TheDate : "_" : TheTime
NewFileName = OldFileName : "_" : TheDate : "_" : TheTime : ".dat"
LinkName = "FileToProcess"
Command = "mkdir" : NewDir : " && mv " : FileName : " " : NewDir : "/" : NewFileName
Call DSExecute("UNIX", Command, Output, ExitStatus)
If ExitStatus 0
Then
* code to handle failure of mkdir or mv command
End
Command = "rm -f " : LinkName : " && ln -s " : NewDir : "/" NewFileName : " " : LinkName
Call DSExecute("UNIX", Command, Output, ExitStatus)
If ExitStatus 0
Then
* code to handle failure of rm or ln command
End



_______________________________________________
datastage-users mailing list
datastage-users@oliver.com
http://www.oliver.com/mailman/listinfo/datastage-users

_________________
PLEASE READ
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
Rate this response:  
Not yet rated
Display posts from previous:       

Add To Favorites
View next topic
View previous topic
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum



Powered by phpBB © 2001, 2002 phpBB Group
Theme & Graphics by Daz :: Portal by Smartor
All times are GMT - 6 Hours