Guillermo writes:
>
Hi Guillermo,
If all three servers (including the DataStage server) are Windows based, such as NT or Windows2000, then you can move or copy files using the DSExecute command and by specifying the UNC path (i.e. \\Path) or with a DataStage job (if you want to read and write the files). But you will need to do a few things with Permissions:
1. In the DataStage Manager, on the Schedule Tab for your project, add a userid (let's say domain\DatastageUID).
2. Grant the DataStageUID and your userid (or group that you are in) permissions to the folder on the two servers you are moving from and to.
Explanation: When you schedule a job, DataStage will use the permissions of the userid specified on the permissions tab (i.e DataStageUID). When you run a job without the scheduler, then permissions from your userid will be used.
Below is some batch code that may be of help to you.
Scott
+++
* Make a double quote version of the parameter file path\Names so that these
* double quoted variables can be used in later commands, so that these
* commands will not fail if there are spaces in the path or name.
DQInputFile = DQuote("\\SomePath\SomeFileIn.txt")
DQOutputFile = DQuote("\\SomePath\SomeFileOut.txt")
* Move the file. Do this by doing the following with each file:
* 1. Use the Move command to move the file.
* 2. Verify the original file name is gone.
* 3. Verify the new file name exists in the move to directory.
* Do the move to the new directory and name
DSECommand = "Move " : DQInputFile : " " : DQOutputFile
Call DSExecute("NT",DSECommand,DSEOutput,DSERC)
If DisplayExtra = "YES" Then
OutMessage = 'DisplayExtra ==> DSEOutput from Move command = ' : DSEOutput
Call DSTransformError(OutMessage, "MoveFile")
End
* Verify the old name is gone
DSECommand = "DIR " : DQInputFile : " /B"
Call DSExecute("NT",DSECommand,DSEOutput,DSERC)
If DSEOutput[1,14] "File Not Found" Then
OutMessage = 'FAILURE to move file ' : DQInputFile
OutMessage = OutMessage : ' to ' : DQOutputFile
OutMessage = OutMessage : ' because the original name still exists after the move.'
Call DSTransformError(OutMessage, "MoveFile")
Return(Ans)
End
* Verify the new name now exists.
DSECommand = "DIR " : DQOutputFile : " /B"
Call DSExecute("NT",DSECommand,DSEOutput,DSERC)
If DSEOutput[1,14] = "File Not Found" Then
OutMessage = 'FAILURE to move file ' : DQInputFile
OutMessage = OutMessage : ' to ' : DQOutputFile
OutMessage = OutMessage : ' because the new name does not exist after the move.'
Call DSTransformError(OutMessage, "MoveFile")
Return(Ans)
End
RES: Copy (or transfer) of files between machines
RES: Copy (or transfer) of files between machines
<b>PLEASE READ</b>
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
RES: Copy (or transfer) of files between machines
Scott:
Thank you for your explanation.
The problem is that I don't know the names of the files. I know that all of
them are are like "*.OK". But I don't know the number and the name.
Guillermo P. Barsky - gbarsky@osde.com.ar
Gerencia de Sistemas - Desarrollo
OSDE Binario - Filial Metropolitana
Alem 1067, Piso 16
TE (5411)4510-4330, Fax (5411)4510-5480
http://www.osde.com.ar
"Scott Brooks"
Para:
cc:
12/01/2004 03:31 Asunto: Re: RES: Copy (or transfer) of files between machines
p.m.
Guillermo writes:
>
Hi Guillermo,
If all three servers (including the DataStage server) are Windows based,
such as NT or Windows2000, then you can move or copy files using the
DSExecute command and by specifying the UNC path (i.e. \\Path) or with a
DataStage job (if you want to read and write the files). But you will need
to do a few things with Permissions:
1. In the DataStage Manager, on the Schedule Tab for your project, add a
userid (let's say domain\DatastageUID).
2. Grant the DataStageUID and your userid (or group that you are in)
permissions to the folder on the two servers you are moving from and to.
Explanation: When you schedule a job, DataStage will use the permissions
of the userid specified on the permissions tab (i.e DataStageUID). When
you run a job without the scheduler, then permissions from your userid will
be used.
Below is some batch code that may be of help to you.
Scott
+++
* Make a double quote version of the parameter file path\Names so that
these
* double quoted variables can be used in later commands, so that these
* commands will not fail if there are spaces in the path or name.
DQInputFile = DQuote("\\SomePath\SomeFileIn.txt")
DQOutputFile = DQuote("\\SomePath\SomeFileOut.txt")
* Move the file. Do this by doing the following with each file:
* 1. Use the Move command to move the file.
* 2. Verify the original file name is gone.
* 3. Verify the new file name exists in the move to directory.
* Do the move to the new directory and name
DSECommand = "Move " : DQInputFile : " " : DQOutputFile
Call DSExecute("NT",DSECommand,DSEOutput,DSERC)
If DisplayExtra = "YES" Then
OutMessage = 'DisplayExtra ==> DSEOutput from Move command = ' :
DSEOutput
Call DSTransformError(OutMessage, "MoveFile")
End
* Verify the old name is gone
DSECommand = "DIR " : DQInputFile : " /B"
Call DSExecute("NT",DSECommand,DSEOutput,DSERC)
If DSEOutput[1,14] "File Not Found" Then
OutMessage = 'FAILURE to move file ' : DQInputFile
OutMessage = OutMessage : ' to ' : DQOutputFile
OutMessage = OutMessage : ' because the original name still exists
after the move.'
Call DSTransformError(OutMessage, "MoveFile")
Return(Ans)
End
* Verify the new name now exists.
DSECommand = "DIR " : DQOutputFile : " /B"
Call DSExecute("NT",DSECommand,DSEOutput,DSERC)
If DSEOutput[1,14] = "File Not Found" Then
OutMessage = 'FAILURE to move file ' : DQInputFile
OutMessage = OutMessage : ' to ' : DQOutputFile
OutMessage = OutMessage : ' because the new name does not exist after
the move.'
Call DSTransformError(OutMessage, "MoveFile")
Return(Ans)
End
Thank you for your explanation.
The problem is that I don't know the names of the files. I know that all of
them are are like "*.OK". But I don't know the number and the name.
Guillermo P. Barsky - gbarsky@osde.com.ar
Gerencia de Sistemas - Desarrollo
OSDE Binario - Filial Metropolitana
Alem 1067, Piso 16
TE (5411)4510-4330, Fax (5411)4510-5480
http://www.osde.com.ar
"Scott Brooks"
Para:
cc:
12/01/2004 03:31 Asunto: Re: RES: Copy (or transfer) of files between machines
p.m.
Guillermo writes:
>
Hi Guillermo,
If all three servers (including the DataStage server) are Windows based,
such as NT or Windows2000, then you can move or copy files using the
DSExecute command and by specifying the UNC path (i.e. \\Path) or with a
DataStage job (if you want to read and write the files). But you will need
to do a few things with Permissions:
1. In the DataStage Manager, on the Schedule Tab for your project, add a
userid (let's say domain\DatastageUID).
2. Grant the DataStageUID and your userid (or group that you are in)
permissions to the folder on the two servers you are moving from and to.
Explanation: When you schedule a job, DataStage will use the permissions
of the userid specified on the permissions tab (i.e DataStageUID). When
you run a job without the scheduler, then permissions from your userid will
be used.
Below is some batch code that may be of help to you.
Scott
+++
* Make a double quote version of the parameter file path\Names so that
these
* double quoted variables can be used in later commands, so that these
* commands will not fail if there are spaces in the path or name.
DQInputFile = DQuote("\\SomePath\SomeFileIn.txt")
DQOutputFile = DQuote("\\SomePath\SomeFileOut.txt")
* Move the file. Do this by doing the following with each file:
* 1. Use the Move command to move the file.
* 2. Verify the original file name is gone.
* 3. Verify the new file name exists in the move to directory.
* Do the move to the new directory and name
DSECommand = "Move " : DQInputFile : " " : DQOutputFile
Call DSExecute("NT",DSECommand,DSEOutput,DSERC)
If DisplayExtra = "YES" Then
OutMessage = 'DisplayExtra ==> DSEOutput from Move command = ' :
DSEOutput
Call DSTransformError(OutMessage, "MoveFile")
End
* Verify the old name is gone
DSECommand = "DIR " : DQInputFile : " /B"
Call DSExecute("NT",DSECommand,DSEOutput,DSERC)
If DSEOutput[1,14] "File Not Found" Then
OutMessage = 'FAILURE to move file ' : DQInputFile
OutMessage = OutMessage : ' to ' : DQOutputFile
OutMessage = OutMessage : ' because the original name still exists
after the move.'
Call DSTransformError(OutMessage, "MoveFile")
Return(Ans)
End
* Verify the new name now exists.
DSECommand = "DIR " : DQOutputFile : " /B"
Call DSExecute("NT",DSECommand,DSEOutput,DSERC)
If DSEOutput[1,14] = "File Not Found" Then
OutMessage = 'FAILURE to move file ' : DQInputFile
OutMessage = OutMessage : ' to ' : DQOutputFile
OutMessage = OutMessage : ' because the new name does not exist after
the move.'
Call DSTransformError(OutMessage, "MoveFile")
Return(Ans)
End
<b>PLEASE READ</b>
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
RES: Copy (or transfer) of files between machines
>
Hi Guillermo,
What we did when we were faced with your situation was to write a custom routine that has as parameters:
InputFileDir = The input file path. Ex. \\Server\Directory
InputFileNameMask = The input file mask. Ex. TheFile*.txt
OldestOrNewest = Which file to look for. Valid values: Oldest or Newest.
CheckIfFileComplete = See if the file is done being created. Values Yes or No.
Then in the custom routine code we used a DSExcecute to execute a DIR command with an /OD option and redirected the output to a work file. Then we read through the work file and determined the newest or oldest file. Lastly, we passed the name back to DataStage where we can either move, copy, or use it in a job with the pound signs (#Name#).
Below is the code for the DSExecute of the DIR.
I hope this helps.
Scott
PS You could actually do all the above coding in a batch job, but by making it a custom routine, we are able to then call it from many jobs.
* Get a list of input files with their date time information. By NOT using
* the /B brief option, date time information will be returned. There will
* also be volume serial information. So, if we look for a "/" in position
* three and an "a" (AM) or a "p" (PM) in position 16, then we know it is
* from the date-time stamp of the info we want. We could just look for the
* "/" only, but this would give us directories as well. Looking for "a" or
* "b" in position 13, just gives us the files only.
* All other lines we can ignore.
* Each record returned is in the format of:
* mm/dd/yyyy hh:mma FileSize FileName
* Example:
* 06/07/2002 02:27p 104 Bart.txt
* 1234567890123456789012345678901234567890123456789 " : WorkFile : " /OD"
Call DSExecute("NT",DSECommand,DSEOutput,DSERC)
If DSERC 0 Then
OutMessage = 'FAILURE to execute the following command: ' : DSECommand
Call DSTransformError(OutMessage, "CustomRoutine - LocateFileViaDir")
Return(Ans)
End
* Open the work file
Hi Guillermo,
What we did when we were faced with your situation was to write a custom routine that has as parameters:
InputFileDir = The input file path. Ex. \\Server\Directory
InputFileNameMask = The input file mask. Ex. TheFile*.txt
OldestOrNewest = Which file to look for. Valid values: Oldest or Newest.
CheckIfFileComplete = See if the file is done being created. Values Yes or No.
Then in the custom routine code we used a DSExcecute to execute a DIR command with an /OD option and redirected the output to a work file. Then we read through the work file and determined the newest or oldest file. Lastly, we passed the name back to DataStage where we can either move, copy, or use it in a job with the pound signs (#Name#).
Below is the code for the DSExecute of the DIR.
I hope this helps.
Scott
PS You could actually do all the above coding in a batch job, but by making it a custom routine, we are able to then call it from many jobs.
* Get a list of input files with their date time information. By NOT using
* the /B brief option, date time information will be returned. There will
* also be volume serial information. So, if we look for a "/" in position
* three and an "a" (AM) or a "p" (PM) in position 16, then we know it is
* from the date-time stamp of the info we want. We could just look for the
* "/" only, but this would give us directories as well. Looking for "a" or
* "b" in position 13, just gives us the files only.
* All other lines we can ignore.
* Each record returned is in the format of:
* mm/dd/yyyy hh:mma FileSize FileName
* Example:
* 06/07/2002 02:27p 104 Bart.txt
* 1234567890123456789012345678901234567890123456789 " : WorkFile : " /OD"
Call DSExecute("NT",DSECommand,DSEOutput,DSERC)
If DSERC 0 Then
OutMessage = 'FAILURE to execute the following command: ' : DSECommand
Call DSTransformError(OutMessage, "CustomRoutine - LocateFileViaDir")
Return(Ans)
End
* Open the work file
<b>PLEASE READ</b>
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
RES: Copy (or transfer) of files between machines
Scott:
Thank you for your answer. I missed to say that the source and target
machines are Unix systems. So, I guess that the "copy ...." will not work.
Do you have an idea of how to hanlde this ?
Guillermo P. Barsky - gbarsky@osde.com.ar
Gerencia de Sistemas - Desarrollo
OSDE Binario - Filial Metropolitana
Alem 1067, Piso 16
TE (5411)4510-4330, Fax (5411)4510-5480
http://www.osde.com.ar
"Scott Brooks"
Para:
cc:
13/01/2004 12:47 Asunto: Re: RES: Copy (or transfer) of files between machines
p.m.
>
Hi Guillermo,
What we did when we were faced with your situation was to write a custom
routine that has as parameters:
InputFileDir = The input file path. Ex. \\Server\Directory
InputFileNameMask = The input file mask. Ex. TheFile*.txt
OldestOrNewest = Which file to look for. Valid values: Oldest or
Newest.
CheckIfFileComplete = See if the file is done being created. Values Yes or
No.
Then in the custom routine code we used a DSExcecute to execute a DIR
command with an /OD option and redirected the output to a work file. Then
we read through the work file and determined the newest or oldest file.
Lastly, we passed the name back to DataStage where we can either move,
copy, or use it in a job with the pound signs (#Name#).
Below is the code for the DSExecute of the DIR.
I hope this helps.
Scott
PS You could actually do all the above coding in a batch job, but by
making it a custom routine, we are able to then call it from many jobs.
* Get a list of input files with their date time information. By NOT using
* the /B brief option, date time information will be returned. There will
* also be volume serial information. So, if we look for a "/" in position
* three and an "a" (AM) or a "p" (PM) in position 16, then we know it is
* from the date-time stamp of the info we want. We could just look for the
* "/" only, but this would give us directories as well. Looking for "a" or
* "b" in position 13, just gives us the files only.
* All other lines we can ignore.
* Each record returned is in the format of:
* mm/dd/yyyy hh:mma FileSize FileName
* Example:
* 06/07/2002 02:27p 104 Bart.txt
* 1234567890123456789012345678901234567890123456789 " : WorkFile : " /OD"
Call DSExecute("NT",DSECommand,DSEOutput,DSERC)
If DSERC 0 Then
OutMessage = 'FAILURE to execute the following command: ' : DSECommand
Call DSTransformError(OutMessage, "CustomRoutine - LocateFileViaDir")
Return(Ans)
End
* Open the work file
Thank you for your answer. I missed to say that the source and target
machines are Unix systems. So, I guess that the "copy ...." will not work.
Do you have an idea of how to hanlde this ?
Guillermo P. Barsky - gbarsky@osde.com.ar
Gerencia de Sistemas - Desarrollo
OSDE Binario - Filial Metropolitana
Alem 1067, Piso 16
TE (5411)4510-4330, Fax (5411)4510-5480
http://www.osde.com.ar
"Scott Brooks"
Para:
cc:
13/01/2004 12:47 Asunto: Re: RES: Copy (or transfer) of files between machines
p.m.
>
Hi Guillermo,
What we did when we were faced with your situation was to write a custom
routine that has as parameters:
InputFileDir = The input file path. Ex. \\Server\Directory
InputFileNameMask = The input file mask. Ex. TheFile*.txt
OldestOrNewest = Which file to look for. Valid values: Oldest or
Newest.
CheckIfFileComplete = See if the file is done being created. Values Yes or
No.
Then in the custom routine code we used a DSExcecute to execute a DIR
command with an /OD option and redirected the output to a work file. Then
we read through the work file and determined the newest or oldest file.
Lastly, we passed the name back to DataStage where we can either move,
copy, or use it in a job with the pound signs (#Name#).
Below is the code for the DSExecute of the DIR.
I hope this helps.
Scott
PS You could actually do all the above coding in a batch job, but by
making it a custom routine, we are able to then call it from many jobs.
* Get a list of input files with their date time information. By NOT using
* the /B brief option, date time information will be returned. There will
* also be volume serial information. So, if we look for a "/" in position
* three and an "a" (AM) or a "p" (PM) in position 16, then we know it is
* from the date-time stamp of the info we want. We could just look for the
* "/" only, but this would give us directories as well. Looking for "a" or
* "b" in position 13, just gives us the files only.
* All other lines we can ignore.
* Each record returned is in the format of:
* mm/dd/yyyy hh:mma FileSize FileName
* Example:
* 06/07/2002 02:27p 104 Bart.txt
* 1234567890123456789012345678901234567890123456789 " : WorkFile : " /OD"
Call DSExecute("NT",DSECommand,DSEOutput,DSERC)
If DSERC 0 Then
OutMessage = 'FAILURE to execute the following command: ' : DSECommand
Call DSTransformError(OutMessage, "CustomRoutine - LocateFileViaDir")
Return(Ans)
End
* Open the work file
<b>PLEASE READ</b>
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
RES: Copy (or transfer) of files between machines
Hi Guillermo,
if you mount the filesystems on unix-machines to an other unix-machine a
copy will work.
Wolfgang
-----Original Message-----
From: gbarsky@osde.com.ar [mailto:gbarsky@osde.com.ar]
Sent: Thursday, January 15, 2004 11:34 PM
To: datastage-users@oliver.com
Subject: Re: RES: Copy (or transfer) of files between machines
Scott:
Thank you for your answer. I missed to say that the source and target
machines are Unix systems. So, I guess that the "copy ...." will not work.
Do you have an idea of how to hanlde this ?
Guillermo P. Barsky - gbarsky@osde.com.ar
Gerencia de Sistemas - Desarrollo
OSDE Binario - Filial Metropolitana
Alem 1067, Piso 16
TE (5411)4510-4330, Fax (5411)4510-5480
http://www.osde.com.ar
"Scott Brooks"
Para:
cc:
13/01/2004 12:47 Asunto: Re: RES: Copy (or
transfer) of files between machines
p.m.
>
Hi Guillermo,
What we did when we were faced with your situation was to write a custom
routine that has as parameters:
InputFileDir = The input file path. Ex. \\Server\Directory
InputFileNameMask = The input file mask. Ex. TheFile*.txt
OldestOrNewest = Which file to look for. Valid values: Oldest or
Newest.
CheckIfFileComplete = See if the file is done being created. Values Yes or
No.
Then in the custom routine code we used a DSExcecute to execute a DIR
command with an /OD option and redirected the output to a work file. Then
we read through the work file and determined the newest or oldest file.
Lastly, we passed the name back to DataStage where we can either move,
copy, or use it in a job with the pound signs (#Name#).
Below is the code for the DSExecute of the DIR.
I hope this helps.
Scott
PS You could actually do all the above coding in a batch job, but by
making it a custom routine, we are able to then call it from many jobs.
* Get a list of input files with their date time information. By NOT using
* the /B brief option, date time information will be returned. There will
* also be volume serial information. So, if we look for a "/" in position
* three and an "a" (AM) or a "p" (PM) in position 16, then we know it is
* from the date-time stamp of the info we want. We could just look for the
* "/" only, but this would give us directories as well. Looking for "a" or
* "b" in position 13, just gives us the files only.
* All other lines we can ignore.
* Each record returned is in the format of:
* mm/dd/yyyy hh:mma FileSize FileName
* Example:
* 06/07/2002 02:27p 104 Bart.txt
* 1234567890123456789012345678901234567890123456789 " : WorkFile : " /OD"
Call DSExecute("NT",DSECommand,DSEOutput,DSERC)
If DSERC 0 Then
OutMessage = 'FAILURE to execute the following command: ' : DSECommand
Call DSTransformError(OutMessage, "CustomRoutine - LocateFileViaDir")
Return(Ans)
End
* Open the work file
if you mount the filesystems on unix-machines to an other unix-machine a
copy will work.
Wolfgang
-----Original Message-----
From: gbarsky@osde.com.ar [mailto:gbarsky@osde.com.ar]
Sent: Thursday, January 15, 2004 11:34 PM
To: datastage-users@oliver.com
Subject: Re: RES: Copy (or transfer) of files between machines
Scott:
Thank you for your answer. I missed to say that the source and target
machines are Unix systems. So, I guess that the "copy ...." will not work.
Do you have an idea of how to hanlde this ?
Guillermo P. Barsky - gbarsky@osde.com.ar
Gerencia de Sistemas - Desarrollo
OSDE Binario - Filial Metropolitana
Alem 1067, Piso 16
TE (5411)4510-4330, Fax (5411)4510-5480
http://www.osde.com.ar
"Scott Brooks"
Para:
cc:
13/01/2004 12:47 Asunto: Re: RES: Copy (or
transfer) of files between machines
p.m.
>
Hi Guillermo,
What we did when we were faced with your situation was to write a custom
routine that has as parameters:
InputFileDir = The input file path. Ex. \\Server\Directory
InputFileNameMask = The input file mask. Ex. TheFile*.txt
OldestOrNewest = Which file to look for. Valid values: Oldest or
Newest.
CheckIfFileComplete = See if the file is done being created. Values Yes or
No.
Then in the custom routine code we used a DSExcecute to execute a DIR
command with an /OD option and redirected the output to a work file. Then
we read through the work file and determined the newest or oldest file.
Lastly, we passed the name back to DataStage where we can either move,
copy, or use it in a job with the pound signs (#Name#).
Below is the code for the DSExecute of the DIR.
I hope this helps.
Scott
PS You could actually do all the above coding in a batch job, but by
making it a custom routine, we are able to then call it from many jobs.
* Get a list of input files with their date time information. By NOT using
* the /B brief option, date time information will be returned. There will
* also be volume serial information. So, if we look for a "/" in position
* three and an "a" (AM) or a "p" (PM) in position 16, then we know it is
* from the date-time stamp of the info we want. We could just look for the
* "/" only, but this would give us directories as well. Looking for "a" or
* "b" in position 13, just gives us the files only.
* All other lines we can ignore.
* Each record returned is in the format of:
* mm/dd/yyyy hh:mma FileSize FileName
* Example:
* 06/07/2002 02:27p 104 Bart.txt
* 1234567890123456789012345678901234567890123456789 " : WorkFile : " /OD"
Call DSExecute("NT",DSECommand,DSEOutput,DSERC)
If DSERC 0 Then
OutMessage = 'FAILURE to execute the following command: ' : DSECommand
Call DSTransformError(OutMessage, "CustomRoutine - LocateFileViaDir")
Return(Ans)
End
* Open the work file
<b>PLEASE READ</b>
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.