execSH Problem..plz help
Moderators: chulett, rschirm, roy
execSH Problem..plz help
Hi , I want to delete a sequential file after run a job. I had write a shell script to delete the sequential file .So , inside the job properties , I set the After-job-routine to "ExecSH" and put the following value inside the input value :
"/u02/dsclear.sh ":DATAPATH:" ":SYSTEMNME:" ":JOBNAME:"" "/u02/dsstg/data/DEV_PDS_MY/UtilityScript/log/dsclear.log"
DATAPATH,SYSTEMNME,JOBNAME are the parameters to pass inside the shell script ,I had set their values as STAGE VARIABLE inside transformer.
is it necessary to put in the log file? r the statement ok?Plz help.Thanks.
i m newbie to DS, plz help and show me in detail.
"/u02/dsclear.sh ":DATAPATH:" ":SYSTEMNME:" ":JOBNAME:"" "/u02/dsstg/data/DEV_PDS_MY/UtilityScript/log/dsclear.log"
DATAPATH,SYSTEMNME,JOBNAME are the parameters to pass inside the shell script ,I had set their values as STAGE VARIABLE inside transformer.
is it necessary to put in the log file? r the statement ok?Plz help.Thanks.
i m newbie to DS, plz help and show me in detail.
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
Re: execSH Problem..plz help
Use the following statement in your after job routine.
Call DSExecute("UNIX","rm FILEPATH", Output, SystemReturnCode)
ErrorCode=0[/quote]
Call DSExecute("UNIX","rm FILEPATH", Output, SystemReturnCode)
ErrorCode=0[/quote]
-
- Participant
- Posts: 3337
- Joined: Mon Jan 17, 2005 4:49 am
- Location: United Kingdom
Re: execSH Problem..plz help
This is the crux of the problem. Stage Variables are (as their name implies) local to the stage in which they are defined and used. Outside of that stage - or after job - they do not exist.chooi wrote:DATAPATH,SYSTEMNME,JOBNAME are the parameters to pass inside the shell script ,I had set their values as STAGE VARIABLE inside transformer.
They would need to be Job Parameters to work in this manner, but that also means they would have to be known when the job starts, not derived during the job. Is that a possibility?
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Craig has explained clearly why what you're attempting to do won't work.
I think Sainath.Srinivasan has given you a pretty clever way to achieve what you desire in his first post in this conversation. Perhaps the brevity of his post didn't make this obvious.
In the transformer in which you know the values of your stage variables:
1) Add an extra output file (e.g. MyShellScript.ksh)
2) Create a single large column in this file (e.g. MyShellScript VarChar(1000))
3) Build the shell script content that you want to execute in the output derivation for this column
4) If you know the values at the beginning of the process, set the output constraint to @OUTROWNUM = 1, or if you don't know the values until the end of the process, pass it through an aggregator using "Last"
Now the shell script is in a persistent store on disk that you can execute "after job".
Mike
I think Sainath.Srinivasan has given you a pretty clever way to achieve what you desire in his first post in this conversation. Perhaps the brevity of his post didn't make this obvious.
In the transformer in which you know the values of your stage variables:
1) Add an extra output file (e.g. MyShellScript.ksh)
2) Create a single large column in this file (e.g. MyShellScript VarChar(1000))
3) Build the shell script content that you want to execute in the output derivation for this column
4) If you know the values at the beginning of the process, set the output constraint to @OUTROWNUM = 1, or if you don't know the values until the end of the process, pass it through an aggregator using "Last"
Now the shell script is in a persistent store on disk that you can execute "after job".
Mike
Hi all,
I already know the parameters that i have to passed in before the job run.I think the easiest way to do is just put in the run statement to run the script in "input value" of after-job subroutine in "job properties" window.
Below is my statement :
"cd /u02 ; ./dsclear.sh "/u02/dsstg/data/DEV_PDS_MY" "CMS" "CMSPrepRefSeqFile/dummy""
Any comment?please correct me if i am wrong.Thanks.
I already know the parameters that i have to passed in before the job run.I think the easiest way to do is just put in the run statement to run the script in "input value" of after-job subroutine in "job properties" window.
Below is my statement :
"cd /u02 ; ./dsclear.sh "/u02/dsstg/data/DEV_PDS_MY" "CMS" "CMSPrepRefSeqFile/dummy""
Any comment?please correct me if i am wrong.Thanks.
Change it to:
Code: Select all
"cd /u02 ; ./dsclear.sh /u02/dsstg/data/DEV_PDS_MY CMS CMSPrepRefSeqFile/dummy"
Mamu Kim
Chooi,
Based on your statement:
The correction that Kim gave you should fix your problem, but you really shouldn't be hard-coding things like path names. These will likely change when you move from DEV to TEST to PROD. Since you know the values before the job starts, you should use job parameters within the job and in the after job routine (e.g. jpScriptDir, jpDataDir, etc.)
Mike
Based on your statement:
I don't think you need to dynamically generate a shell script from within your job (which was the technique that I tried to communicate to you). You would only attempt to do that if the values were unknown when the job started, and the job was responsible for deriving their content."I already know the parameters that i have to passed in before the job run."
The correction that Kim gave you should fix your problem, but you really shouldn't be hard-coding things like path names. These will likely change when you move from DEV to TEST to PROD. Since you know the values before the job starts, you should use job parameters within the job and in the after job routine (e.g. jpScriptDir, jpDataDir, etc.)
Mike