How to hold some dynamic value during the Job run

[Dmitriy]

Hello!
I want to ask about the following:

I have a parallel job, inside which I'am getting the field value from Database.
My purpose is to find - how it is possible to hold this value during the Sequence run and, if necessary, use in downstream jobs. I.e. to hold the value in some variable.

I've tried to pass this value into Job Parameter - inside Transformer stage.
I tried to assign the Value directly to the Job Variable, for example insideTransformer's Stage Variable Section, in Derivation field:

JobParameter = DBColumn

Validation of this Derivation says that it's OK, but the value hadn't been assigned.

Searching this forum I have found that Job Parameters couldn't be changed or assigned during the job run.
So I need some another tool to hold values, obtained during the Job run.

I see this as it was realized in Informatica ETL tool, maping variables and workflow variables, there were ability to assign each value to another and hold the value which is obtained during the run.
What approach of using variables in DataStage, what will fit this task?

Thanks!

[Sainath.Srinivasan]

Are you looking for 'stage variables' or 'lookup stage' ?

[Dmitriy]

Are you looking for 'stage variables' or 'lookup stage' ?

Hello Sainath.Srinivasan!
I'm looking for some type of variable, which will be able to hold my value (which i've got inside the job) on the job level, not only at the stage level.
Correct me if Iam wrong, I thought that Stage Variable is accessible only inside the Stage and can not be accessed in other jobs that follows.

[chulett]

Yes, stage variables are local to a transformer. Perhaps a search for USERSTATUS would help? They can be accessed and transferred at the workflow... err, Sequence job level between jobs easily, especially if you use a Server job to stash that value.

[Dmitriy]

Hello chulett!

Thanks for the advice about USERSTATUS, seems it would require writing some routine. Also to concatenate several values with delimiter into one resulting string, before passing to another job.

I'll try it and then post a result.

A little question - does it matter if I have parallel Job or a Server Job, if there any restrictions to write into UserStatusArea, having parallel Job?

[Sainath.Srinivasan]

Passing user status is for Server jobs. For parallel, you can write in a file and read from that.

[chulett]

Or use a BASIC Transformer? Does that not also work? Never had a chance to actually try that out...

[kumar_s]

You can explore the User Variable activity in Job sequence too.

[Dmitriy]

Or use a BASIC Transformer? Does that not also work? Never had a chance to actually try that out...

chulett, kumar_s - thanks!

I hadn't try to use BASIC Transformer for passing UserStatus in Parallel Job, with my experience I suppose it will be a bit time consuming. I'll try it during a period with less work loads

Thanks to you and some forum search, I reached the goal using a Server Job and DSSetUserStatus routine.
In this job I'am passing the concatenated parameters to UserStatus area, then by means of UserVariables Activity sequence deparsing the values with
Field(Job_Activity.$UserStatus,"|",[1..n]) function.
So seems that all futher job activities can see this variables and it's possible to assign it to Job parameters on a parallel sequence level.

[chulett]

That's exactly what I have done in the past, typically a Server job is perfectly suited for the task of gathering parameter style information even if the main processing 'needs' to be done using parallel technology. Then a Sequence job can easily parse apart the delimited USERSTATUS information from that job and any stage downstream of it can leverage them.

If you'll be doing that for multiple downstream stages, consider doing all of the parsing once in a UserVariables stage and then using them there (with their new 'proper names') downstream.

[arunkumarmm]

Or use a BASIC Transformer? Does that not also work? Never had a chance to actually try that out...

I was trying to do this. Job doesnt gives any warning but doesnt assigns any value to the user status. I'm not sure If I'm missing anything.

[ray.wurlod]

I can't envisage how DSSetUserStatus would work from multiple processing nodes. In parallel jobs you're better off writing to a file unless you call DSSetUserStatus from a BASIC Transformer stage constrained to execute on a single node.