Sequential File path query

[chandra.shekhar@tcs.com]

@Craig/BI-RMA
So my only solution is to use it through script or creating a sequence ??
I don't have any other solution, do I ?

[eph]

Using an external source stage is one option, otherwise you can use a bash command in the file pattern expression of the sequential file stage, like

Code: Select all

find #PATH1#/`echo  "#THEGIVENDATE#" | tr -d '-'`/#PATH2# -name "*#FILEPATTERN#*"
or
ls #PATH1#/`echo  "#THEGIVENDATE#" | tr -d '-'`/#PATH2#/#FILEPATTERN#

Eric

[chulett]

Very nice - that filter option looks viable.

[chandra.shekhar@tcs.com]

@Eric
Thanks for your response.
yes I am using file patterns, but can you tell me where I have to put this code ? Is it in the File Path in the Sequential File or in the Filter property.
Because if its in the filter property then it wont work.
And what #PATH1 and #PATH2 stands for according my example ?
I'll try it tomorrow as I left my office(replying from home )

[eph]

You should use it in the file path (it will retrieve file names and pass it as input files list). Filter property is used to pre-parse your data using an external program (somewhat like external source stage could be used).

PATH1 is the first part of your file path, and PATH2 the second part of it (presuming your timestamped folder is an intermediate folder in your file path).

Here, PATH1 is /data/XXX/ and PATH2 is empty (so you can remove it).

Eric

[chandra.shekhar@tcs.com]

OK, I got it.
I'll try it tomorrow and will let you know.
Thanks for your help.

[chandra.shekhar@tcs.com]

Hi,
I tried your both logics. While viewing the data your code works like charm.
But when I run the job, my job aborts while fetching the last row with the below error

Code: Select all

Source subproc: cat: 0652-050 Cannot open ls.
Filter status 512;
filter process failed: 2;
Import error at record 100.

My file has following properties :-
Final Delimeter: End
Deliemter: Comma
Quote: none
Row Count: 100

I think while fetching the last row, it is finding a problem.
I dont know what it is.

[ray.wurlod]

I don't think so. Contemplate the error message itself "Cannot open ls". What does that suggest to you?

[chandra.shekhar@tcs.com]

@Ray,
Well, to be true, UNIX was never my forte
I have tried many ways to run the job, setting different property with in the sequential file.
Even searched here in dsxhange but its of no use

[eph]

What exact command line did you used?

Eric

[chandra.shekhar@tcs.com]

Eric,
I have used the commands what you have told earlier.
As I mentioned too, while viewing the data from the sequential file, the code works fine.
But when I run the job, it throws some error while reading last row.
I have pasted the error too.

[eph]

Where did you put the command line?

It seems that you are using a filter command instead of a file pattern command.

You should have
Source=>Read Method=>File Pattern
Source=>File Pattern=>the command line I gave you adapted to your needs

No filter can be used (at least in v8.1) when you select File Pattern as Read Method.

I've just tested on my env and it's working well. Ensure that there's no space in your command and that the back apostrophe " ` " are correctly sets

Eric

[chulett]

Tell us exactly what you put where in the stage.

[chandra.shekhar@tcs.com]

@Craig/Eric,
Yes I have put the command lines where you have mentioned

You should have
Source=>Read Method=>File Pattern
Source=>File Pattern=>the command line I gave you adapted to your needs.
It seems that you are using a filter command instead of a file pattern command

I havent used the filter property, as I have told you earlier.
The code what you have given is working fine while viewing the data from the Stage.
But when I run the job, it aborts while reading the last line(telling you this for the third time )
Even the performance statistics shows fetching 100 rows but after a couple of seconds the job fails.

@Eric,
Have you tested this code at your end ?
Didnt you get the same error when you ran the job ?

[chulett]

We know what you've told us. Eric has already told you he tested it and it worked fine. You need to answer the questions asked. Don't just say "I put the command lines where you mentioned" as Eric told you to "adapt them to your needs".

Show us your exact commands (asking this from you for the third time) and where you put them. Then we can stop guessing what might be going wrong.