Reading multiple files from network using Sequential stage

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

SURA
Premium Member
Premium Member
Posts: 1229
Joined: Sat Jul 14, 2007 5:16 am
Location: Sydney

Post by SURA »

This is nothing to do with Datastage.

Before use the command in the file stage, try with dos command line using ls command.

for example

ls \\pd4p-ccbv-w01\d$\Cognos\DATAMA~1\SOURCE~1\FCUNIT~1\*.CSV

In the initial example you have mentioned space in the file name,whereas now so space.

We don't know what exactly you have there! So try to use to the ls command and once you managed to list the files, in theory you can use that without ls should work.
Thanks
Ram
----------------------------------
Revealing your ignorance is fine, because you get a chance to learn.
chandu123
Premium Member
Premium Member
Posts: 40
Joined: Mon Sep 12, 2016 1:26 pm

Post by chandu123 »

I tried ls on command line as mentioned above and it listed all the files in the directory and they are:

//pd4p-ccbv-w01/d$/Cognos/DATAMA~1/SOURCE~1/FCUNIT~1/Historical Volumes-APR-14.csv
//pd4p-ccbv-w01/d$/Cognos/DATAMA~1/SOURCE~1/FCUNIT~1/Historical Volumes-AUG-13.csv
//pd4p-ccbv-w01/d$/Cognos/DATAMA~1/SOURCE~1/FCUNIT~1/Historical Volumes-DEC-13.csv
//pd4p-ccbv-w01/d$/Cognos/DATAMA~1/SOURCE~1/FCUNIT~1/Historical Volumes-DEC-15.csv
//pd4p-ccbv-w01/d$/Cognos/DATAMA~1/SOURCE~1/FCUNIT~1/Historical Volumes-FY09.csv
//pd4p-ccbv-w01/d$/Cognos/DATAMA~1/SOURCE~1/FCUNIT~1/Historical Volumes-FY10.csv

used the same in the sequential file stage as File Pattern and showing no files returned :roll:
HB
SURA
Premium Member
Premium Member
Posts: 1229
Joined: Sat Jul 14, 2007 5:16 am
Location: Sydney

Post by SURA »

Now you have the space in the file names :(

can you able to create few test files without space and place it the same server and try to read it!

This will eliminate the file path issue. Next concentrate with the file space issue.
Thanks
Ram
----------------------------------
Revealing your ignorance is fine, because you get a chance to learn.
chandu123
Premium Member
Premium Member
Posts: 40
Joined: Mon Sep 12, 2016 1:26 pm

Post by chandu123 »

The file names which I mentioned in my previous post without spaces are the short names of the files which I got them using this command in cmd line:

cmd /c for %A in ("\\pd4p-ccbv-w01\d$\Cognos\Data Manager\Source csv Files\FC Units Shipped\Historical Volumes*.csv") do @echo %~sA

Coming to space issue: I don't think space is causing the issue because I ran my job for single file using below filename and it ran fine:

//pd4p-ccbv-w01/d$/Cognos/DATAMA~1/SOURCE~1/FCUNIT~1/Historical Volumes-APR-14.csv

Also, I created 3 files with out spaces and single file works not if I try to read multiple files :(
HB
SURA
Premium Member
Premium Member
Posts: 1229
Joined: Sat Jul 14, 2007 5:16 am
Location: Sydney

Post by SURA »

Is there any errors or warnings when you ran the job?

Sorry to ask, Have you changed the method to file pattern?
Thanks
Ram
----------------------------------
Revealing your ignorance is fine, because you get a chance to learn.
chandu123
Premium Member
Premium Member
Posts: 40
Joined: Mon Sep 12, 2016 1:26 pm

Post by chandu123 »

Yes i have changed the method to File Pattern and also set the environment variable to read File pattern as file set.

Below are the warnings:
Sequential_File_0: When checking operator: No files found for pattern.
Sequential_File_0: When checking operator: Fileset C:/Windows/TEMP/import_tmp_91445e0be0c0.fs contains no files. If a file pattern was specified this indicates the pattern returned no files.
HB
chulett
Charter Member
Charter Member
Posts: 43085
Joined: Tue Nov 12, 2002 4:34 pm
Location: Denver, CO

Post by chulett »

IMHO you are well past the time you should be engaging your official support provider, if you haven't already...
-craig

"You can never have too many knives" -- Logan Nine Fingers
Post Reply