To read input file dynamically

kamalshil · Post by **kamalshil** » Mon Aug 30, 2010 8:01 am

Hi All,
I have 5 input files which but with different structure.
We need to read the input flow and write into output file.

My question is we need to make this read of input file dynamically so that any time when required we will read only 10 columns or 5 columns and write into output file.

And planned to do in one job.I am planning to use RCP.

Is there any way i can really read only the required columns from the file directly?

Or i can read all the columns of file and drop in between the job before target.

arunkumarmm · Post by **arunkumarmm** » Mon Aug 30, 2010 9:09 am

I dont think you can read only the columns you need from a file as you do in a table. If it is a dataset, then yes you can do it. I believe for a file, you need to read the entire record, and need to drop the unwanted columns.

To select/drop columns dynamically, you can use schema files.