Importing sequential file with double quotes in data

Post questions here relative to DataStage Enterprise/PX Edition for such areas as Parallel job design, Parallel datasets, BuildOps, Wrappers, etc.

Moderators: chulett, rschirm, roy

Post Reply
giucar
Premium Member
Premium Member
Posts: 2
Joined: Wed Nov 09, 2016 10:49 am

Importing sequential file with double quotes in data

Post by giucar »

I'm trying to import a file csv using parallel job but there is a problem with a String field. It contains "somevalue" inside, but every field is delimited with " (double) and separated with ;.

For example, if i have, as value of a String field, this value delimited:

;"abcd ""somevalue"";

This outputs a warning and the record is filtered.

Is there a solution with Parallel job without changing delimited value?

Thanks.
boxtoby
Premium Member
Premium Member
Posts: 138
Joined: Mon Mar 13, 2006 5:11 pm
Location: UK

Post by boxtoby »

Hi Guicar,

I have seen this sort of problem before, even to the extent that there were 4, 5 or even 6 double characters in the data!

I solved the problem by using the sed command:

sed s/'"";'/'";'/g input file1 > output file2
sed s/'""'/' '/g input file2 > output file3

This worked for me, even repeating the process several times.

Other may have more elegant solutions, but until then ........

Regards,

Bob.
Bob Oxtoby
giucar
Premium Member
Premium Member
Posts: 2
Joined: Wed Nov 09, 2016 10:49 am

Post by giucar »

Thanks boxtoby,
I prefered to use Server Job and I solved the problem.

Regards.
Post Reply