Filtering special characters

bensonian · Post by **bensonian** » Wed Aug 06, 2008 9:31 am

The source data pull has special charecters, i was wondering if there is a way we could filter that out in the DataStage job. There could be a lot of different special charecters coming in.

Thanks alot gurus

chulett · Post by **chulett** » Wed Aug 06, 2008 9:35 am

Define "special" in this context.

chulett · Post by **chulett** » Wed Aug 06, 2008 4:06 pm

"Alphanumeric" just means a character that is a letter or a number. As Ray notes, you need to tighten down your requirements first.

chulett · Post by **chulett** » Wed Aug 06, 2008 4:24 pm

Why not try to get them to load without aborting? Perhaps if you posted the errors generated, we could help make that happen.

senthilt1 · Post by **senthilt1** » Wed Aug 06, 2008 10:07 pm

Hi,

Even i had the same issue removing the special characters from this input records. We have written a script that search for special characters in the input records and replaces all the special characters with emptyspace. We called this script in external filter or external source depends on the flow. This is how we are doing.

I am not sure whether this is the right way..........FYI.

Thanks,

Senthil P

bensonian · Post by **bensonian** » Thu Aug 07, 2008 9:34 am

chulett wrote:Why not try to get them to load without aborting? Perhaps if you posted the errors generated, we could help make that happen. ...

The error is:

sqlcode is: -12899
esql complaint: ORA-12899: value too large for column "<schema_name>"."<table_name"."<column_name>" (actual: 32, maximum: 30)

the length of <column_name> is varchar(30).

I actually identified the record based upon row_num and
output > file

try to insert into db again. When i was viewing it, from file>view data. DS was actually adding extra bytes to it and it's not the same when i view the file on unix box. The way DS is interpreting it is wierd, so i happened to check the NLS map, which is not turned on at the project level.

Is it the 'NLS map' thats causing the whole issue. Please advise me of how this ('NLS') could be turned on at project level.

Appreciate all your help.
file >

ray.wurlod · Post by **ray.wurlod** » Thu Aug 07, 2008 3:16 pm

NLS is enabled, and default map selected for a project, using the Administrator client NLS button once that project has been selected.

bensonian · Post by **bensonian** » Thu Aug 07, 2008 3:59 pm

ray.wurlod wrote:NLS is enabled, and default map selected for a project, using the Administrator client NLS button once that project has been selected. ...

Ray,

surprisingly for the none of the project's in Administrator 'NLS' tab is neither highlighted or enabled. So how would DS do the charecter mapping between DB and DS engine and how does it interpret the 'accented characters'.

ray.wurlod · Post by **ray.wurlod** » Thu Aug 07, 2008 4:13 pm

What is the value of the NLSMODE parameter in your uvconfig file?