Hi.
If I have a hash file with a certain key, how can I do to access the last
record within the key ?
This is for accessing a SAP table with more than 2 million record via
CPI-C. There is a timeout defined, and I have to download the table by
"sets of records" (by example, by 100000 records). And a way is to store
each 100000 records in a hash file, and each run of the job I can access to
the last key generated in the hash file, and retrieve the following 100000
records next to that key.
Is this clear ? Any ideas ?
Guillermo P. Barsky - gbarsky@osde.com.ar
Gerencia de Sistemas - Desarrollo
OSDE Binario - Filial Metropolitana
Alem 1067, Piso 16
(5411)4510-4330
http://www.osde.com.ar
Accessing to the last record in a hash file
Accessing to the last record in a hash file
<b>PLEASE READ</b>
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
Accessing to the last record in a hash file
The very nature of a hashed file means that there is EXACTLY ONE record per key. That record is, therefore, the first and last record for that key.
If (as it seems from your post) you want the highest key value used so far in the hashed file, then things are no different from any other database. SELECT MAX(@ID) FROM hfile;
However, without an index on @ID, this will perform slowly, as it will require an entire table scan to resolve.
You may be better off loading your hashed file via a UV stage, where the table was created with an autoincrementing key, for example:
CREATE TABLE hashedfilename (key INTEGER NOT NULL PRIMARY KEY DEFAULT NEXT AVAILABLE, ...);
Probably better for performance is to append the partial extracts from SAP into a text file, then load the hashed file in one pass from that.
----- Original Message -----
From: gbarsky@osde.com.ar
Date: Fri, 14 Nov 2003 19:37:40 -0300
To: datastage-users@oliver.com
Subject: Accessing to the last record in a hash file
>
>
>
>
> Hi.
>
> If I have a hash file with a certain key, how can I do to access the last
> record within the key ?
>
> This is for accessing a SAP table with more than 2 million record via
> CPI-C. There is a timeout defined, and I have to download the table by
> "sets of records" (by example, by 100000 records). And a way is to store
> each 100000 records in a hash file, and each run of the job I can access to
> the last key generated in the hash file, and retrieve the following 100000
> records next to that key.
>
> Is this clear ? Any ideas ?
>
>
> Guillermo P. Barsky - gbarsky@osde.com.ar
> Gerencia de Sistemas - Desarrollo
>
> OSDE Binario - Filial Metropolitana
> Alem 1067, Piso 16
> (5411)4510-4330
> http://www.osde.com.ar
>
If (as it seems from your post) you want the highest key value used so far in the hashed file, then things are no different from any other database. SELECT MAX(@ID) FROM hfile;
However, without an index on @ID, this will perform slowly, as it will require an entire table scan to resolve.
You may be better off loading your hashed file via a UV stage, where the table was created with an autoincrementing key, for example:
CREATE TABLE hashedfilename (key INTEGER NOT NULL PRIMARY KEY DEFAULT NEXT AVAILABLE, ...);
Probably better for performance is to append the partial extracts from SAP into a text file, then load the hashed file in one pass from that.
----- Original Message -----
From: gbarsky@osde.com.ar
Date: Fri, 14 Nov 2003 19:37:40 -0300
To: datastage-users@oliver.com
Subject: Accessing to the last record in a hash file
>
>
>
>
> Hi.
>
> If I have a hash file with a certain key, how can I do to access the last
> record within the key ?
>
> This is for accessing a SAP table with more than 2 million record via
> CPI-C. There is a timeout defined, and I have to download the table by
> "sets of records" (by example, by 100000 records). And a way is to store
> each 100000 records in a hash file, and each run of the job I can access to
> the last key generated in the hash file, and retrieve the following 100000
> records next to that key.
>
> Is this clear ? Any ideas ?
>
>
> Guillermo P. Barsky - gbarsky@osde.com.ar
> Gerencia de Sistemas - Desarrollo
>
> OSDE Binario - Filial Metropolitana
> Alem 1067, Piso 16
> (5411)4510-4330
> http://www.osde.com.ar
>
<b>PLEASE READ</b>
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.
Do not contact admin unless you have technical support or account questions. Do not send email or Private Messages about discussion topics to ADMIN. Contact the webmaster concerning abusive or offensive posts.