Your recollection is correct. The reference data are loaded into a hash table in memory and an index to that is built. (That's why the Lookup stage is a composite operator: LUT_CreateTable and LUT_Lookup.) You can do this ahead of time by populating a Lookup File Set.qt_ky wrote:A normal Lookup is doing something more behind the scenes that does not appear to be documented--as it reads data from the reference link, it writes it to some sort of memory mapped file that may or may not have an index built on the key columns (I don't know), and then finally it reads that file into memory.
non-database sparse lookup alternative options
Moderators: chulett, rschirm, roy
-
- Participant
- Posts: 54607
- Joined: Wed Oct 23, 2002 10:52 pm
- Location: Sydney, Australia
- Contact:
IBM Software Services Group
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Any contribution to this forum is my own opinion and does not necessarily reflect any position that IBM may hold.
Aha! Perhaps an option for you... pre-build that before recycling the job?
And kids, don't try this one at home - have the 64bit version of PX installed everywhere EXCEPT production. Large lookups work fine up until that last critical elevation. That was a not fun situation to walk into.
And kids, don't try this one at home - have the 64bit version of PX installed everywhere EXCEPT production. Large lookups work fine up until that last critical elevation. That was a not fun situation to walk into.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers
Thank you, Ray. That gives us another avenue to explore.
Must the server engine's uvconfig file be modified from the default of:
64BIT_FILES 0
to:
64BIT_FILES 1
and regenerated/restarted to enable 64-bit hashed file capability?
Craig, could you elaborate or post a link to your reference about the 64bit version of PX? I'm not sure what you meant by that.
Must the server engine's uvconfig file be modified from the default of:
64BIT_FILES 0
to:
64BIT_FILES 1
and regenerated/restarted to enable 64-bit hashed file capability?
Craig, could you elaborate or post a link to your reference about the 64bit version of PX? I'm not sure what you meant by that.
Choose a job you love, and you will never have to work a day in your life. - Confucius
First answer: NO. That forces every hashed file in the project to be 64bit. You just need to pre-create a hashed file as 64bit, I don't recall the syntax for that but it has been mentioned here and I'm sure it's in the docs somewhere. Ah... discussed here.
I just meant that the Production environment there was running the 32bit version of PX but 64bit everywhere else. Bottom line was things like lookups and their "memory maps" that ran fine in all of the lower environments could fail in production unless they handled the partitioning more better.
I just meant that the Production environment there was running the 32bit version of PX but 64bit everywhere else. Bottom line was things like lookups and their "memory maps" that ran fine in all of the lower environments could fail in production unless they handled the partitioning more better.
-craig
"You can never have too many knives" -- Logan Nine Fingers
"You can never have too many knives" -- Logan Nine Fingers