Page 1 of 1
DS Version Control and UFS filesystem limitation
Posted: Fri Jan 20, 2006 2:42 pm
by bjkilleen
While working with the DS Version Control product, we began experiencing the following error:
Error while creating job SeqGrrServiceStat^1_4
Error on CREATE.FILE commenat (Creating file "DS_TEMP5455" as Type 30.
mkdbfile: cannot create file DS_TEMP544 Unable to create operating system
"DS_TEMP5455" Process cannot continue.)
We believe it may be related to a UFS filesystem limitation on the number of directories/files, 32768. Is there anyone else that has experienced this? What exists as a resolution?
We have tried to "delete" DS components from DS Version Control, but this has not resolved the issue.
Thanks in advance!!!
Posted: Fri Jan 20, 2006 7:33 pm
by ray.wurlod
I have seen this limit on Solaris; max 32K subdirectories in a directory. In that case it was also the VERSION project (for Version Control); we got around it by converting dynamic hashed files (directories) into static hashed files (O/S files).
Given that each version of each job creates dynamic hashed files, the VERSION project is the most likely to hit this limit.
Another possibility is to archive older versions of jobs, and/or to use multiple version control projects.
Find out how many entries there are in the directory. Of these, how many are subdirectories? ls -l | grep ^d | wc -l
Posted: Mon Jan 23, 2006 7:07 am
by bjkilleen
Thanks Ray.
How does one convert the dynamic hashed files into static hashed files? After doing this does DS Version Control continue to use(create) the static hashed files?
We have definitely thought about creating multiple version control projects but feel that we may run into the same situation.
When you state " archive older versions of jobs", you are meaning to export the older versions of jobs and then delete them from DS Manager, right?
When I used the command provided, the result was 32765.
Again Ray, thanks for the information.
Brad
Posted: Mon Jan 23, 2006 8:01 am
by ArndW
The conversion of one type of a hashed file to another is through the RESIZE command. It is important to ensure that the files are not in use when you do a RESIZE and the use of the "CONCURRENT" option is generally discouraged. If a file is going to be very small (such as the RT_LOGnnn or RT_STATUSnnn files in VCS) then you can do a simple "RESIZE RT_STATUS001 2 1 1" to convert the file to a type 2, modulo 1 and separation of 1.
BTW, this 32Kb inode limit is also present under AIX
Posted: Mon Jan 23, 2006 8:04 am
by chulett
ArndW wrote:BTW, this 32Kb inode limit is also present under AIX
Yikes... have to keep that in mind. They are contemplating moving some of the DS processing from H-PUX over to AIX here and this could definitely be an issue.
Posted: Mon Jan 23, 2006 8:14 am
by ArndW
The site that I am at has not seen the problem in the version control project(s), only in certain development projects. But the limitation is present and diagnosing the error when it occurs still takes up time. Converting all RT_STATUS<n>, RT_CONFIG<n> files to static will do the trick.
Posted: Mon Jan 23, 2006 8:23 pm
by ray.wurlod
RT_STATUSnn and RT_CONFIGnn are good choices because they are always small and static.
To get a rough idea of how big to make the static hashed files, look at how big the DATA.30 and OVER.30 files are. For example:
Code: Select all
ls -l RT_CONFIG11
total 56
-rw-rw-r-- 1 aoprxw dsdevt 4096 Jan 24 11:48 DATA.30
-rw-rw-r-- 1 aoprxw dsdevt 22528 Jan 20 09:36 OVER.30
ls -l RT_STATUS11
total 16
-rw-rw-r-- 1 aoprxw dsdevt 4096 Jan 24 08:55 DATA.30
-rw-rw-r-- 1 aoprxw dsdevt 2048 Jan 10 10:48 OVER.30
Add the two values and subtract 4096 (the two header blocks). So, in the above example, the RT_CONFIG11 static hashed file would need to be 22528 bytes in size, and the RT_STATUS11 static hashed file would need to be 2048 bytes in size.
Use type 18 and separation 4, and calculate modulo by dividing bytes by 2048. Thus the modulo needed for RT_CONFIG11 here would be 11 and for RT_STATUS11 would be 1. The commands would be:
Code: Select all
RESIZE RT_CONFIG11 18 11 4
RESIZE RT_STATUS11 18 1 4
Posted: Tue Jan 24, 2006 2:01 pm
by bjkilleen
Ray,
What do you mean by "archive older versions of jobs"?
I want to make sure about this, because I have exported older versions of jobs to a dsx file, and then deleted the component from the VERSION project. But, when I went back into DS Version Control, the component was still present.
What have I actually done? Did I actually delete the component from DS Version Control? How do you refresh DS Version Control so that it recognizes that versions of a DS component are no longer present?
Thanks!
Posted: Tue Jan 24, 2006 5:06 pm
by ray.wurlod
Export doesn't delete anything - it's just another way to take a backup.
Once you've taken a backup - or two - you actually need to start deleting things. And therefore you have to find out which things you want to delete. You can start by getting the job numbers from DS_JOBS.
Code: Select all
SELECT NAME,JOBNO FROM DS_JOBS WHERE NAME NOT LIKE '\\%' ORDER BY NAME;
Posted: Wed Jan 25, 2006 2:25 pm
by bjkilleen
Okay, I have my export, and I have also been able to identify the job numbers with the code provided.
What is the next step?
Isn't deleting the DS components from DS Manager removing them from the VERSION project? I have done this, but the DS components still show in DS Version Control. Is there a "refresh" process for the VERSION project, since it is not your typical DS project?
Thanks!