More Opened Files in MPE/iX Programs
MPE documents | |
 Complete PDF | |
| Table of Contents | |
| Index |
Introduction
When a file is opened by a process, an entry is made in a table of finite size. The number of files that can be opened concurrently in a program depends on the size of this table. It also depends on several other resources used in the program. In a POSIX program, many resources are treated as files for ease of reference. They are sockets, pipes, streams etc. With MPE/iX being used as a web server by more and more customers, entries in the process local file table are being used extensively. This in turn limits the number of files that can be opened by a process. This article describes an enhancement to MPE/iX 7.5, enabling a process to open more files.
Current Limitation
At present, a process can keep only up to 1024 file objects in the opened state. This includes 9 system files, sockets, pipes, streams etc. For regular MPE files, one entry is made in the Process Local File Descriptor (PLFD) table and for pipes, two entries are made. When a device like a serial communication channel is opened, then one entry is made in this table.
Need for Expansion
With MPE/iX being ported to high-end systems, the environment in which it is being used is also changed. Applications running on multiple low-end systems are consolidated into single high-end systems. During consolidation and application enhancements, several customers have run into this limit. They were overcoming this limit by restructuring their applications. Most of them reached a state in which they cannot manage by restructuring alone. The main factor now is the number of devices that are used in the applications. Most of them have exhausted all available options to further reduce the number of files they need to open. The limit on files opened per process is now preventing the number of their clients to grow. This had a significant negative impact on many solution providers. A process can open up to a maximum of 1024 sockets concurrently. This allows 1024 simultaneous external connections. For any useful application, the program needs to open a large number of files also. Unless the size of the PLFD table is increased, one program can never to make use of the maximum number of allowed socket connections. To enable the customers to open a large number of files in their programs and to make more external socket connections, it was decided to increase the number of entries in the PLFD table from 1024 to 4096.
Design Changes
There are 3 tables in which the file information is stored. All of these tables are per process structures. The first table is the Process Local File Descriptor (PLFD), this table is used to keep track of the opened files. This table exists in the system global space. The entry number in this table is same as the file number returned to the user, when a file is opened. The size of this table is increased to the limit based on demand, in multiples of 64 at a time. To begin with, a process starts with a table having a size of 64. The size of this table can be increased 63 times. This allows a maximum of 4096 opened files. Since the size of the table is increased based on demand, the impact on the consumption of system global space is negligible for existing installations which do not need this enhancement. The second table is the Process Local File Access State Table (PLFAST) which is used primarily for unblocked input/output to files. In this table, a series of bitmaps are kept in which each bit represents the state of one of the possible files that a process may have opened. This table is always memory resident. This table also exists in the system global space and is expanded there itself. The PLFAST table is implemented as a single table and all processes will get a table with the enhanced size. The third table, the File Access Rights (FAR) table is used by the file system and virtual space management to keep track of which files a process has a legitimate right to access. This table is hash indexed using the interval timer present in the unique file identifier. At present, this table exists as part of the process information block extension in the system global space. This table is accessed by a very small and localized set of routines, which makes it ideal for moving it out of the system global space. In this enhancement, the size of the table is increased and moved to the process local address space. A pointer to this table is kept in the process information block. No separate locking mechanism is provided for accessing this new table. It is considered as an extension of the process information block in the process local address space.
Performance
With this enhancement, there is not any adverse effect on performance. On the other hand, there are improvements in performance if the program is opening more than 512 files. Prior to this enhancement, only the first 512 entries in the FAR table were hash addressable. Now the entire 4096 entries are made hash addressable and this speeds up the search operation in the FAR table which in turn reduces the time spent in locating an entry.
Compatibility Issues
There are not any compatibility issues for the existing applications. All the intrinsics interface remain without change. An application running on the previous versions runs on the enhanced 7.5 release without any problem.
Limitations of Current Expansion
This enhancement affects only the number of file like resources. The current limits on sockets, pipes and streams are not increased individually. However, this enhancement helps users in using more number of these resources in presence of large number of regular files. Note that a program making use of these enhancements does not run on the earlier versions of MPE/iX.
Conclusion
With this enhancement, users are able to concurrently open more files, sockets, pipes and streams. The current limit of 1024 is increased to 4096. This enables users to develop programs to handle a large number of files in conventional OLTP applications and in Web applications.
|
IODFAULT Generic Device IDs |
Support for 2851 userlog Processes |