HP 3000 Manuals HP 3000 Manuals
KSAM EXTRA DATA SEGMENTS
Another factor that may affect performance when KSAM files are used, is
the number and size of KSAM extra data segments. An extra data segment
(XDS) is an area in memory used as a buffer during KSAM file access.
Each extra data segment contains:
* Statistical information on file use (listed by VERIFY command);
* Control Block and Key Descriptor Block data from the first two
sectors of each KSAM key file;
* A Working Storage area large enough to hold a data record and two key
entries;
* A data block buffer large enough to hold a block from the data file;
* At least one, and up to 20, key block buffers each large enough to
hold one key block.
When the key file is searched for a particular data record, the root
block and lower level blocks, as needed, are moved to key block buffers
in the extra data segment. Key entries are compared in the working
storage area. When the data block is located, it is moved to the data
block buffer of the extra data segment, and when the particular data
record is located, it is moved to the working storage area.
Since each open KSAM file is allocated an extra data segment and each
extra data segment can be up to 32K words long (32,767 words), KSAM files
can use a lot of memory. When there is not enough room in memory for all
the extra data segments, they must be swapped between memory and disc as
needed. This swapping can slow access to KSAM files.
In order to minimize swapping, you can reduce the number of KSAM files by
combining several files into one file. This automatically reduces the
number of extra data segments, and it can be a very effective way to
improve performance, particularly when files are shared by a number of
users. (Refer to Number of Extra Data Segments, below.)
Decreasing the overall size of the extra data segment may reduce swapping
of extra data segments. However, reducing the number of key block
buffers in the extra data segment may increase swapping of key blocks
between the key file and the extra segment during a file search. By
default, KSAM allocates key block buffers according to a formula that
takes into account the type of access for which the file is opened, the
number of levels in the key file structure, and the number of alternate
keys in the file. Since this formula (see Table B-1) keeps the extra
data segment size as small as is compatible with efficiency, the default
number of key block buffers should be used except in special cases. (For
details, refer to Extra Data Segment Size later in this section.)
NUMBER OF EXTRA DATA SEGMENTS
KSAM assigns an extra data segment to each KSAM file opened by an active
process. Since more than one process can use the same file during shared
access, one file may require a number of extra data segments. Thus, the
number of extra data segments depends both on the number of KSAM files
and the number of users concurrently using the file. (Refer to Figure
B-10.)
![[]](../../pic3k/M010088/F0B103c50.gif)
Figure B-10. Extra Data Segments for Shared Access
EXTRA DATA SEGMENT SIZE
The size of each extra data segment associated with an open KSAM file is
determined by the number of key block buffers it contains, the size of
each key block buffer, the size of the data block buffer, and to a lesser
extent, the key entry size and the data record size. (Refer to Figure
B-11.)
Initially (when a file is opened), 12K words are allocated to the extra
data segment. If less actual space is needed, the extra space is not
used, but remains in virtual memory. If more is needed, the original
extra data segment is released and a new extra data segment is allocated
with the actual size needed.
The maximum size of any extra data segment is 32K words. The actual size
is calculated from:
* The total size of the overhead statistics and working storage area;
* The size of the data block buffer;
* The size of each key block buffer and the number of buffers
allocated.
The overhead statistics and working storage area is approximately 1-1/2K
bytes long depending on variables such as the key entry size and the data
entry size. The data block buffer size is based on the size of each data
block in the data file. Each key block buffer must be large enough to
contain all the key entries in a key block plus one key entry used when
new keys are added to a full block (as described earlier, see Figure
B-2).
The default key block size is 2K bytes (1024 words) and the maximum size
of the key block buffer is 4K bytes (2048 words). If a key block is
larger than 4K bytes, KSAM reduces the block size so that no block is
larger than will fit in an extra data segment key buffer. Thus, the main
variable in extra data segment size is the number of key block buffers.
![[]](../../pic3k/M010088/F0B113c50.gif)
Figure B-11. KSAM Extra Data Segment
NUMBER OF KEY BLOCK BUFFERS.. The number of key block buffers depends on
the type of access for which the file is opened, the number of keys in
the file, and the number of levels in the tree structure for each key.
(Refer to Table B-1 for details.) The least number of buffers is
allocated for read only access, unless the primary key has many levels in
its structure. More buffers are usually required for write only,
read/write, or update access. The number of buffers for read only access
increases with the number of levels used by the key, but is never less
than one. The number of buffers for write only access increases with the
number of alternate keys in the file, but is never less than six. The
number of buffers for all other types of access increases with the number
of alternate keys and with the number of levels for each key, but is
never less than four.
Unless you specify a particular number of key block buffers, KSAM
allocates buffers in the extra data segment according to the file
characteristics as shown in Table B-1.
Table B-1. Number of Key Block Buffers Assigned by Default
--------------------------------------------------------------------------------------------
| | |
| Access Type | Buffers Assigned |
| | |
--------------------------------------------------------------------------------------------
| | |
| Read Only Access | 1 buffer per level in key with most levels |
| | (minimum of 1 buffer up to 20 buffers) |
| | |
--------------------------------------------------------------------------------------------
| | |
| Write Only Access | 3 buffers per primary key + 3 buffers per alternate key + 3 buff|rs
| | (minimum of 3 buffers up to 20 buffers) |
| | |
--------------------------------------------------------------------------------------------
| | |
| Other Access | 1 buffer per level in + 1 buffer per level + 3 buffers |
| (Read/Write or | primary key structure in alternate key structure |
| Update) | (minimum of 3 buffers up to a maximum of 20 buffers) |
| | |
--------------------------------------------------------------------------------------------
Note that you can determine the number of levels per key with the
KSAMUTIL command, VERIFY.
For example, if the file is opened for read only, and the only key needs
two levels, two key block buffers are allocated. If the file is opened
for write only, and there is one alternate key in the file, nine key
block buffers are allocated. If this same file is opened for update
access, the primary key uses two levels, and the alternate uses three, a
total of eight buffers is allocated.
If you want to override the number of key block buffers assigned by
default, you can use the MPE :FILE command before opening the file, or
set the numbuffers parameter of FOPEN when you open the file
programmatically.
The file equation is specified as follows:
:FILE filename; DEV=,,#buffers
The KSAM extra data segment will be allocated space for as many key block
buffers as you specify, up to a maximum of 20. (Note that the third DEV=
parameter is interpreted as the number of key block buffers only when the
file name is a KSAM file; for standard MPE files, this parameter
indicates the number of list copies of the file.)
Another way to reduce the number of key block buffers is to use fewer
alternate keys, or to adjust the blocking factor so that the key file
structure uses fewer levels. Either of these methods is effective when
the file is written to or updated more than it is simply read.
Note that when you are loading a KSAM file with large amounts of data,
you should increase the number of key buffers. The more key buffers in
the extra data segment, the more likely it is that, as new data is added,
locations for the new key values will be found in memory. This cuts down
on disc access and can significantly reduce the time it takes to load the
file.
For example, if you are reloading a KSAM file after a system failure, you
should use the :FILE command to increase the number of buffers to maximum
of 20 buffers. Then, after the file is loaded, you can allow it to
revert to the default number of buffers established by KSAM for the
particular file.
EXTRA DATA SEGMENTS WITH SHARED ACCESS
The extra data segment allocated to each open file acts as a control
block for that file. The extra data segment contains not only the
current data block and the current key block buffers, but also the latest
control information for the file. This information includes the logical
and chronological record pointers that indicate the current record being
accessed. Because the current pointer position is not in a "common
block", when several programs open the same file, each can alter the key
file structure by adding or deleting records so that the pointers set by
other programs may point to the wrong record without those other programs
being aware of it.
To make sure that the latest pointer position is stored with the file
rather than in the separate extra data segments, programs that share the
same KSAM file must use a locking scheme. Whenever a program locks a
KSAM file, the control information is transferred from the file to the
extra data segment; and when a program unlocks the file, the contents of
the extra data segment is written back to the file. Thus, each program
should lock a KSAM file before executing any procedure that positions a
record pointer (pointer-independent procedures), and not unlock the
file until all procedures that depend on this pointer position
(pointer-dependent procedures) have completed execution. This is true
regardless of whether the pointer is chronological (points to a record in
the data file) or is logical (points to a key in the key file). Both
types of pointer are maintained in the extra data segment for the open
file.
Table B-2 lists all the procedures that affect or are affected by the
record pointers.
Table B-2. Pointer Dependence
-------------------------------------------------------------------------------------------
| |
| Pointer- Pointer Pointer-Dependent Pointer |
| Independent Type Procedures Type |
| Procedures |
| |
-------------------------------------------------------------------------------------------
| |
| FFINDBYKEY Logical FREAD Logical |
| CKSTART CKREAD |
| BKSTART BKREAD |
| |
-------------------------------------------------------------------------------------------
| |
| FFINDN Logical FSPACE Logical |
| |
-------------------------------------------------------------------------------------------
| |
| FREADBYKEY Logical FREMOVE Logical |
| CKREADBYKEY CKDELETE |
| BKREADBYKEY BKDELETE |
| |
-------------------------------------------------------------------------------------------
| |
| FWRITE Logical FUPDATE Logical |
| CKWRITE CKREWRITE |
| BKWRITE BKREWRITE |
| |
-------------------------------------------------------------------------------------------
| |
| FPOINT Chronological FREADC Chronological |
| |
-------------------------------------------------------------------------------------------
| |
| FREADDIR Chronological* FREADC Chronological |
| |
-------------------------------------------------------------------------------------------
*Each procedure that sets the logical pointer also sets the chronological
pointer; but only FPOINT sets the logical pointer as well as the
chronological pointer.
The pointer-independent calls position the pointer regardless of its
current position. Pointerpendent calls, on the other hand, must know to
which record the pointer is currently positioned in order to operate
correctly.
All the procedures listed in Table B-2 affect the pointer in some way.
In order to use these procedures correctly, it is important to understand
how each moves the pointer, whether it positions the pointer directly or
advances it from its current position.
In general, when access to the file is random, the pointer is positioned
directly. For example, a call to FFINDBYKEY (or CKSTART or BKSTART)
positions the logical pointer to a particular key in the key file based
on a key value specified in the call; and a call to FPOINT positions the
chronological pointer to a particular record determined by its
chronological record number.
When access to a file is sequential or the file is being modified,
pointer positioning is not direct but is relative to its previous
position. Depending on the sequence in which procedures are executed,
the pointer may or may not be advanced to the next record in key or
chronological sequence. Internally, a flag is used to indicate whether
or not to advance the pointer. This flag, the "Do Not Advance" flag, is
set to FALSE if the pointer is to be advanced sequentially, to TRUE if it
is not to be advanced. Some procedures never test the flag; these are,
in general, the pointer-independent procedures that set the pointer
directly. Other procedures test the flag and advance the pointer if the
flag is FALSE; generally, these are procedures that read the file or
position the pointer sequentially. Table B-3 summarizes when the pointer
is set or advanced. (Note that only SPL procedures are listed; check
Table B-2 for the equivalent BASIC or COBOL procedures.)
Table B-3. Record Pointer Summary
![[]](../../pic3k/M010088/TA0B033c50.gif)
For example, if you call FREADBYKEY, it positions the pointer to a
specified key value. After the call, the logical pointer remains
positioned to this key and the Do Not Advance flag is set to FALSE. If
the next call is to FREAD, FSPACE, or FREADC, then the pointer is
advanced to the next key in key sequence before these procedures perform
their other functions. Thus, after FREADBYKEY, a call toFREAD will read
the next record, not reread the same record, and a call to FSPACE will
move the pointer relative to the record following the record just read.