HP 3000 Manuals HP 3000 Manuals
All You Ever Wanted to Know About KSAM
by George Stachnik--Commercial Systems Division
NATIVE MODE KSAM
KSAM has been a feature of MPE V/E systems for many years. Since release
1.0, MPE XL has provided support for KSAM using compatibility mode ported
from MPE V/E. Starting with Release 2.1, MPE XL supports two versions of
KSAM. There is a new native mode implementation called KSAM XL. In
addition, HP is continuing to support the older compatibility mode
version, hereafter called CM KSAM. In most cases, KSAM XL provides
superior performance to CM KSAM.
This article will discuss differences between CM KSAM and KSAM XL, and
the migration issues associated with these two versions of KSAM.
Some Terminology
For clarity's sake, we'll use the following terminology in this article.
* Files created using existing compatibility mode KSAM are referred to
as CM KSAM files, and include both the data and key files. CM KSAM
files support both fixed and variable length records.
* Files created using KSAM XL are referred to as KSAM XL files. KSAM
XL files are not split into data files and key files; one file
contains both data and key information. KSAM XL files do not support
variable length records.
* Application programs may be compiled in either native mode (NM) or
compatibility mode (CM). Both NM applications and CM applications can
access KSAM XL files or CM KSAM files or both.
Features of KSAM XL
KSAM XL includes a number of features that make it easier to use than CM
KSAM:
* KSAM XL files can be created, purged, copied, renamed and saved using
ordinary CI commands. To do the analogous tasks with CM KSAM files,
you will continue to use the KSAMUTIL program.
* KSAM XL files can be created with a REUSE option that permits the
space occupied by deleted records to be reused dynamically* .
_________________
* The REUSE option is not a default. By default, free space in KSAM XL
files is treated in much the same way as in CM KSAM files - that is,
deleted records continue to take up space until the file is reorganized
using FCOPY.
KSAM Performance
A number of benchmarks have been done measuring the performance of KSAM
XL performance as compared to CM KSAM. These have shown that for reads
(especially in key sequence), KSAM XL can provide an advantage of up to
30% over CM KSAM. For writes the differences were less dramatic. In
general, the best performance can be achieved by using NM compilers and
KSAM XL files. This minimizes the use of switch stubs to switch
processing between Native and Compatibility Mode. Because of CM/NM mode
switching, NM programs may run slower if CM KSAM files are accessed than
if KSAM XL files are accessed. Similarly, CM programs may run slower
with KSAM XL files than with CM KSAM files.
Language Issues
As of Release 2.1, KSAM XL files can be accessed by programs written in
any of the following languages:
* SPL
* Pascal
* COBOL1
* FORTRAN
* RPG2
* Business basic
* Basic (in CM only)
________________
1 If you use the CM COBOL II compiler as well as the NM COBOL II compiler,
you may want to maintain any common COPY libraries as CM KSAM files.
This option allows both compilers to read the COPY libraries with keyed
access. The NM compiler may run slightly slower with a CM KSAM COPY
library. For editing and listing COPY libraries, COBEDIT only supports
CM KSAM, although you can optionally convert COPY libraries to KSAM XL
for compiles.
2 RPG users have been able to do record level locking on MPE V/E systems
since UB-MIT. This feature is not supported under MPE XL. CM KSAM and
KSAM XL support locking at the FILE level only.
Migrating to KSAM XL
XL Release 2.1 supports both the CM KSAM and KSAM XL access methods.
Applications that currently use CM KSAM will continue to use it after an
update to Release 2.1. They will not use KSAM XL unless you migrate
them.
Migrating Applications that Use Existing KSAM Files
As long as an application program uses HP supported intrinsics to access
a KSAM file, the file system will determine if it is accessing CM KSAM
files or KSAM XL files, and proceed accordingly. In most cases,
migrating an application to use KSAM XL is simply a matter of converting
the CM KSAM files to the KSAM XL format. The file system does the rest
of the work. This can be done using the following steps:
1. Do a full backup.
2. For each existing CM KSAM file, use the FCOPY utility program to
make a KSAM XL copy of it. (See the footnote for details).3
3. Use KSAMUTIL to purge the CM KSAM file. KSAMUTIL will purge both
the data and key portions.
4. Use the MPE :RENAME command to rename the KSAM XL file. Give it
the same name as the CM KSAM data file you just purged.
_________________
3 The syntax for using FCOPY is as follows:
:FCOPY FROM=MYFILED;TO=(MYFILEN);NEW
In the above example, 'myfiled' is the name of the CM KSAM data file, and
'myfilen' is the name of the new KSAM XL file. FCOPY can also be used to
convert KSAM XL files back into the CM KSAM format. Here's an example:
:FCOPY FROM=MYFILEN;TO=(MYFILEK,MYFILED);NEW
In the above example, 'myfilen' is the name of the existing KSAM XL file,
'myfilek' is the name of the CM KSAM key file, and 'myfiled' is the name
of the CM KSAM data file.
If you plan to transfer KSAM XL files to an MPE V/E system, you must
convert them to CM KSAM with FCOPY before you transfer them. Also, if
you run a CM COBOL II program on a pre-V-Delta-7 MPE V/E system that
accesses a KSAM XL file on an MPE XL system, the program will not be able
to read the file.
Some extremely large CM KSAM files cannot be migrated to KSAM XL. The
problem arises because CM KSAM files are actually made up of two physical
files, the key file and the data file, while KSAM XL combines key and
data information into a single file. So when you're trying to copy very
large files into the KSAM XL format, you may encounter the following
error messages:
:fcopy from=myfile;to=(myfilen);new
HP32212A.03.28 FILE COPIER (C) HEWLETT-PACKARD CO. 1989
*106*CAN'T OPEN TOFILE
DISPLAY FILE INFORMATION (Y OR N) ?y
EXTENT SIZE EXCEEDS MAXIMUM (FSERR 106)
+-F-I-L-E---I-N-F-O-R-M-A-T-I-O-N---D-I-S-P-L-A-Y+
! ERROR NUMBER: 106 RESIDUE: 0 !
! BLOCK NUMBER: 0 NUMREC: 0 !
+------------------------------------------------+
0 RECORDS PROCESSED *** 1 ERROR
END OF SUBSYSTEM
:
Some extremely large CM KSAM files may not migrate to KSAM XL. Since both
versions of KSAM are supported on Release 2.1 of MPE XL the workaround
would be to remain in CM KSAM.
When using FCOPY to convert a CM KSAM file to a KSAM XL file, any deleted
records that remain in the CM KSAM file will not be copied. The
chronological sequence of the active records will be preserved.
Programmatically Creating KSAM Files
In most cases, an application that creates CM KSAM files on pre-Releases
2.1 of MPE XL will continue to create CM KSAM files after an update to
Release 2.1. The application may require some modification in order to
be migrated to KSAM XL.
There are several ways for an application to create a KSAM file. It may
create it using KSAMUTIL, using the FOPEN intrinsic, using the HPFOPEN
intrinsic, or (if it's a COBOL application) using COBOL's ACCESS IS
INDEXED feature. (CKOPEN and CKOPENSHR can only be used on existing
files.)
In order to migrate the application to KSAM XL, follow these procedures:
* Wherever the application uses the KSAMUTIL utility, modify it to use
ordinary CI commands instead (either in batch mode or via an
intrinsic such as HPCICOMMAND). The CI commands :BUILD, :COPY, :FILE,
:LISTEQ, :LISTF, :PURGE, :RENAME and :SAVE can be used to manipulate
KSAM XL files.
* Some applications use FOPEN to create new CM KSAM file by passing a
value of 001 in bit(2:3) of the foptions. After an update to Release
2.1, these applications will continue to create CM KSAM files. In
order to migrate to KSAM XL, issue a file equation specifying the
name of the CM KSAM data file and containing the keyword ;KSAMXL.
This will override the parameters passed to FOPEN. If it isn't
possible to use a file equation, the parameters passed to FOPEN must
be changed in order to get the application to create a KSAM XL file.
Bit(2:3) of foptions should be set to binary '011' for KSAM XL file
type, instead of 001, which specifies the CM KSAM file type. Also,
KSAMparam is an array passed to FOPEN describing the primary and
alternate keys when creating a new KSAM file. The format of
KSAMParam has changed slightly for KSAM XL files.
* Some applications call HPFOPEN and tell it to create a new KSAM file
by passing a value of 1 (CM KSAM) as item number 10 (file type).
After an update to Release 2.1, these applications will continue to
create CM KSAM files. Issue a file equation specifying the name of
the CM KSAM data file and containing the keyword ;KSAMXL. This
overrides the parameters passed to HPFOPEN. If it isn't possible to
use a file equation, the program should be modified to pass HPFOPEN a
value of 3 (KSAM XL) as item number 10 (file type). Some
modification may also be required to the contents of KSAMParam, an
array passed to HPFOPEN describing the primary and alternate keys.
* COBOL II applications can create and access KSAM files without using
intrinsics by specifying ACCESS IS INDEXED in the program. In that
case, NM COBOL II programs build KSAM XL data files, unless they are
specified as variable length files. Variable length files are built
as CM KSAM files. CM COBOL II programs build CM KSAM data files by
default. You can override this by backreferencing a file equation
containing the ";KSAMXL" keyword.
Programmatically Accessing KSAM Files
There are some cases in which applications that read and write CM KSAM
files may require some changes in order to successfully use KSAM XL.
* By default, the data stored in KSAM XL files is stored in
chronological sequence, just as it is in CM KSAM files. However, if
the REUSE option is specified, the records stored in KSAM XL files
will be stored in a random order. Therefore, REUSE should not be
used in applications that depend on data being added in chronological
sequence.
* FCONTROL - Options 7 and 8 are used with CM KSAM files to clear the
buffers so that the next call to a read intrinsic gets the record
from disk instead of from the buffers. This is no longer required
with KSAM XL files.
* FFINDBYKEY, FFINDN, FREADBYKEY, FREADC, FPOINT, FREAD, FREMOVE,
FWRITE - Unlike CM KSAM, KSAM XL does not force you to lock shared
files before calling these intrinsics. However, the user is still
advised to call FLOCK before intrinsic calls to position data
pointers and call FUNLOCK after read, write, remove and update
operations. Failure to do so could cause data consistency problems.
* FGETKEYINFO - This intrinsic returns information about an open KSAM
file. For KSAM XL files, the format of the KSAMCONTROL parameter has
changed.
Accessing KSAM Data in Chronological Order
Application programs can access the records in a KSAM file sequentially.
Sequential access is supported in two ways: in chronological sequence
(that is, the records are read in the order in which they were written to
the file - FIFO), or in key sequence (that is, the records are read in
order, sorted by key value.) When reading records in chronological order
using CM KSAM, it is even possible to read records that have been deleted
by an application program. Under KSAM XL, it is still possible to do so.
But some application changes may be necessary.
One way to access a CM KSAM file in chronological sequence is to use the
;NOKSAM option of FCOPY. The ;NOKSAM option reads the CM KSAM data file
in chronological order. And it reads the 'deleted' records. For KSAM XL
files, the situation is slightly different. You can FCOPY active (that
is, not deleted) records from a KSAM XL file just as you can with a CM
KSAM file. However, FCOPY doesn't support the ;NOKSAM option with KSAM
XL. So you cannot read data in a KSAM XL file in chronological order
using FCOPY, or retrieve deleted records in this way.
Programmatic Chronological Access
There are three ways to read KSAM XL files in chronological sequence.
* You can open the file using a call to the FOPEN intrinsic with bit
3:1 of the AOPTIONS set to 1. This is called copy mode and it tells
the file system to access the file as if it were a standard MPE file,
not a KSAM file. This method works with CM KSAM, but not with KSAM
XL.
* You can use FPOINT and FREADC to read the data in chronological
order. However this method will skip over deleted records. It works
with both versions of KSAM.
* You can use FREADDIR to read the data in chronological order. This
method will retrieve all records, including deleted ones. It works
with both versions of KSAM. However, KSAM XL returns deleted records
in a somewhat different format than CM KSAM.
The first method is the one most likely to cause problems when migrating
to KSAM XL from CM KSAM. Here is sample COBOL code that illustrates the
copy mode method.
01 FOPTIONS PIC S9(4) VALUE 2049.
01 AOPTIONS PIC S9(4) VALUE 4096.
.
CALL INTRINSIC "FOPEN" USING MYFILE-NAME, FOPTIONS, AOPTIONS
GIVING FILENUM.
If an application opens a CM KSAM file as shown using copy mode, the
program will read the data portion of a CM KSAM file in chronological
order, including the deleted records. But if you try to use the same
application program with a KSAM XL file, you'll get only the active
records, and you'll get them in key sequence (not chronological order).
The KSAM XL manual states that "KSAM XL does not allow the copy mode
option". This suggests that if you try to use copy mode with KSAM XL
your program will abort. But if we run this program (using copy mode)
against a KSAM XL File, FOPEN does not return an error. Instead the
program ignores copy mode, and returns the data as if we had opened it
normally.
If you want to look at the deleted records of a KSAM XL file, you have to
use the third method listed above. But even then, KSAM XL differs from
CM KSAM. This is because KSAM XL, unlike CM KSAM, does not overlay user
data with a delete flag. (In fact, KSAM XL keeps the delete flag in a
record header not accessible to applications.)