HPlogo   Communicator 3000 MPE/iX Release 6.0 (Platform Software Release C.60.00):
HP 3000 MPE/iX Computer Systems > Chapter 10 Technical Articles

10.8: Transaction Manager

MPE documents

Complete PDF

 

Table of Contents

Index

 

⇓ Page Bottom

⇑ Page Top

  

The REMSH Client Functionality

New Features for DTS/TIO Subsystem

by Nagarajan and Jyothi
Commercial Systems Division

Overview

Transaction Manager (XM) is an integral part of the MPE/iX file system. It provides a facility to log all the transactions that are done on a file that is attached to XM via a system intrinsic. It ensures data integrity by protecting such files across system crashes.

Files on MPE/iX are of two categories: system files and user files. System files are label tables, directories and SSM bitmaps. User files are those created by the user. All the transactions on system files are logged onto an XM system log file on the master volume of each volume set, just as all the transactions on user files (such as IMAGE/SQL and KSAM files) are logged onto an XM user log file on the master volume.

Both these XM log files are (magic) system files and are created at volume set creation time with a fixed size.

A System Abort 2200 is generated from Transaction Management (XM) subsystem when it cannot reliably continue logging data to the system XM log file. The system XM log file is a 12Mb object allocated contiguously on disk. XM divides this into two portions called log halves, each being 6Mb in size. While data is being logged into one half, the other half is being emptied. The emptying happens when the transactions are committed and the log half is posted to disk. If we fill one half and discover that the other half is not yet empty, XM will generate SA 2200 in order to preserve the data still in the second log half. This scheme ensures data integrity by not allowing uncommitted data to be overwritten.

Typical long-running system transactions are those that involve insertion of a file into a group/directory at the beginning, initializing file contents at creation time, and others.

System Log Expansion

As the higher-end models of HP3000 become faster and faster, the frequency of occurrence of SA 2200 has been going up. To alleviate the severity of this problem, a new solution to expand the XM system log per volume set is being provided. This will allow more logging to be done before the log gets full.

Commands to Expand

The VOLUTIL tool is enhanced to provide two new commands to facilitate the expansion (or truncation after initial expansion) of the XM system log file. They are as follows:
  1. showsyslog <volset-name>

    This command accepts a valid MPE volume set name and shows the current size of the XM system log in sectors. For this to work correctly, the master volume has to be mounted in MASTER state. The default size is 52128 sectors allocated at volset creation time.

  2. altersyslog <volset-name> <new size>

    This command accepts the volume set name and the new size of XM syslog for it in sectors. It cannot be less than 52128 sectors and cannot be more than 2 GB. This is a very large upper limit but as disk capacities are becoming more and more, one can go for a size of about 100Mbytes. One caution is that LDEV 1 has limited reserved space and you cannot expand the XM syslog beyond that. Also, the two residual logs on the master also will be expanded to match the size of the system log.

    The downside of having a larger system log is that the XM checkpoint process will have more data to write to disk at reaching the end of each syslog half. This can have an effect on response time to on-line users.

Actual Expansion

The actual expansion of the XM system log happens when user boots the system next time. This is because the XM log files cannot be expanded/truncated while in use.

During bootup, just after recovery of the system volset, the XM system log file will be expanded and a message will be written onto the console. Also, the two residual logs that reside on the master will be expanded to match it.

For the user volume sets, right after the recovery of system transactions, the expansion of the system log happens.

You can also truncate the system log after expansion if there are any disk space or performance issues. The VOLUTIL interface is same for this operation too.

The residual logs will not be expanded until they are emptied, that is, until all the members of the given volume set are mounted in the MEMBER state.

Backdating the OS

You can backdate the OS to an earlier version with the system log in the expanded state. The only condition is that the residual logs also must have been expanded or truncated to match it before the backdating is done. That would mean a complete, proper recovery of the volume set in question has to be done before the backdate operation.

Warnings

The expansion of the system log or the residual logs may fail due to one or more reasons. In that case, the system will report it through a warning message (in reverse video) on the console.

If it fails due to lack of disk space, make sure that there is sufficient contiguous space on the master that would fit the need for the increment in the system log size. Then reboot the system so as to enable it to complete the task of matching the sizes of residual logs with that of XM system log.

Transaction Copy-forward

The Copy-Forward solution eliminates the frequent occurrence of SA 2200 by copying forward the activity records of slow system transactions into the next log half and continuing logging in the current log half.

This is an enhancement to FOS and there is no user interface needed to make this available.

Additional System Requirements

To accomplish the task of copying forward the slow transactions, a new system file called the copyfwd buffer is created on the master of each volume set. This is done at the time of creating the volume set or at the first time of booting the system with this version of MPE/iX.

The size of this new buffer is half of the XM system log (which is about 12 MBytes).

When It is Activated

When the XM logger sees slow system transactions at the time of wrapping back to the start of the circular XM system log file, it activates this solution to copy forward all such slow transactions first into the copyfwd buffer, sorts them and then continues with logging of the current transaction. This copying forward is done on such slow transactions for a certain number of times (about 100 times, as of now), and then, if they are not over even then, it would issue SA 2200, concluding that there is something that can put the system in jeopardy.

Backdating the OS

While XM logger is doing copyfwd operation, it temporarily elevates the version of the XM system log to a higher value and brings it back to its original value once copy forward is over. But if the system crashes while the XM syslog version is at a higher level, backdating it to an earlier version of the OS wouldn't be possible. The system gives an error and XM recovery fails. So in such situations, you need to complete the recovery and then go for backdating the OS.

Checkpoint Improvement

Checkpoint utility is part of XM, and it is invoked when one-half of the XM log is full. It writes out all the dirty data pages from that half to the disk. During this activity, the system on-line response sometimes is impacted due to the high priority nature of its disk I/O and also due to the fact that you can checkpoint only one-half at a time. This makes the logging slow down if the checkpoint process is slow or it holds up critical system resources for too long. This becomes critical on high end MP machines. This solution attempts to improve the performance of XM checkpoint process so that on-line response becomes better and transaction throughput increases.

This solution achieves an improvement in performance by not holding one of the system-wide semaphores for too long while it's scanning the entire virtual address range (could be 4 GB) of a modified file. It remembers the dirty page ranges for each modified file in the log half being checked and does posting to disk only for those ranges.

This solution affects only those files that are attached to XM user log, typically IMAGE/SQL and KSAM files. It has no effect on system files such as directories and label tables.

Enabling or Disabling It

The VOLUTIL tool is enhanced to provide two new commands to enable or disable this solution for files on a given volume set. By default this solution is disabled, which means the checkpoint process will scan the entire file VA range for dirty pages.

The commands are as follows:
  1. showchkptstat <volset-name>

    This command shows the current state of the solution, that is, whether it is enabled or disabled for the given volume set.

  2. alterchkptstat <volset-name> ENABLE | DISABLE

    This command enables or disables the solution for the given volume set. If it's already in the given state, it will give a warning to that effect.
You need SM capabilities to use these two commands.

Since this solution remembers the dirty page ranges in a data structure associated with each open file, it can't affect files that were open before giving this request. However, it will come into effect for the files that were open (for the first time) after giving the request. So the best way to enable or disable the Checkpoint Improvement is to reboot the machine after giving the command.

Backdating the OS

You can backdate the OS to an earlier version without any problems since this solution doesn't put the file system in an advanced version or make any non-backward compatible changes to the file system.


The REMSH Client Functionality

New Features for DTS/TIO Subsystem