HP 3000 Manuals HP 3000 Manuals
Chapter 3 System Management
Introducing New Workgroups on the HP 3000
by Shelley Nelson and Susan Campbell
Commercial Systems Division
The introduction of the Workload Manager (product number B3879AA) with
Release 5.0 of MPE/iX gives HP 3000 customers the opportunity to more
closely monitor and control CPU scheduling. In this release, the
traditional five scheduling subqueues (AS, BS, CS, DS, and ES) are
replaced by five system-defined workgroups and users who purchase the
Workload Manager can create additional user-defined workgroups for more
discrete partitioning of the system workload.
This article briefly reviews traditional CPU scheduling, introduces
workgroups, and describes the new and modified MPE/iX commands used in
workload management. It is intended for two audiences:
* Customers who won't use the Workload Manager and who want to know
how they will be affected by the introduction of workgroups on the
HP 3000
* Customers who will purchase the Workload Manager and who want a
general, largely conceptual, introduction to the product
Three other articles in this Communicator may be helpful. The "Workload
Manager Technical Overview" provides more detailed information about the
features, benefits and sample uses of the Workload Manager. The articles
"CI Commands for the Workload Manager" and "AIF Enhancements" discuss
changes to the CI and AIFs, some of which are due to the Workload
Manager.
Traditional CPU Scheduling
In traditional CPU scheduling (prior to MPE/iX Release 5.0), system and
user processes run in one of five scheduling subqueues: the AS subqueue,
BS subqueue, CS subqueue, DS subqueue and ES subqueue. In this scheme,
scheduling subqueue is both an entity and a process attribute.
The scheduling subqueue as an entity has defined characteristics (such as
base and limit, the minimum and maximum quantum and the boost property)
that determine the scheduling policies of processes running in that
subqueue. In other words, you can think of the scheduling subqueue as a
dynamically changing collection of processes. The scheduling subqueue is
a process attribute when, for example, a job is described as "having a
priority of CS" or, for example, when "a program is running at DM
priority".
NOTE Most users do not distinguish between the scheduling subqueue as an
entity and as a process attribute, for the simple reason that it
hasn't been necessary. However, understanding this difference can
facilitate your understanding of the workgroup concept.
Users with system supervisor (OP) or system manager (SM) capability can
affect the behavior of the scheduling subqueues as entities. Such users
may alter, or tune, the characteristics of the CS, DS and ES scheduling
subqueues. (These subqueues, which collect user processes, are also
known as the circular scheduling subqueues.) No users, regardless of
capability, can alter the characteristics of the AS and BS subqueues,
which are reserved for system processes.
Users without special capabilities can influence the scheduling subqueue
as a process attribute through one of the following means:
* Users can choose a scheduling subqueue via the ;PRI= option of the
HELLO, RUN or JOB commands. Such a scheduling subqueue change
lasts for the duration of the session, program, or job.
Furthermore, the system manager constrains the priority that a
user can assign to a process (via the MAXPRI option of the
NEWUSER, ALTUSER, NEWACCT, or ALTACCT commands).
* Users can change the scheduling subqueue of a process via the
;PRI= option of the ALTPROC command. Such a scheduling subqueue
change lasts until the process terminates (or until the subqueue
is changed again). A process forced to a workgroup in this manner
is termed an artificial member rather than a natural member
* Programmers or system managers can use one of two Link Editor
options to determine a program's scheduling subqueue when they are
creating an executable program file: the priority_level, to
specify the default execution priority at run time or
max_priority_level, to specify the maximum priority a program can
have at run time. Within this limit, users can alter the
program's priority by issuing the ;PRI= option of the RUN command.
* Programmers can use either the AIFPROCPUT routine or the
GETPRIORITY intrinsic to change the scheduling subqueue of a
process.
With Release 5.0 of MPE/iX, scheduling subqueues as entities will be
replaced by workgroups. Scheduling subqueue remains a process attribute
(users, within the limits of their capabilities, can still assign a
process to a specific subqueue).
Introducing Workgroups
You can think of a workgroup, like its predecessor the scheduling
subqueue, as a dynamically changing collection of system or user
processes. A workgroup consists of three components: the workgroup
name, a set of membership criteria, and specific scheduling
characteristics.
On all systems updated to Release 5.0 of MPE/iX, there will be five
system-defined workgroups, each of which corresponds to one of the five
traditional scheduling subqueues. The AS subqueue, for example, becomes
the AS_Default workgroup, but it retains its former scheduling
characteristics. For customers who don't purchase the Workload Manager,
the most noticeable difference is simply one of nomenclature. In
virtually all other respects, the use and behavior of system-defined
workgroups is the same as scheduling subqueues.
A Comparison of Scheduling Queues and System-Defined Workgroups
------------------------------------------------------------------------------------------------
| | | | |
| Queue | Corresponding Workgroup | Membership criteria | Scheduling |
| | name | | characteristics |
| | | | |
------------------------------------------------------------------------------------------------
| | | | |
| AS | AS_Default | MEMB_QUEUE=(AS) | Uses default values |
| | | | |
------------------------------------------------------------------------------------------------
| | | | |
| BS | BS_Default | MEMB_QUEUE=(BS) | Uses default values |
| | | | |
------------------------------------------------------------------------------------------------
| | | | |
| CS | CS_Default | MEMB_QUEUE=(CS) | Uses default values; |
| | | | allows system manager to |
| | | | make changes. |
| | | | |
------------------------------------------------------------------------------------------------
| | | | |
| DS | DS_Default | MEMB_QUEUE=(DS) | Uses default values; |
| | | | allows system manager to |
| | | | make changes. |
| | | | |
------------------------------------------------------------------------------------------------
| | | | |
| ES | ES_Default | MEMB_QUEUE=(ES) | Uses default values; |
| | | | allows system manager to |
| | | | make changes. |
| | | | |
------------------------------------------------------------------------------------------------
Each of the system-defined workgroups have one and only one criterion for
membership: the scheduling subqueue. This assures complete backward
compatibility with previous versions of the operating system. All user
processes will fall into one of these system-defined workgroups using the
customary scheduling parameters and all existing programs and job scripts
will continue to work. (By contrast, system managers who use the
Workload Manager can create user-defined workgroups whose membership is
based on three different criteria. For more information, read "Creating
User-Defined Workgroups" later in this article.)
NOTE The introduction of workgroups warranted changes in the process
scheduling algorithms and the default timeslice values for the
DS_Default and ES_Default workgroups. From an end-user
perspective, these changes will not alter process scheduling. As
system manager, these changes allow you more flexible control of
priority decay. Read "Process Scheduling Changes", later in this
article, for more information.
Displaying Workgroup Information.
To display CPU scheduling information, you now have a choice of two
commands: SHOWQ or, one of the new commands introduced for the Workload
Manager, SHOWWG. The differences between the two commands include:
* SHOWQ displays scheduling characteristics of the CS, DS and ES
subqueues only, and the format of the display has not changed.
Below is an example of the STATUS display of the SHOWQ command;
note that the left-most column is still labeled QUEUE.
:SHOWQ;STATUS
------QUANTUM------
QUEUE BASE LIMIT MIN MAX ACTUAL BOOST TIMESLICE
----- ---- ----- --- --- ------ ----- ---------
CQ 152 200 l 2000 576 DECAY 200
DQ 202 238 2000 2000 2000 OSC 200
EQ 240 253 2000 2000 2000 DECAY 200
* SHOWWG displays scheduling characteristics for all workgroups on
the system (including, for those customers using the Workload
Manager, any user-defined workgroups you create). Below is an
example of the SUMMARY format of the SHOWWG command on a system
without user-defined workgroups. Note that the traditional
scheduling subqueues are listed in the WORKGROUP column under
their system-defined names. (Whereas CQ appeared in the SHOWQ
command, here it appears as the CS_Default workgroup.)
:SHOWWG @; FORMAT=SUMMARY
-----QUANTUM----- TIME CPU %
WORKGROUP BASE LIMIT MIN MAX ACTUAL BOOST SLICE PROCS MIN MAX
------------------- ---- ----- ----- ---- ------ ----- ------ ----- --- ---
AS_Default 13 99 N/A N/A N/A N/A 1000 45 N/A N/A
BS_Default 100 150 N/A N/A N/A N/A 1000 73 N/A N/A
CS_Default 152 200 1 2000 576 DECAY 200 320 N/A N/A
DS_Default 202 238 2000 2000 2000 DECAY 200 217 N/A N/A
ES_Default 240 253 2000 2000 2000 DECAY 200 81 N/A N/A
NOTE Workload Manager users should use SHOWWG instead of SHOWQ to see
complete information. Any user-defined workgroups on the system
precede the system-defined workgroups in the SUMMARY format.
Tuning a Workgroup.
To alter the scheduling characteristics of system-defined workgroups,
system managers can choose between the TUNE command and ALTWG, another
new command introduced for the Workload Manager.
Use TUNE to alter more than one system-defined workgroup, since the
command allows you to specify one, two, or all three subqueues. Use
ALTWG under the following circumstances:
* To change the scheduling characteristics of a user-defined
workgroup. In this circumstance, Workload Manager users must use
ALTWG since TUNE only modifies three of the system-defined
workgroups.
* To alter a single system-defined workgroup, whether or not you are
using the Workload Manager. ALTWG only examines member processes
of the specified workgroup rather than all processes in all
workgroups on the system, which means greater efficiency.
For example, to use the TUNE command to set the CS subqueue's base to
152, limit to 300, and max quantum (filter) to 300; enter:
:TUNE CQ=152,200,300,300
To use the ALTWG command to set the same scheduling characteristics for
the CS_Default workgroup, enter:
:ALTWG CS_Default, 152,200,300,300
Process Scheduling Changes
In order to provide support for workgroups, changes were made to some of
the scheduling characteristics for the DS_Default and ES_Default
workgroups. New flexibility is available to control priority decay and
minor adjustments have been made to the preemption algorithm and default
timeslice values.
The following information describes these changes, pointing out areas
where the system manager can make changes, if desired.
Priority Decay.
Whereas the DS and ES scheduling subqueues had a fixed quantum value, the
DS_Default and ES_Default workgroups perform a dynamic quantum
calculation. The default values for the minimum and maximum quantum
remain equal to ensure that the quantum will not vary unless the system
manager chooses to change the bounds. Each workgroup uses the quantum to
derive the amount of priority decay that will be applied to each process.
Since Release 3.0, the CS scheduling subqueue has made use of a priority
decay algorithm that provided a wider distribution of priority values
(decaying by small amounts over time, rather than in large jumps.) This
change to a more granular priority decay led to the introduction of the
preemption threshold to avoid increasing the number of preemptions. A
similar decay algorithm has been applied to all workgroups (including the
CS_Default, DS_Default and ES_Default). The algorithm bases priority
decay on the quantum divided by a constant, applying smaller amounts of
priority decay for smaller amounts of CPU consumption.
Preemption.
Prior to Release 5.0, the preemption policy was partially based on a
comparison of the subqueue attributes of the running process and the
potential preemptor. This was due to the assumption that a higher
subqueue indicated a more important process. With the introduction of
workgroups, it was necessary to simplify the preemption algorithm,
removing this dependency on scheduling subqueue attribute.
The algorithm continues to compare priorities, only considering
preemption if the priority of the potential preemptor exceeds the
priority of the running process by the preemption threshold. If system
and boosted processes are able to pass these priority checks, they can
preempt the running process (if it is preemptable).
Timeslice.
Due to the change in the preemption algorithm, the default timeslice
values for the DS_Default and ES_Default workgroups required
modification. Previously, CS subqueue processes had always been able to
preempt DS or ES subqueue processes (since they had a higher subqueue),
and thus the timeslice values for the DS and ES subqueue were set high.
These default values have now been reduced so that processes in these
workgroups cannot monopolize the CPU for long periods of time. The
timeslice value remains tuneable, however, and the system manager can
adjust it to a higher value if it is appropriate to the environment.
Overview of the Workload Manager
The Workload Manager gives the system manager more control over CPU
scheduling by allowing the creation of user-defined workgroups. Using
the Workload Manager, the system manager determines:
* The total number of workgroups on the system
* Which processes become members of each workgroup by specifying up
to three different kinds of membership criteria
* The behavior of processes in each workgroup by assigning
scheduling characteristics
* The current workgroup configuration
The system manager can also implement a new workgroup configuration on
the fly or at a predetermined time (for example, just before a shift
change or to handle scheduled batch jobs) to optimize system performance.
The system can be partitioned into as many workgroups as needed to
control the workload. The workgroups can correspond to a particular set
of users, such as those in one department, or to a group of people
performing the same kind of task, such as updating a common database.
This greatly improves granularity in workload partitioning and, since the
system manager can observe individual workgroups, allows close monitoring
of system performance.
The Workload Manager includes a set of CI commands to create, alter and
purge workgroups and to display workgroup information. The AIF:OS
package includes a set of AIFs to allow programmatic access to these
capabilities. These capabilities provide a basic foundation that can be
used by performance tools such as HP GlancePlus and third party
offerings, system management tools such as OpenView Console, and more
sophisticated Graphical User Interfaces (GUIs).
Creating User-Defined Workgroups
User-defined workgroups consist of a workgroup name, membership criteria,
and scheduling characteristics. The name uniquely identifies the
workgroup and typically indicates the kinds of processes which become
members of the workgroup, such as Program_Development or Payroll_Batch.
Workgroup Membership Criteria.
Each process is assigned to its appropriate workgroup at process
creation, whenever a process attribute is changed, or after it is
explicitly assigned to a workgroup. Membership criteria in the
user-defined workgroup include any one or all of the following:
* Logon identity, of the form job/session name,user.account.
* Program name.
* Queue attribute, which is one of AS, BS, CS, DS, or ES.
The subqueue attribute is not the same as the scheduling subqueue. The
attribute is one element used to determine membership in a workgroup
whereas the traditional scheduling subqueue, as an entity, is comparable
to the workgroup.
Workgroup Scheduling Characteristics.
The scheduling characteristics the system manager can assign to a
workgroup include the following:
* Base and limit priorities
* Quantum bounds
* Boost property
* Timeslice
In addition, system managers using the Workload Manager can assign
minimum and maximum CPU bounds to user-defined workgroups. For more
information about each of these characteristics, and sample uses, refer
to the article "Workload Manager Technical Overview" in the "Technical
Articles" chapter.
New and Modified CI Commands.
The Workload Manager includes four new commands and changes to four
existing commands. The table below describes the availability and
function of each of the commands for customers who purchase the Workload
Manager and for those who do not.
The Workload Manager and CPU Scheduling Commands
-----------------------------------------------------------------------------------------------
| | | |
| Command | Behavior with the Workload Manager | Behavior without the Workload |
| | | Manager |
| | | |
-----------------------------------------------------------------------------------------------
| | | |
| ALTPROC | Use the new ;WG= option to | You may use the new ;WG= option to |
| | explicitly place a process in a | explicitly place a process in one |
| | workgroup or allow a process to | of the system-defined workgroups |
| | migrate to its natural workgroup. | or allow a process to migrate to |
| | All other command functionality is | its natural workgroup. All other |
| | unchanged. | command functionality is |
| | | unchanged. |
| | | |
-----------------------------------------------------------------------------------------------
| | | |
| ALTWG | Use this new command to alter only | You may use this command in place |
| | the scheduling characteristics of | of TUNE to alter only the |
| | an existing workgroup. (Use NEWWG | scheduling characteristics of |
| | to alter name or membership | three of the system-defined |
| | criteria.) | workgroups. You may use all |
| | | parameters except minpercent and |
| | | maxpercent, which are not |
| | | applicable to the five system |
| | | defined workgroups. |
| | | |
-----------------------------------------------------------------------------------------------
| | | |
| NEWWG | Use this new command to create one | This command is unavailable. |
| | or many new, user-defined | |
| | workgroups or to validate the | |
| | syntax of an indirect file. | |
| | | |
-----------------------------------------------------------------------------------------------
| | | |
| PURGEWG | Use this command to purge the | This command is available, but it |
| | specified workgroups or to execute | generates an error message since |
| | all deferred purgescans. | purging the system-defined |
| | | workgroups is not allowed. |
| | | |
-----------------------------------------------------------------------------------------------
| | | |
| SHOWPROC | The DETAIL display format of this | The DETAIL display format of this |
| | command has been modified to show | command has been modified to show |
| | the process' workgroup and | the process' workgroup and |
| | attributes determining workgroup | attributes determining workgroup |
| | membership. | membership. |
| | | |
-----------------------------------------------------------------------------------------------
Table 3-0. The Workload Manager and CPU Scheduling Commands (cont.)
-----------------------------------------------------------------------------------------------
| | | |
| Command | Behavior with the Workload Manager | Behavior without the Workload |
| | | Manager |
| | | |
-----------------------------------------------------------------------------------------------
| | | |
| SHOWQ | Use this command to display | Same behavior with or without the |
| | scheduling data for all processes | Workload Manager. |
| | and the scheduling characteristics | |
| | of the CS_Default, DS_Default and | |
| | ES_Default workgroups. This | |
| | command has not been changed and | |
| | the three system-defined | |
| | workgroups appear as traditional | |
| | scheduling subqueues in the | |
| | left-most column of the STATUS | |
| | display. | |
| | | |
-----------------------------------------------------------------------------------------------
| | | |
| SHOWWG | Use this new command to display | Use SHOWWG in place of SHOWQ to |
| | scheduling and process data for | see scheduling characteristics of |
| | the specified workgroup(s), and to | all five system-defined |
| | create the indirect file used as | workgroups. |
| | input for the NEWWG command. | |
| | | |
-----------------------------------------------------------------------------------------------
| | | |
| TUNE | Use ALTWG in place of TUNE to | Use TUNE to change the scheduling |
| | change the scheduling | characteristics of CS_Default, |
| | characteristics of a workgroup. | DS_Default and ES_Default |
| | | workgroups. The function and |
| | | syntax of the command is |
| | | unchanged. |
| | | |
-----------------------------------------------------------------------------------------------