HP 3000 Manuals HP 3000 Manuals
The Hierarchical File System
by Craig Fairchild and Rajesh Lalwani
Commercial Systems Division
Overview
This version of MPE/iX provides hierarchical directory structure. Now
you can create hierarchical directories and place other hierarchical
directories and files under them. MPE/iX 5.0 supports several new file
and record types including the byte stream record type, and pipes, FIFOs,
symbolic link and device link file types. In addition, filenames can
contain some special characters and can be as long as 255 characters.
This article describes these new 5.0 hierarchical file system (HFS)
features.
Hierarchical Directory Structure
The MPE operating system was originally designed primarily for use in a
department setting. All the users in a particular department are
typically grouped together and placed in one account. Similarly, an
account, such as PAYROLL, could organize its files in groups such as
JAN92, FEB92, and so on. In this model, the directory tree consists of
one or more accounts, with each account containing one or more groups.
Finally, each group has zero or more files in it. Hence, this
file.group.account structure can be looked at as a 3-level hierarchical
structure. The first level has account entries, the second level group
entries, and the third level the actual files. POSIX, however, has an
expanded version of MPE hierarchy. The POSIX directory structure has a
root directory, which is represented by a slash (/). The root directory
may have files and directories under it. The directories, in turn, have
files and directories under them. MPE/iX has integrated the MPE and
POSIX concepts resulting in the new HFS. A typical MPE/iX directory could
look like:
/
|
+--------------------------+--------------------------+
| | |
PAYROLL DEVELOP users
| | |
+------+------+ +------+------+-----+------+ +--------+--+
| | | | | | | |
JAN92 FEB92 MPE dir file PX bruna jeff
| | |
+--+--+ +------+------+-----+ +---+---+
| | | | | | | |
HOURS TAXINFO ledger payable data PRGFILE bin addr_bk
| |
+--+---+ +---+-----+
| | | |
main.c Makefile my_ls down_load
On MPE/iX, the root directory may contain files, directories, and
accounts. For most purposes, accounts and groups are treated as
directories but there are differences. For example, groups and accounts
cannot be purged by the PURGEDIR command. Also, groups and accounts
contain some accounting information associated with them which is not the
case with HFS directories.
Like any MPE system, the accounts contain groups. The groups can, in
turn, contain files or directories. In Figure 1, three different flavors
of the integration of MPE and POSIX concepts can be seen. PAYROLL
account has not used any of the new features of POSIX. The account
PAYROLL contains two groups JAN92 and FEB92. The group FEB92 contains
just two files HOURS and TAXINFO. Notice that the files in the group
FEB92 even have valid MPE names. It seems a person familiar with UNIX
has created the directory users under the root directory. This person
has placed two directories, bruna and jeff under users. The directory
jeff has a simple file addr_bk and a directory bin under it. The
directory bin has two script files my_ls and down_load.
The person using the DEVELOP account has really used both MPE and POSIX
concepts. The account DEVELOP has two groups MPE and PX, as well as a
file and a directory. This account is apparently used by an application
developer. There are two directories under the group PX, ledger and
payable, probably for keeping source and make files related to the two
modules of the accounting package. The whole application, after
compilation and linking, has been kept in the file PRGFILE. Apparently,
this package keeps the data in the file data.
Here are the highlights of the MPE/iX directory structure:
* The root directory (represented by a slash /) can have accounts,
directories, or files under it.
* Accounts may have groups, directories, or files under them.
* Groups may have files or directories under them.
* Directories may have files or directories under them.
* Special directories "." and ".." refer to the directory itself,
and the parent of the directory, respectively. The ".." entry of
root (/) is defined to be root (/).
* Only accounts and groups must have valid MPE names. All others
objects can use the portable HFS character set that consists of
A-Z, a-z, 0-9, and three special characters ., _, and -. No name
can start with a hyphen (-), but may start with any other
character. For example, _a_file and 123 are valid HFS syntax
filenames. Also note that these names are case sensitive; AFILE,
Afile, and afile are all unique filenames and can exist at the
same time.
* All names directly below the root, account, and group directories
can be at most 16 characters in length (except account and group
names that must be valid MPE names). All other names can be up to
255 characters long.
* A pathname may be up to 1023 characters long. This implies
a maximum directory depth of 511 levels (for example,
/a/a/a/a/a/a/a/a...). The system does not prevent the creation of
objects whose absolute pathname would exceed 1023 characters, but
may treat these objects in a degraded manner. For example, STORE
will report an error when attempting to access an object whose
absolute pathname is greater than 1023 characters. These objects
can be removed from the system by using the PURGE command and a
relative pathname.
* All file types (except CIR, RIO, and CM KSAM) may be created
anywhere in the HFS directory. Once opened, access to these files
is exactly the same as if they lived in MPE groups. There are no
HFS files, only HFS names.
Current Working Directory
In traditional MPE, the logon group is used to complete unqualified
names. This allows you to type the shorthand name CATALOG rather than
the full CATALOG.PUB.SYS. The logon group was also used as a security
mechanism and for the location to log connect and CPU utilization
information. In release 5.0, the dual function of the logon group has
been separated into two parts; the current working directory (CWD) and
the logon group. The logon group is still used to define group user (GU)
access, and as the accounting point for connect time and CPU utilization.
However, now all name qualification is done based on the CWD.
The CWD is initialized to be the same value as the logon group when you
logon. Unless the CWD is explicitly changed by you (with the CHDIR
command) it will be impossible to tell any difference in behavior in 5.0
and 4.0 systems. The CHDIR command allows you to place the CWD to any
directory that you can access. This means that if you log onto the
PAYROLL account, you can CHDIR /SYS/PUB to set the CWD to the PUB.SYS
group. user can then refer to CATALOG.PUB.SYS as simply CATALOG.
NOTE The CWD does not have any impact on security. It is used as a
naming shorthand only and is not used to determine GU access (which
is still determined by the logon group).
In HFS terminology, a pathname is called absolute if it starts from the
root and specifies all levels until the target object. An example of an
absolute pathname is /SYS/PUB/CATALOG. A pathname can also be relative,
which means that it does not start at the root, but rather from the CWD.
Examples of relative names include CATALOG, ./CATALOG, and sys/notes. In
each of these examples the name was qualified with the CWD location. If
our CWD was /SYS/PUB, then these relative names would be equivalent to
/SYS/PUB/CATALOG, /SYS/PUB/CATALOG (the dot directory refers to the
directory itself, which in this case is the CWD), and /SYS/PUB/sys/notes.
One other property of the CWD that is different from the logon group is
that the CWD is process local. The logon group of a session is identical
for all processes in that session. However, it is possible for each
process in a session to have different CWDs. This means that if a
program is executed and that program changes the CWD to some location, no
other process is affected by that change. If you were to break out of
the program, you would find that the CI's CWD was the same as when the
program was first run.
Multiple Syntax Support
We just saw where the objects such as groups, accounts, files, and
directories can lie within the MPE/iX directory structure. Let us now
see how we can refer to them. MPE/iX has been designed so that the
existing interfaces (such as intrinsics and CI commands) can refer to all
the objects in the directory without changing the way they refer to the
MPE objects such as files in groups. Also, POSIX interfaces such as
POSIX.1 functions and POSIX.2 commands are able to refer to MPE objects
such as accounts and groups and files within them. Conceptually, there
are three name syntaxes: MPE-only, MPE-escape, and HFS.
MPE-only Syntax.
The familiar MPE objects (e.g., files in groups) can be referred
to by using any of the following forms; FILE, FILE.GROUP, or
FILE.GROUP.ACCOUNT. When the system processes MPE-only names, it first
starts by upshifting the name since all MPE names are uppercase only. It
then qualifies the name. When the filename is fully unqualified (say,
FILE), it is qualified with the CWD. If the CWD is not in an MPE group,
an error will result because the MPE-only syntax requires the name to be
MPE syntax expressible. In the second case when the filename is, say
FILE.GROUP the name is qualified to the logon account regardless of the
location of the CWD. Finally, in the third case when the filename is say,
FILE.GROUP.ACCOUNT, the MPE name server refers to the file FILE in group
GROUP in account ACCOUNT, regardless of where the CWD is and what the
logon account is, which is the traditional behavior that would be
expected.
HFS Syntax.
All objects on the sytem, even MPE objects such as accounts, groups, and
files under them can be refered to by the HFS name server. For naming
purposes, the accounts and groups can be treated as directories. Hence,
the file PRGFILE.PX.DEVELOP can be refered to as /DEVELOP/PX/PRGFILE.
When the system processes HFS syntax names, it performs no upshifting
since HFS syntax names are case sensitive. /develop/px/prgfile is NOT
the same file as /DEVELOP/PX/PRGFILE. If the name is relative, then the
system will start its search from the CWD. If the name is absolute, the
path will be searched from the root.
Because the HFS syntax is defined by the POSIX.1 standard, extensions
such as the ability to specify lockwords are not possible when specifying
an HFS syntax name. If the file TAXINFO.FEB92.PAYROLL had a lockword on
it, any attempt to access the file specifying an HFS syntax name will
result in an error. This also means that no prompting of the name will
be done for interactive users when specifying a lockworded file with an
HFS name.
The HFS name syntax is not directly available to the CI commands or any
intrinsic (except HPFOPEN). The MPE-escaped syntax allows the CI and
intrinsics to use the HFS syntax (see below). The POSIX.2 shell
(SH.HPBIN.SYS) accepts only HFS syntax filenames, as well as all of the
POSIX.1 functions.
MPE-escaped Syntax.
The MPE-escaped syntax is the default syntax used by the CI and the file
system. It is a hybrid of MPE-only and HFS syntax that allows
compatibility with the traditional MPE-only syntax as well as the
flexibility of the new HFS syntax. By default all MPE-escaped syntax
names are assumed to be MPE syntax and are treated that way. The names
will be upshifted, and then qualified in the same manner as MPE-only
syntax. The only difference, in this case, is that is legal for an
unqualified filename to be in a CWD that is not a group. This allows a
user to open a file TEXT in the CWD, regardless of where that CWD is.
The main difference between MPE-only and MPE-escaped syntax is in the
ability to switch, or escape, to HFS syntax. If the first character in
an MPE-escaped name is either a dot (.) or a slash (/), then the system
will assume that the name is an HFS syntax name. This can be done since
both dot and slash are invalid first characters in an MPE syntax name.
When an MPE-escaped name is escaped into the HFS syntax, the system
processes the name according to the HFS syntax rules. No upshifting is
done to preserve the case sensitivity, and the special characters dot
(.), dash (-), and underscore (_) are valid to be in the name as well.
An example of using the MPE-escaped syntax would be in purging the file
/DEVELOP/PX/data. Since the data file is in lower case, an MPE syntax
name cannot name the file. If our CWD is /DEVELOP/PX, then either of
these two commands would have the same effect:
:PURGE ./data
or
:PURGE /DEVELOP/PX/data
Note that the use of the leading dot in the example above is only
necessary to escape the name into the HFS syntax. If you ran the POSIX.2
shell, which uses HFS syntax - not MPE-escaped syntax like the CI, you
would not need to provide the leading "./" characters.
Creating/Purging Hierarchical Directories
Hierarchical directories can be created by the intrinsic HPFOPEN, CI
command NEWDIR (refer to the article "New CI Commands" in this
Communicator) or by POSIX.1 and POSIX.2 interfaces.
The disk space used by hierarchical directories that are somewhere below
an MPE group (like ledger and the files main.c and Makefile), does add up
to the group and account that they are below. However, for directories
that are not below an MPE group (like users or jeff), system managers
cannot put a limit on the disk space that they can take up. However,
only users with SM capability can create a directory directly below the
root directory. Also, a new CI command DISKUSE can be used to see the
disk space used by various directories.
A hierarchical directory cannot be purged unless it is empty. A
directory is considered empty when it only has the dot (.) and dot-dot
(..) entries in it. A new CI command PURGEDIR has an option ;TREE that
will purge the files/directories under the directory recursively and
finally purge the requested directory.
New File Types
This version of MPE/iX provides some new file types. Here is a brief
definition of them. Further information can be obtained from the POSIX
Reference from IEEE or the MPE/iX manual set. Since these interfaces are
based on a commonly used standard, many textbooks are available that
contain a detailed discussion of their use.
Pipes.
A pipe consists of two file descriptors connected such that data written
to one can be read from the other in a first-in-first-out manner.
FIFOs.
A FIFO is a named pipe. Because a FIFO has a directory name, it can be
used by non-related processes to perform interprocess communication.
Symbolic Links.
A symbolic link allows for a directory entry to substitute a different
directory name to search for. This is similar to having a file equation
(for example, :FILE FILE1=FILE2) that lives permanently. In the example
just given any reference to the name "FILE1" actually opens the file
"FILE2". A symbolic link performs the same function.
Device Links.
A device link associates a directory entry with a logical device number.
It is conceptually similar to issuing a file equation such as :FILE
TAPE;DEV=7. Whenever the device link file is opened, the open is
actually performed on the logical device associated with the device link.
Byte-Stream Files.
With MPE/iX Release 4.5 and subsequent releases, MPE/iX has a new record
type: byte-stream. Byte-stream files are simply a sequence (stream) of
bytes, hence the name byte-stream. The term byte-stream file is
frequently used when talking about files with the byte-stream record
type, even though they are not a new file type. Byte-stream files do not
have any record structure associated with them. One can build a
byte-stream file as follows:
BUILD filename;rec=,,B;disc=100000
Notice that you do not have to specify the record size. This is because
byte-stream files have a record size of 1 byte. B means byte-stream in
the same way as F means Fixed length record file. Also, since a
byte-stream file is implemented with record length of one byte, disc=
parameter would indeed mean the number of bytes in the file.
By convention, applications use the linefeed character (ASCII 10) to
separate records in byte-stream files. Byte-stream files also allow
reading and writing of data in arbitrary chunks. In order for an
application to read logical records, it must read a chunk into a buffer
and scan for the linefeed record separator. This buffering can be
complicated and can also be handled for the application by C Library
routines. Most traditional applications, however, are not written in C
and would not operate correctly with true byte-stream access.
So that the current applications can work correctly with byte-stream
files, intrinsics such as FOPEN and HPFOPEN will bind a Variable Record
Emulator type manager with a byte-stream file by default. In such a case
the byte-stream file behaves like a variable record file with the
linefeed character acting as the end of the record marker. If the
application wants to use the file as a byte-stream file, it needs to
specify HPFOPEN option 77 as 2 while calling HPFOPEN.
Miscellaneous
* With this release of MPE/iX, users with SM capability can create
and rename files across account boundaries
* You no longer have to be the creator of a file in order to :RENAME
it. If you have read access to the file, can purge the old file,
and can create the new file, then you can :RENAME the file.
* The file label will have several new fields such as file_group,
state_chg_time, hd_link_count. Also it is possible that the
file_name field will be empty for some files because the filenames
can now be 255 characters in length. Finally, the creator field
can contain a fully qualified name of the creator because the
files can now be created by users from a different account.