HP 3000 Manuals HP 3000 Manuals
Numbers
There are three number types in HP ALLBASE/BRW:
* Integer
* Fixed-point Numeric
* Floating-point Real
For each type you can:
* use all the arithmetic operators
* use the standard function DIGIT_OF
Additionally, you can use functions that only apply to Real numbers
(square root, etc). Numeric and Integer types are covered first, and
then Real numbers. It is more efficient to use Numeric types than Real
types.
The range of each type of number is shown below.
-------------------------------------------------------------------------------------------
| |
| Type Range and Description |
| |
-------------------------------------------------------------------------------------------
| |
| Fixed-point Numeric (N) +/- 9999999999999.999999 |
| |
| That is, 13 digits before the decimal point |
| and 6 decimal places. The number of |
| decimal places of a Numeric item is taken, |
| as far as possible, from the number of |
| places in the data dictionary definition. |
| |
| Integer (I) +/-999999999 |
| |
| No decimals are allowed for Integer types. |
| |
| Floating-point Real (R) Between 10 to the power -77 and 10 to the |
| power 76 |
| |
| Floating-point Real is accurate to 16 |
| decimal places. |
| |
-------------------------------------------------------------------------------------------
Operations on Numbers
Operations on numbers can include arithmetic operations and functions.
You can control calculation precision by selecting the appropriate item
type (Numeric, Real, or Integer) and by changing the precision of the
calculation result. You can change precision at several stages in a
report.
Arithmetic Operators
Arithmetic operators include:
-------------------------------------------------------------------------------------------
| |
| Operator Description Example |
| |
-------------------------------------------------------------------------------------------
| |
| + Addition 5 + 4 = 9 |
| |
| - Subtraction 5 - 4 = 1 |
| |
| * Multiplication 5 * 4 = 20 |
| |
| / Division 5.0 / 4 = 1.25 |
| |
| ** Exponentiation 5 ** 4 = 625 |
| |
| MOD Modulo 5 MOD 4 = 1 |
| |
-------------------------------------------------------------------------------------------
The following are some notes on arithmetic in HP ALLBASE/BRW:
* Integer Division. If both arguments are integers HP ALLBASE/BRW
uses Integer division and the result is an Integer. Using Integer
division, the result of the division (5 / 4) would be 1.
* Real or Integer Exponentiation (**). If you use the
exponentiation function (**), HP ALLBASE/BRW chooses whether to
use Real or Integer exponentiation depending on the values of the
mantissa and exponent. If the mantissa is Real or if the exponent
is not an Integer, Real computation is used. Otherwise Integer
computation is used.
For example:
2 ** 3 (=8) Exponent and mantissa are both Integer,
therefore Integer computation is used.
2 ** 3.5 (=11.3137) The exponent not an Integer, therefore
Real computation is used.
3.5 ** 2 (=12.25) The mantissa is Real, therefore Real
computation is used.
* Modulo (MOD) Calculation. The modulo (MOD) function is calculated
according to the following formula:
X mod Y = X - (Y * INT(X / Y))
where INT(Z) returns the largest Integer less than or equal to Z.
For example:
13 MOD -2 = 13 - (-2 * INT(13 / -2))
= 13 - (-2 * -7) = -1
-13 MOD -2 = -13 - (-2 * INT(-13 / -2))
= -13 - (-2 * 6) = -1
13 MOD .5E2 = 13 - (.5E2 * INT(13 / .5E2))
= 13 - (.5E2 * 0) = 13
13 MOD 2 = 13 - 2 * INT ((3 / 2 ))
= 13 - 2 * 6 = 1
Precision of Numbers
Calculation precision in a calculation can be controlled by any of these
options:
* selecting the appropriate item type (Numeric, Real, or Integer)
* optionally changing the precision of the calculation result
* changing the precision of arguments used in the calculation
formula
You can change the calculation precision when:
* there is a result of a table calculated item, defined on the Table
Calc Item screen
* an item is projected from a table via the Project Item screen
* there is a result of a layout calculated item, defined on the
Layout Calc Item screen
* an item is used in a lineset, defined on the Define Lines screen
* there is a result of a column calculation, defined on the Define
Lines screen
* an item is to be printed and is defined using scaling/decimals
All parts of the report performed after you set a precision will use the
item with that precision.
NOTE The screens that control calculation precision are listed
alphabetically in Chapter 16 .
Defining Precision
You define precision for an item by specifying a number and method code.
The number is a power of 10, or 10th exponent, that indicates the
precision. The method code indicates how superfluous digits are to be
treated.
------------------------------------------------------------------------------------------
| |
| Valid range for Numeric (type N): -6 through 12; |
| |
| for Integer (type I): 0 through 8 |
| |
| for Real (type R): -90 through 76. |
| |
------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------
| |
| Method Codes: Digits are: |
| |
-------------------------------------------------------------------------------------------
| |
| R Rounded |
| |
| T Truncated |
| |
| C Ceiled |
| |
-------------------------------------------------------------------------------------------
If you round a number, it is rounded up if the first digit outside your
precision is a 5 or greater, and rounded down if the first digit outside
your precision is less than 5. (You can choose, on the BRWSETUP program,
whether you want 5 to round up or down.) If you ceil a number, it will
always be rounded up. See the "Real Numbers" description in the
"Numbers" section of this chapter for information on real numbers and
precision.
Example
On the Table Calc Item screen below, the table calculated item TAX-AMOUNT
is specified with a numeric precision of -2T.
![[]](../../pic3k/M011795/FFN1683c50.gif)
Then, on the Table Calc Item Formula screen for TAX- AMOUNT, the
following formula is entered:
AMOUNT * TAX-RATE
In this example, if the value of AMOUNT is $416.95, and the value of
TAX-RATE is 0.07, then the calculation result $29.1865 is truncated to
$29.18. Note that the precision applies only to the result. The
computation is made at full precision, and the truncation performed AFTER
the computation. Note also that, with a precision of -2T, item
TAX-AMOUNT can never have more than two decimals.
The following table shows the value of TAX-AMOUNT (29.1865) with various
precisions to go with the above example.
------------------------------------------------------------------------------------------
| |
| Had you specified the result |
| a precision of: would be: |
| |
------------------------------------------------------------------------------------------
| |
| -5C 29.18650 |
| |
| -4T 29.1865 |
| |
| -3T 29.186 |
| |
| -1T 29.1 |
| |
| -1R 29.2 |
| |
| 0C 30 |
| |
| 1R 30 |
| |
| 1T 20 |
| |
| 2T 0 |
| |
| 2R 0 |
| |
| 3R 0 |
| |
| 3C 1000 |
| |
| 6C 1000000 |
| |
------------------------------------------------------------------------------------------
Digit Extraction Function
This function is used to extract single digits from a number.[REV BEG]
The function uses a digit index to determine which digit to extract. The
index looks like this:
. . . 9 8 6 5 4 3 2 1 0 -1 -2 -3 -4 . . .
The blank between 0 and -1 corresponds to the decimal point. When you
use this function in a calculation, you specify the number and the digit
index like this:
DIGIT_OF (1024.38, 1)
where 1 is the digit index.
In this example, the index is applied to the number as shown here:
1 0 2 4 . 3 8
downarrow downarrow downarrow downarrow downarrow downarrow downarrow
. . .6 5 4 3 2 1 0 -1 -2 -3 -4 . . .
The digit index specifies the 10th exponent (in this case, 2) for the
number 1024.38.
[REV END]
The digit extraction function results in an integer-type item.
Related Sections
* Calculation Language
* Operators/Operations
* Types
Real Numbers
This section describes features, functions, and operations that apply to
Real numbers only. You can use Real numbers the same way as other number
types. For example, you can define file or database items, HP
ALLBASE/BRW parameters, or calculated items of type R (Real).
When to Use Real Numbers
Unless a Real number is required, it is more efficient to use a Numeric
number.
Take care when using Real numbers. Real numbers are externally
represented in powers of 10 and must be converted to an internal format
where they are represented in powers of 2. This conversion can cause
inconsistencies, for example, when comparing an external Real number such
as a parameter with an internal value read from a file. Two ostensibly
identical values may not be equal.
Real Items and Numeric Items
The HP ALLBASE/BRW type N (Numeric) allows a maximum of 19 digits.
Numeric items are commonly used for numbers with 13 digits left of the
decimal and 6 digits right of the decimal. Real number processing
requires more CPU time than Numeric. Therefore, unless Real values or
Real functions are required, it is better to use a Numeric item.
How HP ALLBASE/BRW Holds Real Numbers
HP Dictionary/V floating point types E and R, and HP System Dictionary
elements with attribute element-type E or R, convert to HP ALLBASE/BRW
type R. HP ALLBASE/BRW uses the Long Real format internally so Real
values are converted to Long Real. HP ALLBASE/BRW can convert Real
numbers whether or not they are held in IEEE or HP 3000 format.
How to Write Real Numbers inHP ALLBASE/BRW Formulas
In HP ALLBASE/BRW formulas, such as for calculated items or functions,
you can express Real values in Integer, fixed point, or scientific
notation formats. The following are legal Real values in HP ALLBASE/BRW
formulas:
247 Integer format
12.36 fixed point format
3.45E2 scientific notation
-.56E27 scientific notation
23.45E-6 scientific notation
Values expressed using scientific notation are subject to the following
conditions:
* A value must all be on one line.
* There must be no embedded spaces in the value.
* Signs can optionally occur at the beginning of the value and/or
following the exponent (E).
* The exponent must not have more than 2 digits.
How to Write Real Numbers inHP ALLBASE/BRW Parameters
You must use a set syntax for parameter values so that the SATISFIES
selection condition will work with comparison predicate parameters.
Default and run-time parameter values obey the following syntax rules:
* They can have an optional sign.
* They must have a decimal point.
* They must have either a non-zero digit optionally followed by up
to 15 additional digits, or, for a zero value, a zero followed by
up to 16 zero digits. That is, if the first digit is zero, the
Real number must also be zero.
* They must contain the letter E.
* They must have an exponent sign.
* They must have a 1 or 2 digit exponent.
Correct Real Parameters
The following are correct Real parameters:
.12345E+3
-.45678E-55
+.654E+0
.0E+0
The following are incorrect Real parameters:
2.34E+3 : A digit must not appear before the decimal
point.
.34 E-3 : Embedded blanks are not allowed.
.6789E : The exponent must have a value.
.478E3 : The exponent must have a sign.
.789E+109 : The exponent cannot have more than two digits.
.123456789012345678E+2 : The mantissa cannot have more than 16 digits.
.01234E-4 : The leading digit is a zero and the value is
not zero.
Real Numeric Precision
You can set the Numeric precision for Real items in the same way as you
can for Numeric and Integer items. Set Numeric precision for Real items
by entering a number from -90 to 76 and a letter T, R, or C. The number
specifies the degree of precision and the letter specifies whether to
truncate, round, or ceil the value. The degree of precision is based on
how the Real number looks in Fixed Point form. For example:
-------------------------------------------------------------------------------------------------
| | | | |
| Real Value | Fixed Point | NumPrec | New Real Value |
| | | | |
-------------------------------------------------------------------------------------------------
| | | | |
| .12347E2 | 12.347 | -2R | .1235E2 |
| | | | |
| .76543E2 | 76.543 | 2R | .1E3 |
| | | | |
| .12347E4 | 1234.7 | 0T | .1234E4 |
| | | | |
-------------------------------------------------------------------------------------------------
Real Item Functions and Operations
You can use Real items in all the operations and functions currently
available for Numeric and Integer items. Some new functions are also
available. These functions are available for use with Real numbers ONLY.
The functions that are only available with Real numbers are:
SIN, COS, TAN, ARCTAN, ARCSIN, ARCCOS, LN (natural logarithm), LOG (base
10 logarithm), E, SQR, SQRT, and ABS.
Calculated items and functions using these Real functions must have a
result of type Real.
Real Item Edits
As for other item types, you can set up default item edits for Reals
using the program BRWSETUP.PUB.SYS, and you can modify the item edit for
each item in a report. For more information about BRWSETUP, see Chapter
17 , "Configuration and Security."
Related Sections
* Formulas
* Parameters
* Standard Deviation
* Types