Home
last modified time | relevance | path

Searched refs:rounding (Results 1 – 25 of 159) sorted by relevance

1234567

/device/linaro/bootloader/edk2/StdLib/LibC/Softfloat/
Dtimesoftfloat.txt13 point routines. Each routine can be evaluated for every relevant rounding
14 mode, tininess mode, and/or rounding precision.
63 the four rounding modes, one after the other. If the rounding mode is not
65 some operations do not require rounding--only the nearest/even rounding mode
68 tininess detected before rounding and after rounding. For extended double-
69 precision operations affected by rounding precision control, `timesoftfloat'
70 also times the function for all three rounding precision modes, one after
71 the other. Evaluation of a function can be limited to a single rounding
72 mode, a single tininess mode, and/or a single rounding precision with
108 For extended double-precision functions affected by rounding precision
[all …]
Dsoftfloat.txt135 All four rounding modes prescribed by the IEC/IEEE Standard are implemented
136 for all operations that require rounding. The rounding mode is selected
139 `float_round_down', or `float_round_up'. The rounding mode is initialized
146 For extended double precision (`floatx80') only, the rounding precision
157 (`float64'), respectively. When rounding to reduced precision, additional
158 bits in the result significand beyond the rounding point are set to zero.
183 for underflow either before or after rounding. The choice is made by
186 Detecting tininess after rounding is better because it results in fewer
219 format are always exact and so require no rounding. Conversions from 32-bit
231 integer value, the operand is rounded according to the current rounding
[all …]
/device/linaro/bootloader/edk2/AppPkg/Applications/Python/Python-2.7.2/Lib/test/decimaltestdata/
Drounding.decTest2 -- rounding.decTest -- decimal rounding modes testcases --
23 -- order to get correct results for some rounding modes. Rather than
24 -- single rounding tests we therefore need tests for most operators.
26 -- is rounding of negatives (if the latter works for addition, assume it
41 rounding: down
73 rounding: half_down
105 rounding: half_even
137 rounding: half_up
169 rounding: up
201 rounding: floor
[all …]
Dtointegral.decTest31 rounding: half_up
166 rounding: half_up
178 -- all rounding modes
179 rounding: half_even
188 rounding: half_up
197 rounding: half_down
206 rounding: up
215 rounding: down
224 rounding: ceiling
233 rounding: floor
DdqToIntegral.decTest26 -- elsewhere; the tests here are for integrity, rounding mode, etc.
40 rounding: half_even
170 rounding: half_up
181 -- all rounding modes
182 rounding: half_even
190 rounding: half_up
199 rounding: half_down
208 rounding: up
217 rounding: down
226 rounding: ceiling
[all …]
DddToIntegral.decTest26 -- elsewhere; the tests here are for integrity, rounding mode, etc.
40 rounding: half_even
170 rounding: half_up
181 -- all rounding modes
182 rounding: half_even
190 rounding: half_up
199 rounding: half_down
208 rounding: up
217 rounding: down
226 rounding: ceiling
[all …]
Dtointegralx.decTest31 rounding: half_up
166 rounding: half_up
178 -- all rounding modes
179 rounding: half_even
188 rounding: half_up
197 rounding: half_down
206 rounding: up
215 rounding: down
224 rounding: ceiling
233 rounding: floor
Dadd.decTest23 rounding: half_up
238 rounding: half_up
260 rounding: half_even
286 rounding: down
310 rounding: half_up
331 rounding: half_down
341 rounding: half_up
394 rounding: half_up
403 rounding: half_down
408 rounding: half_up
[all …]
DddAdd.decTest29 rounding: half_even
85 -- same, without rounding
218 rounding: half_up
236 rounding: half_up
258 rounding: half_even
284 rounding: down
307 rounding: half_up
358 rounding: half_up
366 rounding: half_down
369 rounding: half_up
[all …]
Dfma.decTest24 rounding: half_up
379 -- test some edge cases with exact rounding
416 rounding: half_up
647 -- test subnormals rounding
651 rounding: half_even
748 rounding: half_up
769 rounding: half_up
782 rounding: half_up
996 rounding: half_up
1018 rounding: half_even
[all …]
DdqAdd.decTest29 rounding: half_even
85 -- same, without rounding
218 rounding: half_up
235 rounding: half_up
257 rounding: half_even
283 rounding: down
306 rounding: half_up
357 rounding: half_up
366 rounding: half_down
369 rounding: half_up
[all …]
DddBase.decTest25 -- Note that unlike other operations the operand is subject to rounding
38 rounding: half_even
389 rounding: half_up
447 -- check rounding modes heeded
448 rounding: ceiling
453 rounding: up
458 rounding: floor
463 rounding: half_down
469 rounding: half_even
475 rounding: down
[all …]
DdqBase.decTest25 -- Note that unlike other operations the operand is subject to rounding
38 rounding: half_even
432 -- check rounding modes heeded
433 rounding: ceiling
438 rounding: up
443 rounding: floor
448 rounding: half_down
454 rounding: half_even
460 rounding: down
465 rounding: half_up
[all …]
DdsBase.decTest25 -- Note that unlike other operations the operand is subject to rounding
38 rounding: half_even
411 -- check rounding modes heeded
412 rounding: ceiling
417 rounding: up
422 rounding: floor
427 rounding: half_down
433 rounding: half_even
439 rounding: down
444 rounding: half_up
[all …]
Dbase.decTest26 -- Note that unlike other operations the operand is subject to rounding
31 rounding: half_up
427 -- all-nines rounding
429 rounding: half_up
443 -- check rounding modes heeded
445 rounding: ceiling
450 rounding: up
455 rounding: floor
460 rounding: half_down
466 rounding: half_even
[all …]
DdqSubtract.decTest29 rounding: half_even
113 -- with input rounding.
353 rounding: half_up
375 rounding: half_even
401 rounding: down
424 rounding: half_up
446 rounding: half_even
472 rounding: down
496 -- input rounding would have affected the following
497 rounding: half_up
[all …]
DddSubtract.decTest29 rounding: half_even
113 -- with input rounding.
353 rounding: half_up
375 rounding: half_even
401 rounding: down
424 rounding: half_up
446 rounding: half_even
472 rounding: down
496 -- input rounding would have affected the following
497 rounding: half_up
[all …]
Dsubtract.decTest24 rounding: half_up
122 -- with input rounding.
446 rounding: half_up
468 rounding: half_even
494 rounding: down
517 rounding: half_up
539 rounding: half_even
565 rounding: down
589 -- input rounding would have affected the following
591 rounding: half_up
[all …]
DddDivide.decTest27 rounding: half_even
223 -- These were 'input rounding'
285 rounding: half_up
289 rounding: half_up
566 rounding: half_down
568 rounding: half_even
699 -- Division probably has pre-rounding, so need to test rounding
701 -- tests include simple rounding and also the tricky cases of sticky
710 rounding: ceiling
716 rounding: floor
[all …]
DdqFMA.decTest27 rounding: half_even
143 rounding: half_even
266 -- test some edge cases with exact rounding
583 rounding: half_up
597 rounding: half_even
653 -- same, without rounding
786 rounding: half_up
803 rounding: half_up
825 rounding: half_even
851 rounding: down
[all …]
Dpower.decTest28 rounding: half_even
185 -- Modified 2002.10.06 -- finally, no input rounding
186 -- With input rounding, result would be 8.74E-2226
630 rounding: half_even
828 rounding: half_even
868 rounding: ceiling
872 rounding: half_even
928 rounding: half_even
995 -- Ntiny boundary, these edge cases determined using half_up rounding
996 rounding: half_up
[all …]
/device/linaro/bootloader/edk2/StdLib/LibC/gdtoa/
Ddtoa.c72 #define Rounding rounding
141 int rounding; local
191 if ((rounding = Flt_Rounds) >= 2) {
193 rounding = rounding == 2 ? 0 : 2;
195 if (rounding != 2)
196 rounding = 0;
333 if (mode > 1 && rounding != 1)
470 switch(rounding) {
553 && rounding == 1
666 && rounding >= 1
[all …]
Dstrtod.c70 #define Rounding rounding
105 int rounding; in strtod() local
146 case FE_TOWARDZERO: fpi1.rounding = 0; break; in strtod()
147 case FE_UPWARD: fpi1.rounding = 2; break; in strtod()
148 case FE_DOWNWARD: fpi1.rounding = 3; in strtod()
409 if ((rounding = Flt_Rounds) >= 2) { in strtod()
411 rounding = rounding == 2 ? 0 : 2; in strtod()
413 if (rounding != 2) in strtod()
414 rounding = 0; in strtod()
433 switch(rounding) { in strtod()
[all …]
/device/linaro/bootloader/edk2/AppPkg/Applications/Python/Python-2.7.2/Lib/
Ddecimal.py345 if context.rounding in (ROUND_HALF_UP, ROUND_HALF_EVEN,
349 if context.rounding == ROUND_CEILING:
354 if context.rounding == ROUND_FLOOR:
1074 if not self and context.rounding != ROUND_FLOOR:
1096 if not self and context.rounding != ROUND_FLOOR:
1153 if context.rounding == ROUND_FLOOR and self._sign != other._sign:
1166 ans = other._rescale(exp, context.rounding)
1171 ans = self._rescale(exp, context.rounding)
1367 self._rescale(ideal_exp, context.rounding))
1517 ans = self._rescale(ideal_exponent, context.rounding)
[all …]
/device/linaro/bootloader/edk2/StdLib/LibC/Main/X64/
Dfpu_rmode.asm2 ; Return the current FPU rounding mode.
38 ret ; Return rounding mode in RAX

1234567