Home
last modified time | relevance | path

Searched refs:rounding (Results 1 – 25 of 343) sorted by relevance

12345678910>>...14

/external/valgrind/none/tests/s390x/
Drounding-5.stdout.exp1 setting FPC rounding mode to [-> near]
3 setting FPC rounding mode to [-> zero]
5 setting FPC rounding mode to [-> +inf]
7 setting FPC rounding mode to [-> -inf]
10 setting FPC rounding mode to [-> near]
12 setting FPC rounding mode to [-> zero]
14 setting FPC rounding mode to [-> +inf]
16 setting FPC rounding mode to [-> -inf]
19 setting FPC rounding mode to [-> near]
21 setting FPC rounding mode to [-> zero]
[all …]
Drounding-4.stdout.exp1 setting FPC rounding mode to [-> near]
3 setting FPC rounding mode to [-> zero]
5 setting FPC rounding mode to [-> +inf]
7 setting FPC rounding mode to [-> -inf]
10 setting FPC rounding mode to [-> near]
12 setting FPC rounding mode to [-> zero]
14 setting FPC rounding mode to [-> +inf]
16 setting FPC rounding mode to [-> -inf]
19 setting FPC rounding mode to [-> near]
21 setting FPC rounding mode to [-> zero]
[all …]
Dsrnmt.stdout.exp1 initial rounding mode = 0
2 rounding mode = 3
3 rounding mode = 2
4 rounding mode = 1
5 rounding mode = 0
6 rounding mode = 7
7 rounding mode = 6
8 rounding mode = 5
9 rounding mode = 4
Dsrnm.stdout.exp1 initial rounding mode = 0
2 rounding mode = 3
3 rounding mode = 2
4 rounding mode = 1
5 rounding mode = 0
6 rounding mode = 3
7 rounding mode = 2
8 rounding mode = 3
Dsrnmb.stdout.exp1 initial rounding mode = 0
2 rounding mode = 3
3 rounding mode = 2
4 rounding mode = 1
5 rounding mode = 0
6 rounding mode = 1
7 rounding mode = 0
Drounding-2.stdout.exp1 initial rounding mode: 0
2 setting rounding mode to 0
4 setting rounding mode to 1
6 setting rounding mode to 2
8 setting rounding mode to 3
Dfpext_warn.stderr.exp3 The specified rounding mode cannot be supported. That
6 the rounding mode from FPC. Results may differ!
9 The specified rounding mode cannot be supported. That
12 the rounding mode from FPC. Results may differ!
15 The specified rounding mode cannot be supported. That
18 the rounding mode from FPC. Results may differ!
DMakefile.am12 rounding-1 rounding-2 rounding-3 rounding-4 rounding-5 bfp-1 \
14 ex clst mvc test_fork test_sig rounding-6 rxsbg popcnt \
44 mul.h or.h sub.h xor.h table.h svc.h rounding.h \
/external/valgrind/none/tests/mips32/
DFPUarithmetic.stdout.exp2 rounding mode: near
27 rounding mode: zero
52 rounding mode: +inf
77 rounding mode: -inf
102 rounding mode: near
127 rounding mode: zero
152 rounding mode: +inf
177 rounding mode: -inf
202 rounding mode: near
227 rounding mode: zero
[all …]
DFPUarithmetic.stdout.exp-mips322 rounding mode: near
27 rounding mode: zero
52 rounding mode: +inf
77 rounding mode: -inf
102 rounding mode: near
127 rounding mode: zero
152 rounding mode: +inf
177 rounding mode: -inf
202 rounding mode: near
227 rounding mode: zero
[all …]
/external/webp/src/dsp/
Dyuv.h190 static WEBP_INLINE int VP8ClipUV(int uv, int rounding) { in VP8ClipUV() argument
191 uv = (uv + rounding + (128 << (YUV_FIX + 2))) >> (YUV_FIX + 2); in VP8ClipUV()
197 static WEBP_INLINE int VP8RGBToY(int r, int g, int b, int rounding) { in VP8RGBToY() argument
199 return (luma + rounding + (16 << YUV_FIX)) >> YUV_FIX; // no need to clip in VP8RGBToY()
202 static WEBP_INLINE int VP8RGBToU(int r, int g, int b, int rounding) { in VP8RGBToU() argument
204 return VP8ClipUV(u, rounding); in VP8RGBToU()
207 static WEBP_INLINE int VP8RGBToV(int r, int g, int b, int rounding) { in VP8RGBToV() argument
209 return VP8ClipUV(v, rounding); in VP8RGBToV()
217 static WEBP_INLINE int VP8RGBToY(int r, int g, int b, int rounding) { in VP8RGBToY() argument
219 return (luma + rounding) >> YUV_FIX; // no need to clip in VP8RGBToY()
[all …]
/external/valgrind/none/tests/ppc32/
Dtest_dfp2.stdout.exp_Without_dcffix3 test with rounding mode 0
20 test with rounding mode 1
37 test with rounding mode 2
54 test with rounding mode 3
71 test with rounding mode 4
88 test with rounding mode 5
105 test with rounding mode 6
122 test with rounding mode 7
140 test with rounding mode 0
157 test with rounding mode 1
[all …]
Dtest_dfp2.stdout.exp3 test with rounding mode 0
20 test with rounding mode 1
37 test with rounding mode 2
54 test with rounding mode 3
71 test with rounding mode 4
88 test with rounding mode 5
105 test with rounding mode 6
122 test with rounding mode 7
140 test with rounding mode 0
157 test with rounding mode 1
[all …]
/external/valgrind/none/tests/ppc64/
Dtest_dfp2.stdout.exp_Without_dcffix3 test with rounding mode 0
20 test with rounding mode 1
37 test with rounding mode 2
54 test with rounding mode 3
71 test with rounding mode 4
88 test with rounding mode 5
105 test with rounding mode 6
122 test with rounding mode 7
140 test with rounding mode 0
157 test with rounding mode 1
[all …]
Dtest_dfp2.stdout.exp3 test with rounding mode 0
20 test with rounding mode 1
37 test with rounding mode 2
54 test with rounding mode 3
71 test with rounding mode 4
88 test with rounding mode 5
105 test with rounding mode 6
122 test with rounding mode 7
140 test with rounding mode 0
157 test with rounding mode 1
[all …]
/external/python/cpython2/Lib/test/decimaltestdata/
Drounding.decTest2 -- rounding.decTest -- decimal rounding modes testcases --
23 -- order to get correct results for some rounding modes. Rather than
24 -- single rounding tests we therefore need tests for most operators.
26 -- is rounding of negatives (if the latter works for addition, assume it
41 rounding: down
73 rounding: half_down
105 rounding: half_even
137 rounding: half_up
169 rounding: up
201 rounding: floor
[all …]
Dtointegral.decTest31 rounding: half_up
166 rounding: half_up
178 -- all rounding modes
179 rounding: half_even
188 rounding: half_up
197 rounding: half_down
206 rounding: up
215 rounding: down
224 rounding: ceiling
233 rounding: floor
DddToIntegral.decTest26 -- elsewhere; the tests here are for integrity, rounding mode, etc.
40 rounding: half_even
170 rounding: half_up
181 -- all rounding modes
182 rounding: half_even
190 rounding: half_up
199 rounding: half_down
208 rounding: up
217 rounding: down
226 rounding: ceiling
[all …]
DdqToIntegral.decTest26 -- elsewhere; the tests here are for integrity, rounding mode, etc.
40 rounding: half_even
170 rounding: half_up
181 -- all rounding modes
182 rounding: half_even
190 rounding: half_up
199 rounding: half_down
208 rounding: up
217 rounding: down
226 rounding: ceiling
[all …]
Dadd.decTest23 rounding: half_up
238 rounding: half_up
260 rounding: half_even
286 rounding: down
310 rounding: half_up
331 rounding: half_down
341 rounding: half_up
394 rounding: half_up
403 rounding: half_down
408 rounding: half_up
[all …]
/external/icu/icu4j/demos/src/com/ibm/icu/dev/demo/number/
DCurrencyDemo.java50 result.setRoundingIncrement(hack.rounding); in getCurrencyFormat()
60 double rounding; field in CurrencyDemo.HackCurrencyInfo
62 HackCurrencyInfo(int decimals, double rounding, String symbol) { in HackCurrencyInfo() argument
64 this.rounding = rounding; in HackCurrencyInfo()
/external/libhevc/common/arm/
Dihevc_inter_pred_chroma_vert_w16inp.s161 vqrshrun.s16 d0,q0,#6 @rounding shift
162 vqrshrun.s16 d30,q15,#6 @rounding shift
219 vqrshrun.s16 d30,q15,#6 @rounding shift
234 vqrshrun.s16 d28,q14,#6 @rounding shift
247 vqrshrun.s16 d26,q13,#6 @rounding shift
259 vqrshrun.s16 d24,q12,#6 @rounding shift
272 vqrshrun.s16 d30,q15,#6 @rounding shift
286 vqrshrun.s16 d28,q14,#6 @rounding shift
301 vqrshrun.s16 d26,q13,#6 @rounding shift
311 vqrshrun.s16 d24,q12,#6 @rounding shift
[all …]
Dihevc_itrans_recon_4x4_ttype1.s169 vqrshrn.s32 d14,q3,#shift_stage1_idct @ (pi2_out[0] + rounding ) >> shift_stage1_idct
170 vqrshrn.s32 d15,q4,#shift_stage1_idct @ (pi2_out[1] + rounding ) >> shift_stage1_idct
171 vqrshrn.s32 d16,q5,#shift_stage1_idct @ (pi2_out[2] + rounding ) >> shift_stage1_idct
172 vqrshrn.s32 d17,q6,#shift_stage1_idct @ (pi2_out[3] + rounding ) >> shift_stage1_idct
209 vqrshrn.s32 d0,q3,#shift_stage2_idct @ (pi2_out[0] + rounding ) >> shift_stage1_idct
210 vqrshrn.s32 d1,q4,#shift_stage2_idct @ (pi2_out[1] + rounding ) >> shift_stage1_idct
211 vqrshrn.s32 d2,q5,#shift_stage2_idct @ (pi2_out[2] + rounding ) >> shift_stage1_idct
212 vqrshrn.s32 d3,q6,#shift_stage2_idct @ (pi2_out[3] + rounding ) >> shift_stage1_idct
/external/libvpx/libvpx/vp9/encoder/x86/
Dtemporal_filter_sse4.c114 const int strength, const int rounding, in average_8() argument
118 const __m128i rounding_u16 = _mm_set1_epi16(rounding); in average_8()
141 const int rounding, const int weight) { in average_16() argument
143 const __m128i rounding_u16 = _mm_set1_epi16(rounding); in average_16()
243 const int rounding = strength > 0 ? 1 << (strength - 1) : 0; in vp9_temporal_filter_apply_sse4_1() local
263 sum_row_c = average_8(sum_row_c, mul_constants, strength, rounding, weight); in vp9_temporal_filter_apply_sse4_1()
281 average_8(sum_row_a, mul_constants, strength, rounding, weight); in vp9_temporal_filter_apply_sse4_1()
298 sum_row_a = average_8(sum_row_a, mul_constants, strength, rounding, weight); in vp9_temporal_filter_apply_sse4_1()
321 strength, rounding, weight); in vp9_temporal_filter_apply_sse4_1()
346 strength, rounding, weight); in vp9_temporal_filter_apply_sse4_1()
[all …]
/external/gemmlowp/doc/
Dless-than-8-bit.txt40 a rounding mode, based on the accumulation size (a.k.a. the
159 need to multiply result values by a constant fraction, rounding to nearest.
178 It depends on the bit depth and on a rounding mode, and requantizes an input
182 Naive, bad rounding, that's plainly biased
191 effectively implements rounding towards 0.
196 Round-to-nearest rounding: unbiased in principle but not in practice
199 The simplest fix is to avoid the bias in 2. by rounding-to-nearest instead
200 of rounding towards 0. This can be achieved by doing
209 to achieve rounding-to-nearest (while the above rounding towards 0
212 In principle, rounding-to-nearest is unbiased and optimal in various ways.
[all …]

12345678910>>...14