Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Python's decimal module doesn't like any number from 1 upwards

Tags:

python

math

Any idea why Python's decimal module doesn't like numbers 1 or more but 0.9 and less is okay?

>>> import decimal
>>> max_digits = 5
>>> decimal_places = 5
>>> context = decimal.getcontext().copy()
>>> context.prec = max_digits

1 itself has too many digits:

>>> value = decimal.Decimal('1')
>>> '%s' % str(value.quantize(decimal.Decimal(".1") ** decimal_places, context=context))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/decimal.py", line 2470, in quantize
    'quantize result has too many digits for current context')
  File "/usr/lib/python2.7/decimal.py", line 3872, in _raise_error
    raise error(explanation)
decimal.InvalidOperation: quantize result has too many digits for current context

But anything below 1 is fine:

>>> value = decimal.Decimal('0.9')
>>> '%s' % str(value.quantize(decimal.Decimal(".1") ** decimal_places, context=context))
'0.90000'

Anyone care to explain?

like image 883
Mark L Avatar asked Dec 20 '22 22:12

Mark L


1 Answers

That's because you set the maximum precision context.prec to be 5 digits, while you also set the decimal_places into 5 places after the decimal point. Putting values 1 and above will give you 6 digits of precision (significant figures):

1.00000
^ ^^^^^

which is the 1 plus 5 decimal places. That's why it complains, saying "result has too many digits for current context". note: see, the error message actually explained it! =D

For numbers below 1, there are exactly 5 digits of precision, because the part before the decimal point is not significant.

0.90000
  ^^^^^

You don't need to set the context.prec, or, set it to larger number. Why did you want to set the context in the first place?

Setting the max_digits to 6 works for me.

like image 127
justhalf Avatar answered Mar 24 '23 13:03

justhalf