Logo Questions Linux Laravel Mysql Ubuntu Git Menu
HTML CSS JAVASCRIPT SQL PYTHON PHP BOOTSTRAP JAVA JQUERY R React Kotlin
 

New posts in precision

round() breaks when PHP precision is set to 18+

php math rounding precision

Floating Point Arithmetic: Deriving Wobble [duplicate]

floating-point precision numerical-methods

atof accuracy with double is causing grief

c precision c-standard-library

Accuracy and polynomial evaluation in Fortran and Python

python fortran precision floating-accuracy polynomial-math

Implementing the generalized birthday paradox in Python

python numpy precision probability

MouseLeftButtonDown on canvas requires too much precision

c# wpf canvas mouse precision

What is the standard way to maintain accuracy when dealing with incredibly precise floating point calculations in C++?

c++ math precision gmp

How can I optimize conversion from half-precision float16 to single-precision float32?

c++ performance precision sse intrinsics

Logstash 8.0 @timestamp with nanoseconds precision

timestamp logstash precision

How to avoid math.sin(math.pi*2*VERY LARGE NUMBER) having a much larger error margin than math.sin(math.pi*2)?

python numpy math precision trigonometry

Matlab Importdata Precision

matlab file-io precision data-import

I'm losing 2's powers in IEEE754 multiplication

c++ c double precision ieee-754

Lowest double positive value - can cout change its value?

c++ double precision cout ieee

How to deal with inexact floating point arithmetic results in Rust? [closed]

floating-point rust precision floating-accuracy

Does the implementation of pow() function in C/C++ vary with platform or compiler?

c++ precision terminology pow

Large numerical errors in my Mathematica program

wolfram-mathematica precision numerical

Python decimal.Decimal - getcontext().prec setting and number of decimals in output

python floating-point decimal rounding precision
Older Entries »