How to get bc to handle numbers in scientific (aka exponential) notation?

Question

bc doesn't like numbers expressed in scientific notation (aka exponential notation).

$ echo "3.1e1*2" | bc -l (standard_in) 1: parse error 

but I need to use it to handle a few records that are expressed in this notation. Is there a way to get bc to understand exponential notation? If not, what can I do to translate them into a format that bc will understand?

Answer 1

Unfortunately, bc doesn't support scientific notation.

However, it can be translated into a format that bc can handle, using extended regex as per POSIX in sed:

sed -E 's/([+-]?[0-9.]+)[eE]\+?(-?)([0-9]+)/(\1*10^\2\3)/g' <<<"$value" 

you can replace the "e" (or "e+", if the exponent is positive) with "*10^", which bc will promptly understand. This works even if the exponent is negative or if the number is subsequently multiplied by another power, and allows keeping track of significant digits.

If you need to stick to basic regex (BRE), then this should be used:

sed 's/\([+-]\{0,1\}[0-9]*\.\{0,1\}[0-9]\{1,\}\)[eE]+\{0,1\}\(-\{0,1\}\)\([0-9]\{1,\}\)/(\1*10^\2\3)/g' <<<"$value" 

---

From Comments:

• A simple bash pattern match could not work (thanks @mklement0) as there is no way to match a e+ and keep the - from a e- at the same time.

• A correctly working perl solution (thanks @mklement0)

  $ perl -pe 's/([-\d.]+)e(?:\+|(-))?(\d+)/($1*10^$2$3)/gi' <<<"$value" 

• Thanks to @jwpat7 and @Paul Tomblin for clarifying aspects of sed's syntax, as well as @isaac and @mklement0 for improving the answer.

Edit:

The answer changed quite a bit over the years. The answer above is the latest iteration as of 17th May 2018. Previous attempts reported here were a solution in pure bash (by @ormaaj) and one in sed (by @me), that fail in at least some cases. I'll keep them here just to make sense of the comments, which contain much nicer explanations of the intricacies of all this than this answer does.

value=${value/[eE]+*/*10^}  ------> Can not work. value=`echo ${value} | sed -e 's/[eE]+*/\\*10\\^/'` ------> Fail in some conditions 

Answer 2

Let me try to summarize the existing answers, with comments on each below:

• (a) If you indeed need to use bc for arbitrary-precision calculations - as the OP does - use the OP's own clever approach, which textually reformats the scientific notation to an equivalent expression that bc understands.

• If potentially losing precision is not a concern,

  • (b) consider using awk or perl as bc alternatives; both natively understand scientific notation, as demonstrated in jwpat7's answer for awk.
  • (c) consider using printf '%.<precision>f' to simply textually convert to regular floating point representation (decimal fractions, without the e/E) (a solution proposed in a since-deleted post by ormaaj).

---

(a) Reformatting scientific notation to an equivalent bc expression

The advantage of this solution is that precision is preserved: the textual representation is transformed into an equivalent textual representation that bc can understand, and bc itself is capable of arbitrary-precision calculations.

See the OP's own answer, whose updated form is now capable of transforming an entire expression containing multiple numbers in exponential notation into an equivalent bc expression.

---

(b) Using awk or perl instead of bc as the calculator

Note: The following approaches assume use of the built-in support for double-precision floating-point values in awk and perl. As is in inherent in floating-point arithmetic,
^{"given any fixed number of bits, most calculations with real numbers will produce quantities that cannot be exactly represented using that many bits. Therefore the result of a floating-point calculation must often be rounded in order to fit back into its finite representation. This rounding error is the characteristic feature of floating-point computation." (http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html)}

That said,

• GNU awk offers the option to be built with support for arbitrary-precision arithmetic - see https://www.gnu.org/software/gawk/manual/html_node/Gawk-and-MPFR.html; however, distributions may or may not include that support - verify support by checking the output from gawk --version for GNU MPFR and GNU MP.
  If support is available, you must activate it with -M (--bignum) in a given invocation.

• Perl offers optional arbitrary-precision decimal support via the Math::BigFloat package - see https://metacpan.org/pod/Math::BigFloat

awk

awk natively understands decimal exponential (scientific) notation.
(You should generally only use decimal representation, because awk implementations differ with respect to whether they support number literals with other bases.)

awk 'BEGIN { print 3.1e1 * 2 }'  # -> 62 

If you use the default print function, the OFMT variable controls the output format by way of a printf format string; the (POSIX-mandated) default is %.6g, meaning 6 significant digits, which notably includes the digits in the integer part.

Note that if the number in scientific notation is supplied as input (as opposed to a literal part of the awk program), you must add +0 to force it to the default output format, if used by itself with print:

^{Depending on your locale and the awk implementation you use, you may have to replace the decimal point (.) with the locale-appropriate radix character, such as , in a German locale; applies to BSD awk, mawk, and to GNU awk with the --posix option.}

awk '{ print $1+0 }' <<<'3.1e1' # -> 31; without `+0`, output would be the same as input 

Modifying variable OFMT changes the default output format (for numbers with fractional parts; (effective) integers are always output as such).
Alternatively, use the printf function with an explicit output format:

awk 'BEGIN { printf "%.4f", 3.1e1 * 2.1234 }' # -> 65.8254 

Perl

perl too natively understands decimal exponential (scientific) notation.

Note: Perl, unlike awk, isn't available on all POSIX-like platforms by default; furthermore, it's not as lightweight as awk.
However, it offers more features than awk, such as natively understanding hexadecimal and octal integers.

perl -le 'print 3.1e1 * 2'  # -> 62 

I'm unclear on what Perl's default output format is, but it appears to be %.15g. As with awk, you can use printf to choose the desired output format:

perl -e 'printf "%.4f\n", 3.1e1 * 2.1234' # -> 65.8254 

---

(c) Using printf to convert scientific notation to decimal fractions

If you simply want to convert scientific notation (e.g., 1.2e-2) into a decimal fraction (e.g., 0.012), printf '%f' can do that for you. Note that you'll convert one textual representation into another via floating-point arithmetic, which is subject to the same rounding errors as the awk and perl approaches.

printf '%.4f' '1.2e-2' # -> '0.0120'; `.4` specifies 4 decimal digits.