Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to force a Float to display with full precision w/o scientific notation and not as a string?

How do you force a float to display with all significant places / full precision without scientific notation in Ruby?

Presently I convert a BigDecimal to Float, BigDecimal(0.000000001453).to_f, however this yields a resultant float of 1.453e-09. If I do something like "%14.12f" % BigDecimal("0.000000001453").to_f I get a string. In this case, however, a string as output is unacceptable as I need it as an actual numeric float without scientific notation.

--- Edit---

Alright, let me give some context here, which will probably require a change of my original question.

I'm attempting to create a graph with Highstock & lazy_high_chart. Earlier today I was able to draw graphs just fine when the floats were emitting to the resultant js as full precision floats vs showing up in scientific notation. Hence I felt that the problem resides in this issue.

But after the few inputs I'm getting here, perhaps I need some further review of the source and my assumption is misplaced. I'll let you decide:

@h = LazyHighCharts::HighChart.new('graph') do |f|
  hours_of_readings = 1
  reading_intervals = 1 #hour
  readings_per_hour = 60

  readings = ModelName.order("date DESC").select('data_readings.data2, data_readings.data1, data_readings.date').limit(hours_of_readings * readings_per_hour).all

  data1_and_date_series = Array.new
  data2_and_date_series = Array.new
  dates = Array.new

  # I have been thinking that the problem lies here in the "row.data1.to_f" and
  #   "row.data2.to_f" and thus this is the root of my initial question in terms 
  #   of it emitting scientific notation to the js output in the format of:
  #   [[1.0e-09], [1.04e-09],[9.4e-09], ... [3.68e-09]]
  data1_and_date_series = readings.map{|row| [(row.date.to_i * 1000), (row.data1.to_f if BigDecimal(row.data1) != BigDecimal("-1000.0"))] }
  data2_and_date_series = readings.map{|row| [(row.date.to_i * 1000), (row.data2.to_f if BigDecimal(row.data2) != BigDecimal("-1000.0"))] }


  f.series(
      :name => 'Data1',
      :data => data1_and_date_series,
      :pointStart => Time.now.to_i * 1000,
      :pointEnd => hours_of_readings.hours.ago.to_i * 1000,
      :pointInterval => reading_intervals.hour * 1000,
      :color => 'blue'
  )
  f.series(
      :name => 'Data2)',
      :data => data2_and_date_series,
      :pointStart => Time.now.to_i * 1000,
      :pointEnd => hours_of_readings.hours.ago.to_i * 1000,
      :pointInterval => reading_intervals.hour.to_i * 1000,
      :color => 'red'
  )

  f.chart({:defaultSeriesType=>"spline" })
  f.yAxis [
        {:title => { :text => "Label 1", :margin => 10} },
        {:title => { :text => "Label 2 (groups)"}, :opposite => true},
        {:max => 0},
        {:min => -0.000000001}
  ]
  f.options[:xAxis] = {
      :title => { :text => "Time"},
      :type => "datetime"
  }
  f.title(:text => "Title")
  f.legend(:align => 'right', :verticalAlign => 'top', :y => 75, :x => -50, :layout => 'vertical') # override the default values
end
like image 840
ylluminate Avatar asked Jan 10 '12 08:01

ylluminate


People also ask

Is it possible to get rid of scientific notation in float?

It is well-known that the repr of a float is written in scientific notation if the exponent is greater than 15, or less than -4: If str is used, the resulting string again is in scientific notation: It has been suggested that I can use format with f flag and sufficient precision to get rid of the scientific notation:

How to suppress scientific notation of floating point values in pandas Dataframe?

Format Display settings of Floats in Pandas You can use the pandas set_option () function to change the display settings of a pandas dataframe. This function allows you to change a range of options. For this tutorial, we will focus on how to suppress scientific notation of floating-point values appearing in the dataframe.

How can I get a string representation of a float?

For the background of this change see here. A simple solution to get a string representation of a float with full precision is to use either repr or json.dumps. JSON serialization/deserialization has to make sure that roundtrips are loss-less, and thus, the implementation produces a string representation you are looking for:

What is the precision of STR on float in Python 3?

UPDATE: In Python 3 str on a float is guaranteed to produce a string literal with full precision This was not the case in Python 2. For instance str (1.0000000000000002) was '1.0' in Python 2, but in Python 3 it gives '1.0000000000000002' as expected.


2 Answers

The string representation and the actual value of a float are two different things.

What you see on screen/print-out is always a string representation, be it in scientific notation or "normal" notation. A float is converted to its string representation by to_s, puts, "%.10f" % and others.

The float value itself is independent of that. So your last sentence does not make much sense. The output is always a string.

To enforce a certain float format in Rails' to_json you can overwrite Float#encode_json, e.g.

class ::Float
  def encode_json(opts = nil)
    "%.10f" % self
  end
end

Put this before your code above. Note that -- depending on your actual values -- you might need more sophisticated logic to produce reasonable strings.

like image 152
undur_gongor Avatar answered Nov 14 '22 21:11

undur_gongor


Will this work for you -

>> 0.000000001453
=> 1.453e-09 # what you are getting right now
>> puts "%1.12f" % 0.000000001453
0.000000001453 # what you need
=> nil
like image 30
jaypal singh Avatar answered Nov 14 '22 21:11

jaypal singh