CharlesO

CharlesO

Rounding Error with Decimal Sum

Hi.

Please I’m getting this unusual rounding error when summing numbers using the Decimal Library

My result is off by 0.01

  208,505,442,949.08 # Excel
  208,505,442,949.09 # sum function with Decimal

Any suggestions please?

Thanks.

def sum() do
    f = "C:/Downloads/MMR100.txt"
    {:ok, b} = :file.read_file(f)
    a = String.split(b,"\n")
    c = Enum.count(a)
    IO.puts c # 299,725

    _sum(a, "0.00") |> Decimal.to_string(:normal)

    # 208,505,442,949.08 - Excel
    # 208,505,442,949.09 - sum
  end

  defp _sum([], acc), do: acc
  defp _sum(["" | rest], acc), do: _sum(rest, Decimal.add("0", acc))
  defp _sum([v | rest], acc), do: _sum(rest, Decimal.add("#{v}", acc))

Marked As Solved

NobbZ

NobbZ

Parsing and summing the list as float gives: 208505442949.07806, rounding this to 2 decimal places returns 208505442949.08, so obviously Excel and your verification program do use floats and therefore are inaccurate by design…

PS: I’ve heard there are still people using Excel to manage their money…

PPS: my iex session:

iex(3)> File.read!("MMR100.txt") |> String.split() |> Enum.map(&String.to_float/1) |> Enum.sum
208505442949.07806
iex(4)> File.read!("MMR100.txt") |> String.split() |> Enum.map(&String.to_float/1) |> Enum.sum |> Float.round(2)
208505442949.08

Also Liked

NobbZ

NobbZ

The magic of floats.

http://0.30000000000000004.com/

LostKobrakai

LostKobrakai

This exactly is the reason, why you’d even need a library like Decimal. Because floats are not suited for calculating correctly.

NobbZ

NobbZ

No, I do get 9:

iex()> File.read!("MMR100.txt") |> String.split() |> Enum.map(&String.graphemes/1) |> Enum.map(&List.last/1) |> Enum.map(&String.to_integer/1) |> Enum.sum
1349409
LostKobrakai

LostKobrakai

OvermindDL1

OvermindDL1

That’s horrifying… They really need to be taught how very very VERY bad that is…

If they want to use a Microsoft product then they should use Access instead of Excel, it can do everything Excel can except it has types, like a Decimal type, that does things right for such numbers…

Where Next?

Popular in Questions Top

Tee
can someone please explain to me how Enum.reduce works with maps
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
tduccuong
Hi, is there any work on GUI with Elixir, that is similar to Electron/Javascript? My idea is to bundle Phoenix and BEAM into a single se...
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
script
If I have a string “1000 cfu/ml” . I want to remove the characters and / and space . So the string is like this "1000" What is the ...
New
freewebwithme
Using vs code and installed ElixirLS: support and debugger. And I got an error popped up on start up says Failed to run ‘elixir’ comma...
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
jc00ke
Expanding on this topic: https://forum.elixirforum.net/t/map-typespec-question/19217 Let’s say I have a map with required and optional k...
New
lanycrost
Hi everyone! I need implement if…else if…else condition from my elixir code, and anymore of this control flow structures not work proper...
New

Other popular topics Top

lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
malloryerik
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
New
quazar
How to set Jason to encode all fields in ecto schema, I don’t care about security and implementing only is taking long list of attributes...
New
lastday4you
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
New
vac
Hi, I'm quite new in Elixir and I'm trying to format a string to a PEM format. I have the certificate value like MIIDBTCCAe2...... and ...
New
fayddelight
I tried installing elixir 1.11.2 erlang 23.3.4 via asdf in my zsh shell. Enabled the versions locally and globally. When I list them ...
New
stefanluptak
Hello everybody, usually, I use a 29" ultra-wide monitor for VSCode which can easily accomodate explorer (files panel) + file with code ...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New

We're in Beta

About us Mission Statement