| commit | d44d7d6bebdb7fa52090b744854d49f34099e044 | |
| author | Erik Lindahl <erik@kth.se> | |
| Sun, 8 May 2016 20:49:36 +0000 (8 22:49 +0200) | ||
| committer | Mark Abraham <mark.j.abraham@gmail.com> | |
| Tue, 10 May 2016 14:55:17 +0000 (10 16:55 +0200) | ||
| tree | 9034deaf9dddae3ad388c956557244477fe7bc08 | treesnapshot (tar.gz zip) |
| parent | 4ce2aa504d36a646f3b084c871ee0059446144a9 | commitdiff |
Fix precision loss in tabulated normal distribution
The tabulated normal distribution was generated by
simply integrating a gaussian function, which led to
severe loss-of-precision for double precision tables.
Fixed by implementing proper inverse error functions
in the math module, and using this to fill the table
directly.
Fixes #1930.
Change-Id: I74b72b4a6d36f1eb382c6456501ce5644c92725e
The tabulated normal distribution was generated by
simply integrating a gaussian function, which led to
severe loss-of-precision for double precision tables.
Fixed by implementing proper inverse error functions
in the math module, and using this to fill the table
directly.
Fixes #1930.
Change-Id: I74b72b4a6d36f1eb382c6456501ce5644c92725e