Commit graph

5 commits

Author SHA1 Message Date
Martin Storsjö
6668268e16 checkasm: lls: Use relative tolerances rather than absolute ones
Depending on the magnitude of the output values, the potential
errors can be larger.

This fixes errors in the lls tests on x86_32 for some seeds,
observed with GCC 11 (on Ubuntu 22.04, with the distro compiler,
with -m32).

Signed-off-by: Martin Storsjö <martin@martin.st>
2024-10-09 15:52:56 +03:00
James Almer
97fd5d3363 checkasm/lls: increase epsilon value for the update_lls test
Should fix failures for some seeds on x86_32.

Signed-off-by: James Almer <jamrial@gmail.com>
2024-07-19 09:24:59 -03:00
James Almer
a7e9f1c1e7 checkasm/lls: add missing random values to the test buffers
Fixes valgrind warnings after 18adaf9fe5.

Signed-off-by: James Almer <jamrial@gmail.com>
2024-06-13 14:21:18 -03:00
Rémi Denis-Courmont
18adaf9fe5 checkasm/lls: adjust buffer sizes and alignments
var must be padded.
param has `order + 1`, not `order` elements and is *not* over-aligned.
2024-06-11 20:07:55 +03:00
Rémi Denis-Courmont
fc85aff72f checkasm: add linear least square tests 2024-06-01 18:05:58 +03:00