Open Access

Table 6

Error on k in pcm after lossy compression by truncated HT on the UGd dataset with separate matrices M per nuclide and reaction.

τ μ P5 P50 Mean P95 max tail
1.0e−02 96.34 −13173.143 −753.096 +1863.423 +3045.504 −15670.448
1.0e−03 95.05 −22.007 −0.672 −11.201 +64.281 +80.737
1.0e−04 94.74 −28.077 −2.302 −15.814 +68.213 +87.911
1.0e−05 93.14 −13.382 −0.584 +1.949 +13.465 +18.070
1.0e−06 89.42 −7.557 +0.128 −0.419 +16.102 +19.781
1.0e−07 84.81 −6.244 −0.088 −0.987 +6.494 +9.490
1.0e−08 76.87 −1.087 −0.031 −0.092 +1.324 −2.495