<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
As a followup, I believe there is at least once instance where we would not want to use float64 as the accumulator: if the field itself is integer and the sum does not overflow. Something like this might be better:
<div><br>
</div>
<div>if 'int' in str(data[field].dtype) :</div>
<div> totals.append(data[field].sum(dtype=np.int64))</div>
<div>elif 'float' in str(data[field].dtype) :</div>
<div> totals.append(data[field].sum(dtype=np.float64))</div>
<div>else :</div>
<div> totals.append(data[field].sum())</div>
<div><br>
</div>
<div>Using np.float64 in the case of integers as well would alleviate the overflow problem, but would lead to approximate results where the user might expect an exact answer.</div>
<div><br>
</div>
<div>
<div apple-content-edited="true">
<div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">
<div>Douglas Rudd</div>
<div>Scientific Computing Consultant<br>
Research Computing Center<br>
</div>
<div><a href="mailto:drudd@uchicago.edu">drudd@uchicago.edu</a></div>
</div>
<br class="Apple-interchange-newline">
<br class="Apple-interchange-newline">
</div>
<br>
<div>
<div>On Jul 10, 2013, at 6:26 AM, Britton Smith <<a href="mailto:brittonsmith@gmail.com">brittonsmith@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div dir="ltr">
<div>
<div>Hi Doug,<br>
<br>
</div>
I don't seem any reason why we wouldn't want to fix this to always use higher precision. Could you issue a PR?<br>
<br>
</div>
Britton<br>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Wed, Jul 10, 2013 at 2:36 AM, Douglas Harvey Rudd <span dir="ltr">
<<a href="mailto:drudd@uchicago.edu" target="_blank">drudd@uchicago.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi all,<br>
<br>
I'm working on getting the ARTIO frontend working with a script used by the AGORA collaboration to measure density profiles. This script uses the "TotalQuantity" quantity to sum all particle masses within a given sphere, then differences a nested set of spheres
to measure the mean density in spherical shells.<br>
<br>
I noticed a 2% difference at large radius from my own profiling routines, and tracked it down to the precision of the sum, which used the same precision as the input variable, float32 for ARTIO. Since there are a large number of particles, summing 1e5 x 10^-7
floats led to O(1%) error.<br>
<br>
While this is clearly not an ideal way to measure density profiles, it's probably a good idea to help avoid such issues in future by changing TotalQuantity (and other places, like _TotalMass) to use a 64-bit accumulator, e.g.<br>
totals.append(data[field].sum(dtype=np.float64))<br>
<br>
The alternative is to promote all float32 input variables to float64, which is wasteful of memory, or otherwise warn the user that they are likely to encounter loss of precision.<br>
<br>
Douglas Rudd<br>
Scientific Computing Consultant<br>
Research Computing Center<br>
<a href="mailto:drudd@uchicago.edu">drudd@uchicago.edu</a><br>
<br>
<br>
<br>
_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
</blockquote>
</div>
<br>
</div>
_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org<br>
</blockquote>
</div>
<br>
</div>
</body>
</html>