[yt-users] TotalQuantity vs. summing manually

Geoffrey So gsiisg at gmail.com
Sun Dec 22 16:51:06 PST 2013


Thanks, Matt, John, some follow-up

"In any case, the difference in the total quantity is about a few percent
it looks like. Are you attempting to quantify something with that level of
sensitivity?"
- I'm trying to compare star particle mass, and if I use TotalQuantity and
StarMassMsun comparing to summing the star particle's mass.  For small
halos that are only a few cells, the difference can be quite high ~O(10%).
 So from the perspective of actually particles inside the halo, I should
use the particle list inside the 3D object, right?  Same thing with dark
matter particle mass?

"The output isn't what you might expect, because numpy uses the dtype of
the array for the accumulation."
- I don't quite understand why would float32 accumulation with sum() not
equal to the actual thing but casting it as float64 does (isn't 2^32 <
512^3 so no overflow should occur)?  Should I always use
arr.sum(dtype='float64') when using sum() with possible large datasets?

From
G.S.



On Sat, Dec 21, 2013 at 5:43 AM, Matthew Turk <matthewturk at gmail.com> wrote:

> Hi Geoffrey,
>
> On Fri, Dec 20, 2013 at 9:02 PM, Geoffrey So <gsiisg at gmail.com> wrote:
> > Hi all,
> >
> > I found that using a Enzo dataset I was getting slightly different
> numbers
> > when
> > 1) using the list of dark matter particles selected by creation_time <
> 0.0
> >
> > In [97]: sph_dm = sph['creation_time'] < 0.0
> > In [98]: print "%12.12e" % (sph['ParticleMassMsun'][sph_dm]).sum()
> > 1.211311468567e+11
> >
> > 2) compared to summing the dark matter particles inside a 3D container
> with
> > TotalQuantity
> >
> > In [101]: print "%12.12e" %
> > (sph.quantities['TotalQuantity']('Dark_Matter_Density')[0]*vol/Msun)
> > 1.188937185993e+11
> >
> > I'm wondering if the field and particles are handled differently when
> being
> > counted as inside or outside the 3D container?
> >
>
> I think John's answer is completely correct, and likely the dominant
> problem, but there's a funny thing that happens if you're not extra
> careful with big arrays.  As an example, something that I ran into a
> couple years ago:
>
> # This is an array filled with ones.
> arr = numpy.ones((512, 512, 512), dtype="float32")
> print arr.sum()
> print arr.size
> print arr.sum(dtype = "float64")
>
> The output isn't what you might expect, because numpy uses the dtype
> of the array for the accumulation.  The quantities -- thanks to Doug
> Rudd -- are uniformly careful about upgrading to 64 bits the
> accumulators used for quantities.  Again, I don't think this is
> dominant, but it is something to be aware of.
>
> -Matt
>
> > From
> > G.S.
> >
> > _______________________________________________
> > yt-users mailing list
> > yt-users at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20131222/76681fd2/attachment.htm>


More information about the yt-users mailing list