[Yt-dev] 1024^3 amr hop problem

Matthew Turk matthewturk at gmail.com
Wed Apr 29 23:34:53 PDT 2009


Hi Stephen,

I think this is the right location for this -- I believe it's a bug,
and we should be addressing it as such.  Are you using the old version
of HOP, without the patch I constructed to do single-array addressing
of particles?

Can you tell me a bit more about the results of running top?  How much
free memory was there?  Is there any reason to believe that the
distribution of particles would be enough to compensate for this and
blow out the ram on another node?  As far as it being a bug, I
honestly don't know -- lists should always have an append method.  The
only reason I can think of this being incorrect would be some problem
in the C-backend, which could be manifesting in this way.  I'll ask
around.

It's unclear to me if this is related, but I believe that HOP, right
now, is too memory intensive.  I think we can fix this, but it will
require a bit of thought.  Britton has been running into the same
problems that you have.  I'll see if I can profile the memory and get
an idea what's going on.

Please let us know what happens on Kraken.  This is unacceptable and
we need to fix it.

-Matt

On Wed, Apr 29, 2009 at 10:36 PM, Stephen Skory <stephenskory at yahoo.com> wrote:
>
> (I'd send this to -users, but since we're not emphasizing parallel hop just yet, I'll put it here.)
>
> I am trying to run hop on Ranger on a 1024^3 AMR dataset I got from Robert Harkness. I've been running it with a varying number of threads, always greater than 64. I'm pretty sure it's not a memory problem, I ssh-ed into the head node and ran 'top.' I think I'll try this on Kraken tomorrow, but for now, can any of you make heads or tails of the error message below:
>
> AttributeError: 'list' object has no attribute 'append'
> When can a list not have 'append' as an attribute?
>
> http://paste.enzotools.org/show/112/
>
> I can sucessfully run parallel hop on L7 with this exact same install of yt.
>
> The script I'm running is dead simple:
>
> from yt.mods import *
>
>
> pf = load("DD0082")
>
> hop = HaloFinder(pf,padding=0.02)
>
> hop.write_out(filename="benchmark-hop.out")
>
>
> Britton, I know you've been doing some large-scale stuff lately. Have you run hop on something this large?
>
> Thanks!
>
>  _______________________________________________________
> sskory at physics.ucsd.edu           o__  Stephen Skory
> http://physics.ucsd.edu/~sskory/ _.>/ _Graduate Student
> ________________________________(_)_\(_)_______________
>
> _______________________________________________
> Yt-dev mailing list
> Yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>



More information about the yt-dev mailing list