[yt-dev] Zombie jobs on eudora?

[Matthew Turk]

This shouldn't be preferentially affecting flash, though. I think it is a
leftover from when we moved to unify index; when the hierarchy had the
classes, pf was already a weakref.
On Jun 11, 2014 5:46 PM, "Nathan Goldbaum" <nathan12343 at gmail.com> wrote:

> After noticing that the FLASHDataset objects seem to have backreferences
> to the data object aliases that are attached to the dataset, I made the
> following modification:
>
> diff -r d1de2160a4a8 yt/data_objects/static_output.py
> --- a/yt/data_objects/static_output.py  Wed Jun 11 13:25:17 2014 -0700
> +++ b/yt/data_objects/static_output.py  Wed Jun 11 15:42:56 2014 -0700
> @@ -498,7 +498,7 @@
>                  continue
>              cname = cls.__name__
>              if cname.endswith("Base"): cname = cname[:-4]
> -            self._add_object_class(name, cname, cls, {'pf':self})
> +            self._add_object_class(name, cname, cls,
> {'pf':weakref.proxy(self)})
>          if self.refine_by != 2 and hasattr(self, 'proj') and \
>              hasattr(self, 'overlap_proj'):
>              mylog.warning("Refine by something other than two: reverting
> to"
>
> After doing so, I find that the FLASHDataset objects are ending up in
> gc.garbage. The memory is still leaking, but now the garbage collector is
> able to at least find the objects that are supposed to be collected.
>
> From my reading of the docs things will only end up in gc.garbage if they
> have a __del__ method that doesn't actually free the object.  Any idea what
> might be happening here?
>
> -Nathan
>
>
> On Wed, Jun 11, 2014 at 3:37 PM, Matthew Turk <matthewturk at gmail.com>
> wrote:
>
>> On Wed, Jun 11, 2014 at 5:28 PM, Nathan Goldbaum <nathan12343 at gmail.com>
>> wrote:
>> > On Wed, Jun 11, 2014 at 3:04 PM, Matthew Turk <matthewturk at gmail.com>
>> wrote:
>> >>
>> >>
>> >> Please do, yeah. That should help us track down the memory increase.
>> >> It's possible FLASH needs additional work; have you checked the
>> >> refcounts for its grid objects?
>> >>
>> >
>> > With this script: http://paste.yt-project.org/show/4779/
>> >
>> > I also see steadily increasing memory usage, although it's not clear if
>> > that's just because each successive Enzo dataset is larger than the one
>> > before. The peak memory usage is 230 MB, so substantially better than
>> the
>> > FLASH dataset.
>>
>> My guess is there is a leak in flash. You can evaluate whether it is the
>> data size by looking at what the memory use is if you just load the final
>> one or the final ten.
>>
>> When I have a minute I will try to replicate here. I suspect it's not
>> important that it's cylindrical.
>>
>> >
>> > I can privately share the FLASH dataset Philip was originally having
>> trouble
>> > with if that will help.
>> >
>> > The root grid seems to have a refcount of 7. I'm not sure how many of
>> those
>> > references were generated by objgraph itself.
>> >
>> > _______________________________________________
>> > yt-dev mailing list
>> > yt-dev at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>>
>>
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20140611/3047d028/attachment.htm>