[yt-dev] Memory leak issue with ParallelHOP code

Geoffrey So gsiisg at gmail.com
Mon Nov 21 15:37:29 PST 2011


Thanks to Britton's addition to support memory output on macs, I was able
to benchmark the different kd-tree options Stephen put in on a small
dataset on my laptop.

The runs were using the default Fortran kd-tree with the version of YT
mentioned in my previous mail (pulling in the changeset of the memory
output for mac), and the new one with the "C" kd-tree ran with the
following tip:

changeset:   4980:a9d0a3e89a59
branch:      yt
tag:         tip
user:        Britton Smith <brittonsmith at gmail.com>
date:        Sun Nov 20 13:44:40 2011 -0500
summary:     Put fallback method for getting memory usage inside the if
tree so

All the runs are on the 256 cube Enzo dataset with 4 mpi tasks

Results:

1)  WholeVol.png
Running parallelHF on the whole volume multiple times in the same script, I
see no noticeable peak memory increase after 11 runs, which is good.

2) RegionFtree.png
Running parallelHF on subvolume 1/64th the size of the whole volume at a
time, I see a steady increase in the minimum memory.  Although there are
peaks at different subvolumes due to the different distribution of
particles at different spacial region, I would expect the minimum to
fluctuate, but it seems to go up constantly.

3) RegionCtree.png
Running paralllelHF on subvolume 1/64th the size of the whole volume, but
with the "C" kd-tree.  Memory seems to increase a bit like the Fortran
case, but it just took way too long so I killed it after the 17th subvolume
was done.

4) CompareF_Ctree.png
This shows that the C kd-tree's memory savings is definitely noticeable.
 However, the cost in time is great.  For the same 17 subvolumes, the
Fortran kd-tree took 7 minutes according to the log, but it took the C
kd-tree method 1 hour and 31 minutes!

5) CompareF_restart.png
This is the Fortran kd-tree ran from subvolume 0 to 63 non-stop, compare to
just starting at subvolume 62 to 63.  I've shifted the restart with zeroes
at the front so you can get a sense of the corresponding value if it were
non-stop.  There's a gap between the memory used, showing less memory at
the same subvolume, but from a fresh restart of python.

After looking at the graphs, I think the memory only builds up when doing
parallelHF with the subvolume defined.  And as long as the code doesn't
crash on the first subvolume, it would probably be faster to just let the
machine run out of memory and restart from where it stopped, than to use
the C kd-tree.

From
G.S.






On Sat, Nov 19, 2011 at 12:31 PM, Britton Smith <brittonsmith at gmail.com>wrote:

> Hey guys,
>
> I just issued a pull request to include more robust calculation of memory
> usage for when the /proc directory doesn't exist.  This is why it doesn't
> work on macs.  If someone wants to check that out and pull it in, then that
> should help Geoffrey.
>
> Britton
>
>
> On Sat, Nov 19, 2011 at 11:48 AM, Stephen Skory <s at skory.us> wrote:
>
>> Geoffrey,
>>
>> > I must have missed something, because I'm re-running the 256 cube on my
>> > lappy but I kept getting the following:
>> > yt : [INFO     ] 2011-11-19 11:03:51,866 All done! Max Memory = 0 MB
>>
>> Yes, I should have warned you. Sorry. The way memory is calculated on
>> Linux doesn't work on Mac OS X. You are welcome to try to find a way
>> that does! The pertinent code is here:
>>
>> http://hg.yt-project.org/yt/src/68227951f9b3/yt/funcs.py#cl-123
>>
>> and google/yahoo/bing/ask jeeves/lycos/altavista/stackoverflow/etc..
>> are your friend. Good luck, and if you find success, make a pull
>> request and we'll add it!
>>
>> --
>> Stephen Skory
>> s at skory.us
>> http://stephenskory.com/
>> 510.621.3687 (google voice)
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20111121/45fab3f4/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: WholeVol.png
Type: image/png
Size: 48373 bytes
Desc: not available
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20111121/45fab3f4/attachment.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: RegionFtree.png
Type: image/png
Size: 55925 bytes
Desc: not available
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20111121/45fab3f4/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: RegionCtree.png
Type: image/png
Size: 48695 bytes
Desc: not available
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20111121/45fab3f4/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: compareF_Ctree.png
Type: image/png
Size: 80130 bytes
Desc: not available
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20111121/45fab3f4/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: compareF_restart.png
Type: image/png
Size: 62230 bytes
Desc: not available
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20111121/45fab3f4/attachment-0004.png>


More information about the yt-dev mailing list