[yt-users] RAMSES multiple sims memory issue

Nathan Goldbaum nathan12343 at gmail.com
Wed Dec 14 07:17:39 PST 2016


On Wed, Dec 14, 2016 at 5:30 AM, Thor Andreas Seiff Ellewsen <
tellewsen at gmail.com> wrote:

> Hi all,
>
> I'm working on a project where I'm comparing different RAMSES simulations.
> However I'm running into a problem with memory.
> It seems that yt loads the whole simulation into memory even though I'm
> only interested in a number of dark matter halos which I have the
> coordinates for.
>

Yes, this is a limitation of our current approach for octree data, we must
build a global octree for indexing purposes. For large simulations this can
be a prohibitive memory cost. We have long-term plans to fix this but as
far as I know no one is actively working on it.


>
> It also seems like it doesn't drop the first simulation out of memory when
> I'm done making the figures I want for that one.
>

> This quickly fills up the memory.
>
> You can find the relevant parts of my code at the pastebin at
> http://paste.yt/p4946.html
> I'm running yt version 3.3.2
>
> What am I doing wrong?
>

In principle this shouldn't happen but it's entirely possible there is a
bug somewhere. In the python memory model, an object is retained in memory
until no other object have a reference to it, at which point it is garbage
collected. I don't see anything in your script that would cause references
to old datasets to be retained, so it might be the case that yt is
inadvertantly retaining a reference to datasets in a ramses time series
when it shouldn't be or if we need the reference that we should be using a
weakref. Unfortunately it looks like we don't have any Ramses time series
datasets available in our repository of public test datasets (
http://yt-project.org/data/) so I don't have a way of reproducing this
behavior. Is there any chance you can make a ramses time series available
for testing? You might find the yt curldrop to be useful for sharing a
large dataset:

https://docs.hub.yt/services.html#curldrop

A test that this is isolated to the time series object would be to instead
use a loop over filenames, loading a single filename in each loop
iteration, and then explicitly deleting the dataset object using the `del`
statement at the end of the loop. In principle this shouldn't be necessary
but it might help reveal where the issue is localized to.


>
> Also, is there a way to change the axis names?
> I'm making some dark matter density profiles using the
> ('deposit','io_density') field and this gives IO Density on the y axis, but
> I just want it to say Dark matter density so my readers actually understand
> what I'm talking about.
>

For the ProjectionPlot you can use the set_colorbar_label method:

http://yt-project.org/doc/reference/api/generated/yt.visualization.plot_window.ProjectionPlot.set_colorbar_label.html#yt.visualization.plot_window.ProjectionPlot.set_colorbar_label


>
> Thanks for helping
>
> Best,
> Andreas Ellewsen
>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20161214/a0850fda/attachment.htm>


More information about the yt-users mailing list