[yt-users] How to efficiently control the memory in yt with large simulation?

[Matthew Turk]

On Fri, Dec 6, 2013 at 2:52 PM, Geoffrey So <gsiisg at gmail.com> wrote:
> I believe you can access rectangular sub volume of the data with something
> like
>
> sv = pf.h.region([0.5]*3, [0.21, .21, .72], [.28, .28, .79])
>
> Shown here when running halo finder on a sub volume
> http://yt-project.org/docs/2.6/analyzing/analysis_modules/running_halofinder.html?highlight=sub%20region
>
> And you can use other 3D objects like spheres, ellipsoids etc, for a thin
> slice of the data you can make your rectangular region small in one of the
> dimension.
>
> When making plots just pass in the sv 3D sub volume object instead of the
> usual pf for plotting the entire simulation.

This will work if the Octree is parsed on demand; what I think Junhwan
is running into right now is that the entire octree is loaded at
startup.  So you end up with a memory constant that gets added on for
each process; this will reduce the *additional* overhead of loading
data, but he still has memory for each leaf node of the oct.

>
> Hope that helps.
>
>
>
> From
>
> G.S.
>
>
>
>
>
> On Fri, Dec 6, 2013 at 8:41 AM, Junhwan Choi (최준환) <choi.junhwan at gmail.com>
> wrote:
>>
>> Thank you Matt and Romain,
>>
>> Now, I understand it is a more fundamental issue in the yt with
>> Ramses, and I need to wait yt to be updated.
>> Meanwhile, is there any way I can access small fraction of the
>> simulation data (such as thin slice of the data) and make some
>> visualization plots such as thin projection and slice?
>> If so, I can reduce the required memory.
>> Is it possible?
>>
>> Thank you,
>> Junhwan
>>
>>
>> On Fri, Dec 6, 2013 at 8:56 AM, Matthew Turk <matthewturk at gmail.com>
>> wrote:
>> > On Fri, Dec 6, 2013 at 9:54 AM, Romain Teyssier
>> > <romain.teyssier at gmail.com> wrote:
>> >> One possibility would be to define a bounding box that yt uses to
>> >> upload
>> >> ramses data only when contained in this bounding box.
>> >> This can be done efficiently using the Hilbert key.
>> >>
>> >> Romain
>> >
>> > That would be good, and we could probably port the code Doug Rudd
>> > wrote to use SFC logic for detecting which zones to parse over to
>> > RAMSES, as well.
>> >
>> > Today is out for me on this, because of other things, but with this I
>> > think we might be able to make it work in the near future, like
>> > mid-next week, assuming all goes well otherwise.
>> >
>> > -Matt
>> >
>> >>
>> >> On 06 Dec 2013, at 15:49, Matthew Turk <matthewturk at gmail.com> wrote:
>> >>
>> >> Hi Junhwan,
>> >>
>> >> There's some more background on this issue here:
>> >>
>> >>
>> >> http://lists.spacepope.org/pipermail/yt-users-spacepope.org/2013-November/004265.html
>> >>
>> >> Basically, what it amounts to is:
>> >>
>> >> * Right now RAMSES uses too much memory and duplicates the full mesh
>> >> on every processor
>> >> * This is not how it will always be
>> >> * Unfortunately changing this can't be prioritized in the next couple
>> >> weeks
>> >>
>> >> Your mesh is *particularly* large for what we've dealt with before.
>> >> As it stands, I would be surprised if it will work.  The fix for this
>> >> would be to change the Octrees to only parse on demand rather than at
>> >> instantiation of the RAMSESHierarchy.  Sam Geen and I have talked a
>> >> bit about this, and he may be interested in working on it.  It's a
>> >> change that also will happen for N-body datasets (differently) and
>> >> it's definitely planned to come, but it isn't being prioritized at
>> >> this very moment because of other pressing concerns.
>> >>
>> >> On Thu, Dec 5, 2013 at 10:47 PM, Junhwan Choi (최준환)
>> >> <choi.junhwan at gmail.com> wrote:
>> >>
>> >> Hi all,
>> >>
>> >> I try to make some visualizations (density/temperature project) for
>> >> large simulation with yt.
>> >> The simulation is 4096^3 unigrid Ramses simulation.
>> >>
>> >> I try to implement the basics density and temperature projection plot
>> >> with following script and I got memory problem and yt run is crashed.
>> >> =====
>> >> from yt.mods import *
>> >>
>> >> ds = load("../output_00048/info_00048.txt", fields =
>> >> ["Density","x-velocity",
>> >> "y-velocity","z-velocity","Pressure","Metallicity","Rad"])
>> >> center = [0., 0., 0.]
>> >>
>> >> pw = ProjectionPlot(ds, "x", ("gas", "Density"),
>> >> weight_field="Density", center=center)
>> >> pw.zoom(1.01)
>> >> pw.save("allviewGas")
>> >> pw = ProjectionPlot(ds, "y", ("gas", "Density"),
>> >> weight_field="Density", center=center)
>> >> pw.zoom(1.01)
>> >> pw.save("allviewGas")
>> >> pw = ProjectionPlot(ds, "z", ("gas", "Density") ,
>> >> weight_field="Density", center=center)
>> >> pw.zoom(1.01)
>> >> pw.save("allviewGas")
>> >>
>> >> pw = ProjectionPlot(ds, "x", "Temperature", weight_field="Density",
>> >> center=center)
>> >> pw.zoom(1.01)
>> >> pw.save("allviewGas")
>> >> pw = ProjectionPlot(ds, "y", "Temperature", weight_field="Density",
>> >> center=center)
>> >> pw.zoom(1.01)
>> >> pw.save("allviewGas")
>> >> pw = ProjectionPlot(ds, "z", "Temperature" , weight_field="Density",
>> >> center=center)
>> >> pw.zoom(1.01)
>> >> pw.save("allviewGas")
>> >> ====
>> >>
>> >> Is there any way to reduce the memory usage when I make visualization?
>> >> If I use slice instead of projection, can I save the memory?
>> >> I saw there is parallelization for yt. Does parallelization distribute
>> >> the data at the beginning of the read?
>> >> (But, I prefer to do so w/o parallelization at this moment.)
>> >>
>> >>
>> >> In principle, yes, slices will considerably reduce the memory, modulo
>> >> the overhead from having all your octrees in memory at once -- which
>> >> will be considerable.
>> >>
>> >> I will spend some time thinking if there's a hotfix we can apply to
>> >> make this work for you right now, as is, but I suspect it may be a
>> >> little time until it can be properly implemented.
>> >>
>> >> -Matt
>> >>
>> >>
>> >> Thank you in advance,
>> >> Junhwan
>> >> _______________________________________________
>> >> yt-users mailing list
>> >> yt-users at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> >>
>> >> _______________________________________________
>> >> yt-users mailing list
>> >> yt-users at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> yt-users mailing list
>> >> yt-users at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> >>
>> > _______________________________________________
>> > yt-users mailing list
>> > yt-users at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>