[yt-users] 14-hour load time for Enzo dataset with VR, vs. 30 minutes with ProjectionPlot?

Stuart Levy salevy at illinois.edu
Thu Mar 3 13:52:30 PST 2016


We haven't tried parallelizing.   We could do that.    But the main 
problem is, why should it take 30x longer using the volume-rendering 
pathway than using ProjectionPlot, both of which should need to examine 
all the data (right?)?

On 3/3/16 3:47 PM, Nathan Goldbaum wrote:
> I don't think too many people have done a volume rendering this big, 
> so you're likely hitting scaling issues that haven't been looked at 
> closely.
>
> Have you tried doing any sort of parallel volume rendering? yt 
> supports decomposing in the image plane in parallel using the 
> MosaicCamera.
>
> -Nathan
>
> On Thu, Mar 3, 2016 at 3:38 PM, Stuart Levy <salevy at illinois.edu 
> <mailto:salevy at illinois.edu>> wrote:
>
>     Hello yt people,
>
>     We're trying to render imagery of a pretty large Enzo snapshot
>     (~160GB, in 330,000 grids in 512 HDF5 domains) with yt-3.3dev.
>
>     On a reasonably fast Linux machine, we can do a ProjectionPlot of
>     a few variables in about 30 minutes, running single-threaded while
>     it scans the data (which is what takes most of the time).   Data
>     access pattern: we see it reading through each of the HDF5 files
>     in numerical order (cpu0000, cpu0001, ...), taking a few seconds
>     each, and opening each file exactly once.
>
>     On the same machine and same dataset, using the volume rendering
>     API, the data-scanning process takes about*14 hours* (not counting
>     any rendering time).   (On Blue Waters, Kalina using a similar
>     dataset couldn't get it to finish within a 24-hour wall-clock
>     limit.)   Data access pattern: it opens an HDF5 file many times in
>     quick succession, then opens another, then opens the previous file
>     a bunch more times.  I'm guessing it grabs one AMR grid from each
>     HDF5 open:
>
>         open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0074",
>         O_RDONLY) = 3
>         open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0075",
>         O_RDONLY) = 3
>         open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0357",
>         O_RDONLY) = 3
>         open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0357",
>         O_RDONLY) = 3
>         open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0357",
>         O_RDONLY) = 3
>         open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0357",
>         O_RDONLY) = 3
>         open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0357",
>         O_RDONLY) = 3
>         open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0357",
>         O_RDONLY) = 3
>         open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0074",
>         O_RDONLY) = 3
>         open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0075",
>         O_RDONLY) = 3
>         open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0235",
>         O_RDONLY) = 3
>         open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0357",
>         O_RDONLY) = 3
>
>     This is trouble.  Is there anything we can do to make load times
>     less extravagant when using VR on Enzo?   What if we ran
>     "ds.index" before
>
>     I tried running cProfile on it, as in
>        python -m cProfile myscript.py ...
>     Happy to point anyone at the dataset on our systems or BW, but at
>     this scale it's not a very portable problem.
>
>
>
>     _______________________________________________
>     yt-users mailing list
>     yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
>     http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>     <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.spacepope.org_listinfo.cgi_yt-2Dusers-2Dspacepope.org&d=BQMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=hgcBC3x6dKFoTrmFmMYYbKNfiHZlGLKliIidd1LwmHI&m=UyLvCYKpiXUBKLrqXhtKCwKz9CWKL7otbw6OkO6Ci-Q&s=_mbv1RRR0JtnLehEjg-lWjUDf2X8i12-iVZPu8UHSns&e=>
>
>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20160303/e8c2d69e/attachment.htm>


More information about the yt-users mailing list