[yt-users] 14-hour load time for Enzo dataset with VR, vs. 30 minutes with ProjectionPlot?
Stuart Levy
salevy at illinois.edu
Thu Mar 3 13:52:30 PST 2016
We haven't tried parallelizing. We could do that. But the main
problem is, why should it take 30x longer using the volume-rendering
pathway than using ProjectionPlot, both of which should need to examine
all the data (right?)?
On 3/3/16 3:47 PM, Nathan Goldbaum wrote:
> I don't think too many people have done a volume rendering this big,
> so you're likely hitting scaling issues that haven't been looked at
> closely.
>
> Have you tried doing any sort of parallel volume rendering? yt
> supports decomposing in the image plane in parallel using the
> MosaicCamera.
>
> -Nathan
>
> On Thu, Mar 3, 2016 at 3:38 PM, Stuart Levy <salevy at illinois.edu
> <mailto:salevy at illinois.edu>> wrote:
>
> Hello yt people,
>
> We're trying to render imagery of a pretty large Enzo snapshot
> (~160GB, in 330,000 grids in 512 HDF5 domains) with yt-3.3dev.
>
> On a reasonably fast Linux machine, we can do a ProjectionPlot of
> a few variables in about 30 minutes, running single-threaded while
> it scans the data (which is what takes most of the time). Data
> access pattern: we see it reading through each of the HDF5 files
> in numerical order (cpu0000, cpu0001, ...), taking a few seconds
> each, and opening each file exactly once.
>
> On the same machine and same dataset, using the volume rendering
> API, the data-scanning process takes about*14 hours* (not counting
> any rendering time). (On Blue Waters, Kalina using a similar
> dataset couldn't get it to finish within a 24-hour wall-clock
> limit.) Data access pattern: it opens an HDF5 file many times in
> quick succession, then opens another, then opens the previous file
> a bunch more times. I'm guessing it grabs one AMR grid from each
> HDF5 open:
>
> open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0074",
> O_RDONLY) = 3
> open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0075",
> O_RDONLY) = 3
> open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0357",
> O_RDONLY) = 3
> open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0357",
> O_RDONLY) = 3
> open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0357",
> O_RDONLY) = 3
> open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0357",
> O_RDONLY) = 3
> open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0357",
> O_RDONLY) = 3
> open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0357",
> O_RDONLY) = 3
> open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0074",
> O_RDONLY) = 3
> open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0075",
> O_RDONLY) = 3
> open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0235",
> O_RDONLY) = 3
> open("/fe0/deslsst/renaissance/normal/RD0074/RedshiftOutput0074.cpu0357",
> O_RDONLY) = 3
>
> This is trouble. Is there anything we can do to make load times
> less extravagant when using VR on Enzo? What if we ran
> "ds.index" before
>
> I tried running cProfile on it, as in
> python -m cProfile myscript.py ...
> Happy to point anyone at the dataset on our systems or BW, but at
> this scale it's not a very portable problem.
>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org <mailto:yt-users at lists.spacepope.org>
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.spacepope.org_listinfo.cgi_yt-2Dusers-2Dspacepope.org&d=BQMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=hgcBC3x6dKFoTrmFmMYYbKNfiHZlGLKliIidd1LwmHI&m=UyLvCYKpiXUBKLrqXhtKCwKz9CWKL7otbw6OkO6Ci-Q&s=_mbv1RRR0JtnLehEjg-lWjUDf2X8i12-iVZPu8UHSns&e=>
>
>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20160303/e8c2d69e/attachment.htm>
More information about the yt-users
mailing list