[yt-dev] yt for lagrangian hydro

[Matthew Turk]

Hi Matt,

On Fri, Aug 10, 2012 at 12:08 PM, Matt Terry <matt.terry at gmail.com> wrote:
>> Well, so Tom and Jeff let me know that they think this could be
>> avoided if ghost zones were written out.  Does your code have the
>> ability to write out ghost zones?
>
> None of our methods are better than 2nd order in space, so stencils
> only need neighboring cells.  On disk, there is no duplicate data, so
> we'd need to construct ghost zones along block boundaries ourselves.
> as for domain boundary ghost zones, i don't think the ghost zones are
> saved to disk.  i'm not sure if boundary conditions are saved so that
> we can reconstruct them.  adding information to the binary file
> shouldn't be a big deal either way.

Okay, that sounds promising.  We can revisit this after we get an
initial implementation working.

>
>> That would alleviate a substantial
>> amount of the trickiness of spatial locality, as we could just read
>> those in and use them for any finite difference stencil fields.  (And
>> if not, we can just ignore it, and raise a "Don't know how to generate
>> ghost zones" error.)  So I think we can ignore spatial locality for
>> the moment as a problem.
>>
>>>
>>> right now, all my visualization/analysis is slices and profiles, so
>>> not having volume rendering isn't a loss.  having it would be a very
>>> persuasive feature, though.  on that topic, volume rendering will take
>>> a bit of work, since we'll have to rewrite the ray casting routines to
>>> deal with an irregular structured mesh.  i'll say from experience,
>>> that ray trace through Lagrangian meshes is unpleasant.  its all
>>> corner cases.
>>
>> Okay, let's hold off on this for now.  Maybe once we've seen the data
>> we can test some ideas, but let's explore this later.
>
> there are scripts floating around llnl for doing the ray trace.  so i
> don't think i'd need to write this from the ground up.  but i agree,
> baby steps.
>
>>>
>>> assuming we drop volume rendering, this sounds like a few days work to
>>> get all the pipes connected to the right places.  it also sounds like
>>> i should wait for v3 to finish baking before diving in.  this is all
>>> well and good, because it will be a month or so before i really have
>>> the bandwidth to work on this in earnest.
>>
>> Sounds great.  Let's plan on revisiting this -- but I definitely think
>> it's feasible.  And fun.
>
> is yt3 is a state to begin poking around?

Definitely in a state to start poking around, but it's not feature
complete yet.  I'll take this opportunity to update everybody who's
listening as to the current state.

These tickets cover the main milestones in its development, which
essentially cover the necessary pieces of functionality for it to have
parity with the 2.x series:

https://bitbucket.org/yt_analysis/yt/issue/406/milestone-all-patch-based-amr-codes
https://bitbucket.org/yt_analysis/yt/issue/407/milestone-parallelism-is-functional-for
https://bitbucket.org/yt_analysis/yt/issue/408/milestone-all-patch-based-amr-codes
https://bitbucket.org/yt_analysis/yt/issue/409/milestone-all-patch-based-amr-codes
https://bitbucket.org/yt_analysis/yt/issue/410/milestone-all-patch-based-amr-codes
https://bitbucket.org/yt_analysis/yt/issue/411/milestone-octree-based-amr-codes-support
https://bitbucket.org/yt_analysis/yt/issue/412/milestone-octree-based-amr-codes-ramses

All of them are still open, and my milestone scripts are located at:

https://bitbucket.org/MatthewTurk/yt.milestones

The basic gist is that I've implemented support for multiple fluids,
multiple species of particle, flexible chunked IO and geometry
selction.  Patch-based codes are now able to do most all data
selection, slices, basic visualization, but not yet projections or a
handful of the data selections.  (Boolean and extracted data regions,
primarily, but those can be addressed with a bit of work.)  I've just
this morning finished preliminary support for RAMSES octree data,
which in the 2.x series was analyzed using a regridding operation but
is now handled directly (to the tune of an order of magnitude
speedup.)  The new IO / geometry system essentially outsources all
selection routines to two components -- one that identifies based on a
position and a size whether a cell / grid is included in an object,
and another that governs how to divide the on-disk data up to feed to
the geometric selection.  For instance, the RAMSES octree data is
split up into "domain files" so that provides a natural way of
accessing the data.

The next major things that need to be done:

 * Rewrite projections to work with the new generalized geometry (medium)
 * Port the Boxlib codes (Orion, Nyx, Castro, etc) over to the new system (easy)
 * (hardest) Determine the best way to handle spatially-local fields,
which currently are handled by creating 3D arrays including ghost
zones
 * Volume rendering: Sam and I have to brainstorm the best way to handle this.

My gut feeling is that for irregularly shaped or data that's somehow
otherwise outside of the 2.x comfort zone in some way, 3.0 would be a
better starting place.  Right now, I am only developing in the 3.0
branch, and I hope that it will reach a mostly-usable state very soon,
and probably be usable on a day-to-day basis in about a month,
although timelines always slip, so let's say a couple months to be
safe.  If you'd like to poke around, most of the interesting stuff
happens inside the yt/geometry/ directory, specifically in the
selection_routines and the *_geometry_handler.py files.  The Enzo,
FLASH and RAMES readers have been fleshed out, and they're in
yt/frontends/[enzo|flash|ramses]/*.py, but the more interesting areas
are in io.py, in the _read_*_selection routines.

So if you wanted to, you could grab any of the data at
http://yt-project.org/data/ and probably get many but not most/all
operations to work.  I'd also be happy to flesh out the differences,
the timeline, the difficulties and so on a bit more in depth.  But
basically for any non-patchlike data, the overhead to implementing a
fast reader will be a lot lower with the new system.

-Matt

> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org