[yt-dev] A New Fontend

[Nathan Goldbaum]

Hey Stewart,

Take a look at the FLASH frontend, specifically
yt/frontends/flash/data_structures.py.

In FLASH, the IO is handled by relying on h5py.  If you look at the
beginning of FLASHStaticOutput.__init__, you'll see that a _handle
attribute is assigned to the instance. This _handle is a reference to the
open h5py File object.  Optionally, FLASH allows a particle file which is
written to disk seperately from the main hydro data file, so that is
optionally opened as well and the File handle is assigned to
_particle_handle.

FLASHStaticOutput._handle is then used many times elsewhere to read data
from the output file and to create an in-memory view of the FLASH AMR data
structure.  The AMR hierarchy is handled by FLASHHierarchy, and data IO is
also logically handled there, although via a FLASHIOHandler instance.  The
IOHandler is defined using a bit of metaclass magic via the io_registry,
which is set up in the IOHandler base class in yt/utilities/io_handler.py.
 If you look in yt/frontends/flash/io.py, you'll find the definition for
FLASHIOHandler, which implements all the IO for hydro data.  Since it uses
h5py, this all happens via a dict-like interface.

I'm not sure if there is an exhaustive list of things that a StaticOutput
object needs to implement.  There is probably a minimal number of
parameters that the rest of the code needs to know about and that list of
things should probably be written down somewhere.  I would look at a few of
the frontends (FLASH and Orion are simple, relatively clean examples; the
enzo frontend is unfortunately a bit messy, there's been a lot of work
recently on the Ramses, SPH, ART, and ARTIO frontends) and try to implement
functionality that is common between the codes.  A couple examples spring
to mind.  All StaticOutput sublcasses need to define a parameters dict and
a minimal set of parameters needed elsewhere.  Similarly, there needs to be
a units dict which defines unit conversions.

Frontends also define a fields.py file.  Right now, frontends needs to
implement fields with enzo field names (so, 'Density', 'Pressure',
'x-velocity', 'GasEnergy', 'TotalEnergy', etc.) because universal_fields.py
assumes these field names are defined and in CGS units.  There's an effort
to make this situation better, both by providing an updated list of field
names, and a unit system, but that's currently on the backburner while Matt
pushes ahead on necessary yt-3.0 infrastructure work.

You should also supply on-disk fields without conversions to CGS and with
native field names - this is how we distinguish between 'yt' fields, which
always have units and enzo field names, and on-disk fields, which are
always in code units. As a curious aside, this ad hoc system is broken for
enzo, making it a little non-trivial to access on-disk enzo data.  Like I
said, the units refactor will make this better :)

I'll let Matt explain the distinction between known fields and unknown
fields... ;)

Anyway, like Matt said on IRC, please let us know about sticking points in
this process.  We really should have better documentation for how to write
a frontend and it would be cool if we could streamline the process or
simplify by refactoring and cleaning up the frontend code.

HTH,

-Nathan


On Fri, Aug 23, 2013 at 4:37 PM, Stuart Mumford <stuart at mumford.me.uk>wrote:

> Hello,
>
> First an introduction, I am Stuart Mumford I am doing a solar physics PhD
> at Sheffield University in the UK. I work primarily on MHD wave simulations
> of the solar atmosphere studying energy propagation and wave excitation.
> The code SAC (http://www.aanda.org/10.1051/0004-6361:200809800) solves
> for perturbations ontop of a fixed background to deal with the very wide
> range of the parameters e.g. density and temperature in the solar
> atmosphere. I also attended SciPy this year, where I met some of you and
> discovered yt, mainly as a consequence of my contributions to SunPy (
> www.sunpy.org) which is a solar physics data analysis library.
>
> So that's that over with, now having been playing with yt on and off since
> SciPy I started using it properly this week, as the need arose. I am very
> impressed with yt's capabilities and I want to be able to make better use
> of it. To that end I am trying to write a frontend for SAC, as I have been
> speaking to some of you about in IRC (I am Cadair in IRC and on GH).
>
> I have some questions regarding this, as far as I can ascertain a frontend
> provides a mapper between the stored data and a abstracted format that yt
> understands? I have started looking in the code for FLASH and Stream to try
> and get an idea where to start and not got very far. So some questions to
> start:
>
> Where should the file handling be done?
> Where is the file opened?
> What things should the StaticOutput subclass define?
>
> Thanks once again for your help on IRC over the last few days and I hope
> you can point me in the right direction.
>
> Stuart
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130823/6c567e3a/attachment.htm>