[yt-dev] A New Fontend

Stuart Mumford stuart at mumford.me.uk
Sat Aug 24 00:46:45 PDT 2013


Hello,

Just a quick question that jumps out at me, it is possible to not use yt's
own hdf5 opening code and do it all in the frontend because I have a
library that abstracts the two data formats we have for the simulations,
only one is hdf5.

Stuart


On 24 August 2013 06:51, Nathan Goldbaum <nathan12343 at gmail.com> wrote:

> Hey Stewart,
>
> Take a look at the FLASH frontend, specifically
> yt/frontends/flash/data_structures.py.
>
> In FLASH, the IO is handled by relying on h5py.  If you look at the
> beginning of FLASHStaticOutput.__init__, you'll see that a _handle
> attribute is assigned to the instance. This _handle is a reference to the
> open h5py File object.  Optionally, FLASH allows a particle file which is
> written to disk seperately from the main hydro data file, so that is
> optionally opened as well and the File handle is assigned to
> _particle_handle.
>
> FLASHStaticOutput._handle is then used many times elsewhere to read data
> from the output file and to create an in-memory view of the FLASH AMR data
> structure.  The AMR hierarchy is handled by FLASHHierarchy, and data IO is
> also logically handled there, although via a FLASHIOHandler instance.  The
> IOHandler is defined using a bit of metaclass magic via the io_registry,
> which is set up in the IOHandler base class in yt/utilities/io_handler.py.
>  If you look in yt/frontends/flash/io.py, you'll find the definition for
> FLASHIOHandler, which implements all the IO for hydro data.  Since it uses
> h5py, this all happens via a dict-like interface.
>
> I'm not sure if there is an exhaustive list of things that a StaticOutput
> object needs to implement.  There is probably a minimal number of
> parameters that the rest of the code needs to know about and that list of
> things should probably be written down somewhere.  I would look at a few of
> the frontends (FLASH and Orion are simple, relatively clean examples; the
> enzo frontend is unfortunately a bit messy, there's been a lot of work
> recently on the Ramses, SPH, ART, and ARTIO frontends) and try to implement
> functionality that is common between the codes.  A couple examples spring
> to mind.  All StaticOutput sublcasses need to define a parameters dict and
> a minimal set of parameters needed elsewhere.  Similarly, there needs to be
> a units dict which defines unit conversions.
>
> Frontends also define a fields.py file.  Right now, frontends needs to
> implement fields with enzo field names (so, 'Density', 'Pressure',
> 'x-velocity', 'GasEnergy', 'TotalEnergy', etc.) because universal_fields.py
> assumes these field names are defined and in CGS units.  There's an effort
> to make this situation better, both by providing an updated list of field
> names, and a unit system, but that's currently on the backburner while Matt
> pushes ahead on necessary yt-3.0 infrastructure work.
>
> You should also supply on-disk fields without conversions to CGS and with
> native field names - this is how we distinguish between 'yt' fields, which
> always have units and enzo field names, and on-disk fields, which are
> always in code units. As a curious aside, this ad hoc system is broken for
> enzo, making it a little non-trivial to access on-disk enzo data.  Like I
> said, the units refactor will make this better :)
>
> I'll let Matt explain the distinction between known fields and unknown
> fields... ;)
>
> Anyway, like Matt said on IRC, please let us know about sticking points in
> this process.  We really should have better documentation for how to write
> a frontend and it would be cool if we could streamline the process or
> simplify by refactoring and cleaning up the frontend code.
>
> HTH,
>
> -Nathan
>
>
> On Fri, Aug 23, 2013 at 4:37 PM, Stuart Mumford <stuart at mumford.me.uk>wrote:
>
>> Hello,
>>
>> First an introduction, I am Stuart Mumford I am doing a solar physics PhD
>> at Sheffield University in the UK. I work primarily on MHD wave simulations
>> of the solar atmosphere studying energy propagation and wave excitation.
>> The code SAC (http://www.aanda.org/10.1051/0004-6361:200809800) solves
>> for perturbations ontop of a fixed background to deal with the very wide
>> range of the parameters e.g. density and temperature in the solar
>> atmosphere. I also attended SciPy this year, where I met some of you and
>> discovered yt, mainly as a consequence of my contributions to SunPy (
>> www.sunpy.org) which is a solar physics data analysis library.
>>
>> So that's that over with, now having been playing with yt on and off
>> since SciPy I started using it properly this week, as the need arose. I am
>> very impressed with yt's capabilities and I want to be able to make better
>> use of it. To that end I am trying to write a frontend for SAC, as I have
>> been speaking to some of you about in IRC (I am Cadair in IRC and on GH).
>>
>> I have some questions regarding this, as far as I can ascertain a
>> frontend provides a mapper between the stored data and a abstracted format
>> that yt understands? I have started looking in the code for FLASH and
>> Stream to try and get an idea where to start and not got very far. So some
>> questions to start:
>>
>> Where should the file handling be done?
>> Where is the file opened?
>> What things should the StaticOutput subclass define?
>>
>> Thanks once again for your help on IRC over the last few days and I hope
>> you can point me in the right direction.
>>
>> Stuart
>>
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130824/f4ca81c3/attachment.htm>


More information about the yt-dev mailing list