[yt-dev] A New Fontend

Sat Aug 24 00:50:20 PDT 2013

On 24.08.2013 09:46, Stuart Mumford wrote:
> Hello,
> 
> Just a quick question that jumps out at me, it is possible to not use yt's
> own hdf5 opening code and do it all in the frontend because I have a
> library that abstracts the two data formats we have for the simulations,
> only one is hdf5.

Hi Stuart,
I doesn't matter what you use to get data from disk to memory in
your_frontend/io.py as long as you get the job done ;)
Cheers,
Kacper

> 
> Stuart
> 
> 
> On 24 August 2013 06:51, Nathan Goldbaum <nathan12343 at gmail.com> wrote:
> 
>> Hey Stewart,
>>
>> Take a look at the FLASH frontend, specifically
>> yt/frontends/flash/data_structures.py.
>>
>> In FLASH, the IO is handled by relying on h5py.  If you look at the
>> beginning of FLASHStaticOutput.__init__, you'll see that a _handle
>> attribute is assigned to the instance. This _handle is a reference to the
>> open h5py File object.  Optionally, FLASH allows a particle file which is
>> written to disk seperately from the main hydro data file, so that is
>> optionally opened as well and the File handle is assigned to
>> _particle_handle.
>>
>> FLASHStaticOutput._handle is then used many times elsewhere to read data
>> from the output file and to create an in-memory view of the FLASH AMR data
>> structure.  The AMR hierarchy is handled by FLASHHierarchy, and data IO is
>> also logically handled there, although via a FLASHIOHandler instance.  The
>> IOHandler is defined using a bit of metaclass magic via the io_registry,
>> which is set up in the IOHandler base class in yt/utilities/io_handler.py.
>>  If you look in yt/frontends/flash/io.py, you'll find the definition for
>> FLASHIOHandler, which implements all the IO for hydro data.  Since it uses
>> h5py, this all happens via a dict-like interface.
>>
>> I'm not sure if there is an exhaustive list of things that a StaticOutput
>> object needs to implement.  There is probably a minimal number of
>> parameters that the rest of the code needs to know about and that list of
>> things should probably be written down somewhere.  I would look at a few of
>> the frontends (FLASH and Orion are simple, relatively clean examples; the
>> enzo frontend is unfortunately a bit messy, there's been a lot of work
>> recently on the Ramses, SPH, ART, and ARTIO frontends) and try to implement
>> functionality that is common between the codes.  A couple examples spring
>> to mind.  All StaticOutput sublcasses need to define a parameters dict and
>> a minimal set of parameters needed elsewhere.  Similarly, there needs to be
>> a units dict which defines unit conversions.
>>
>> Frontends also define a fields.py file.  Right now, frontends needs to
>> implement fields with enzo field names (so, 'Density', 'Pressure',
>> 'x-velocity', 'GasEnergy', 'TotalEnergy', etc.) because universal_fields.py
>> assumes these field names are defined and in CGS units.  There's an effort
>> to make this situation better, both by providing an updated list of field
>> names, and a unit system, but that's currently on the backburner while Matt
>> pushes ahead on necessary yt-3.0 infrastructure work.
>>
>> You should also supply on-disk fields without conversions to CGS and with
>> native field names - this is how we distinguish between 'yt' fields, which
>> always have units and enzo field names, and on-disk fields, which are
>> always in code units. As a curious aside, this ad hoc system is broken for
>> enzo, making it a little non-trivial to access on-disk enzo data.  Like I
>> said, the units refactor will make this better :)
>>
>> I'll let Matt explain the distinction between known fields and unknown
>> fields... ;)
>>
>> Anyway, like Matt said on IRC, please let us know about sticking points in
>> this process.  We really should have better documentation for how to write
>> a frontend and it would be cool if we could streamline the process or
>> simplify by refactoring and cleaning up the frontend code.
>>
>> HTH,
>>
>> -Nathan
>>
>>
>> On Fri, Aug 23, 2013 at 4:37 PM, Stuart Mumford <stuart at mumford.me.uk>wrote:
>>
>>> Hello,
>>>
>>> First an introduction, I am Stuart Mumford I am doing a solar physics PhD
>>> at Sheffield University in the UK. I work primarily on MHD wave simulations
>>> of the solar atmosphere studying energy propagation and wave excitation.
>>> The code SAC (http://www.aanda.org/10.1051/0004-6361:200809800) solves
>>> for perturbations ontop of a fixed background to deal with the very wide
>>> range of the parameters e.g. density and temperature in the solar
>>> atmosphere. I also attended SciPy this year, where I met some of you and
>>> discovered yt, mainly as a consequence of my contributions to SunPy (
>>> www.sunpy.org) which is a solar physics data analysis library.
>>>
>>> So that's that over with, now having been playing with yt on and off
>>> since SciPy I started using it properly this week, as the need arose. I am
>>> very impressed with yt's capabilities and I want to be able to make better
>>> use of it. To that end I am trying to write a frontend for SAC, as I have
>>> been speaking to some of you about in IRC (I am Cadair in IRC and on GH).
>>>
>>> I have some questions regarding this, as far as I can ascertain a
>>> frontend provides a mapper between the stored data and a abstracted format
>>> that yt understands? I have started looking in the code for FLASH and
>>> Stream to try and get an idea where to start and not got very far. So some
>>> questions to start:
>>>
>>> Where should the file handling be done?
>>> Where is the file opened?
>>> What things should the StaticOutput subclass define?
>>>
>>> Thanks once again for your help on IRC over the last few days and I hope
>>> you can point me in the right direction.
>>>
>>> Stuart
>>>
>>> _______________________________________________
>>> yt-dev mailing list
>>> yt-dev at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>
>>>
>>
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>>
> 
> 
> 
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 901 bytes
Desc: OpenPGP digital signature
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130824/f95341ba/attachment.pgp>