[yt-dev] A New Fontend

Sat Aug 24 02:48:58 PDT 2013

Cool,

So how can I get the filename?

Ok so more generally what is the pipeline for reading a file that I need to
replicate?

how to go from
>>>load('myfile')
...
...
...

to returning a subclass of StaticOutput?

Stuart

On 24 August 2013 08:50, Kacper Kowalik <xarthisius.kk at gmail.com> wrote:

> On 24.08.2013 09:46, Stuart Mumford wrote:
> > Hello,
> >
> > Just a quick question that jumps out at me, it is possible to not use
> yt's
> > own hdf5 opening code and do it all in the frontend because I have a
> > library that abstracts the two data formats we have for the simulations,
> > only one is hdf5.
>
> Hi Stuart,
> I doesn't matter what you use to get data from disk to memory in
> your_frontend/io.py as long as you get the job done ;)
> Cheers,
> Kacper
>
> >
> > Stuart
> >
> >
> > On 24 August 2013 06:51, Nathan Goldbaum <nathan12343 at gmail.com> wrote:
> >
> >> Hey Stewart,
> >>
> >> Take a look at the FLASH frontend, specifically
> >> yt/frontends/flash/data_structures.py.
> >>
> >> In FLASH, the IO is handled by relying on h5py.  If you look at the
> >> beginning of FLASHStaticOutput.__init__, you'll see that a _handle
> >> attribute is assigned to the instance. This _handle is a reference to
> the
> >> open h5py File object.  Optionally, FLASH allows a particle file which
> is
> >> written to disk seperately from the main hydro data file, so that is
> >> optionally opened as well and the File handle is assigned to
> >> _particle_handle.
> >>
> >> FLASHStaticOutput._handle is then used many times elsewhere to read data
> >> from the output file and to create an in-memory view of the FLASH AMR
> data
> >> structure.  The AMR hierarchy is handled by FLASHHierarchy, and data IO
> is
> >> also logically handled there, although via a FLASHIOHandler instance.
>  The
> >> IOHandler is defined using a bit of metaclass magic via the io_registry,
> >> which is set up in the IOHandler base class in
> yt/utilities/io_handler.py.
> >>  If you look in yt/frontends/flash/io.py, you'll find the definition for
> >> FLASHIOHandler, which implements all the IO for hydro data.  Since it
> uses
> >> h5py, this all happens via a dict-like interface.
> >>
> >> I'm not sure if there is an exhaustive list of things that a
> StaticOutput
> >> object needs to implement.  There is probably a minimal number of
> >> parameters that the rest of the code needs to know about and that list
> of
> >> things should probably be written down somewhere.  I would look at a
> few of
> >> the frontends (FLASH and Orion are simple, relatively clean examples;
> the
> >> enzo frontend is unfortunately a bit messy, there's been a lot of work
> >> recently on the Ramses, SPH, ART, and ARTIO frontends) and try to
> implement
> >> functionality that is common between the codes.  A couple examples
> spring
> >> to mind.  All StaticOutput sublcasses need to define a parameters dict
> and
> >> a minimal set of parameters needed elsewhere.  Similarly, there needs
> to be
> >> a units dict which defines unit conversions.
> >>
> >> Frontends also define a fields.py file.  Right now, frontends needs to
> >> implement fields with enzo field names (so, 'Density', 'Pressure',
> >> 'x-velocity', 'GasEnergy', 'TotalEnergy', etc.) because
> universal_fields.py
> >> assumes these field names are defined and in CGS units.  There's an
> effort
> >> to make this situation better, both by providing an updated list of
> field
> >> names, and a unit system, but that's currently on the backburner while
> Matt
> >> pushes ahead on necessary yt-3.0 infrastructure work.
> >>
> >> You should also supply on-disk fields without conversions to CGS and
> with
> >> native field names - this is how we distinguish between 'yt' fields,
> which
> >> always have units and enzo field names, and on-disk fields, which are
> >> always in code units. As a curious aside, this ad hoc system is broken
> for
> >> enzo, making it a little non-trivial to access on-disk enzo data.  Like
> I
> >> said, the units refactor will make this better :)
> >>
> >> I'll let Matt explain the distinction between known fields and unknown
> >> fields... ;)
> >>
> >> Anyway, like Matt said on IRC, please let us know about sticking points
> in
> >> this process.  We really should have better documentation for how to
> write
> >> a frontend and it would be cool if we could streamline the process or
> >> simplify by refactoring and cleaning up the frontend code.
> >>
> >> HTH,
> >>
> >> -Nathan
> >>
> >>
> >> On Fri, Aug 23, 2013 at 4:37 PM, Stuart Mumford <stuart at mumford.me.uk
> >wrote:
> >>
> >>> Hello,
> >>>
> >>> First an introduction, I am Stuart Mumford I am doing a solar physics
> PhD
> >>> at Sheffield University in the UK. I work primarily on MHD wave
> simulations
> >>> of the solar atmosphere studying energy propagation and wave
> excitation.
> >>> The code SAC (http://www.aanda.org/10.1051/0004-6361:200809800) solves
> >>> for perturbations ontop of a fixed background to deal with the very
> wide
> >>> range of the parameters e.g. density and temperature in the solar
> >>> atmosphere. I also attended SciPy this year, where I met some of you
> and
> >>> discovered yt, mainly as a consequence of my contributions to SunPy (
> >>> www.sunpy.org) which is a solar physics data analysis library.
> >>>
> >>> So that's that over with, now having been playing with yt on and off
> >>> since SciPy I started using it properly this week, as the need arose.
> I am
> >>> very impressed with yt's capabilities and I want to be able to make
> better
> >>> use of it. To that end I am trying to write a frontend for SAC, as I
> have
> >>> been speaking to some of you about in IRC (I am Cadair in IRC and on
> GH).
> >>>
> >>> I have some questions regarding this, as far as I can ascertain a
> >>> frontend provides a mapper between the stored data and a abstracted
> format
> >>> that yt understands? I have started looking in the code for FLASH and
> >>> Stream to try and get an idea where to start and not got very far. So
> some
> >>> questions to start:
> >>>
> >>> Where should the file handling be done?
> >>> Where is the file opened?
> >>> What things should the StaticOutput subclass define?
> >>>
> >>> Thanks once again for your help on IRC over the last few days and I
> hope
> >>> you can point me in the right direction.
> >>>
> >>> Stuart
> >>>
> >>> _______________________________________________
> >>> yt-dev mailing list
> >>> yt-dev at lists.spacepope.org
> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>>
> >>>
> >>
> >> _______________________________________________
> >> yt-dev mailing list
> >> yt-dev at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >>
> >>
> >
> >
> >
> > _______________________________________________
> > yt-dev mailing list
> > yt-dev at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
>
>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130824/0b61a032/attachment.htm>