<div dir="ltr"><div>Hello,<br><br>Just a quick question that jumps out at me, it is possible to not use yt's own hdf5 opening code and do it all in the frontend because I have a library that abstracts the two data formats we have for the simulations, only one is hdf5.<br>
<br></div>Stuart<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On 24 August 2013 06:51, Nathan Goldbaum <span dir="ltr"><<a href="mailto:nathan12343@gmail.com" target="_blank">nathan12343@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hey Stewart,<br></div><div><br></div><div>Take a look at the FLASH frontend, specifically yt/frontends/flash/data_structures.py.</div>
<div><br></div><div>In FLASH, the IO is handled by relying on h5py. If you look at the beginning of FLASHStaticOutput.__init__, you'll see that a _handle attribute is assigned to the instance. This _handle is a reference to the open h5py File object. Optionally, FLASH allows a particle file which is written to disk seperately from the main hydro data file, so that is optionally opened as well and the File handle is assigned to _particle_handle.</div>
<div><br></div><div>FLASHStaticOutput._handle is then used many times elsewhere to read data from the output file and to create an in-memory view of the FLASH AMR data structure. The AMR hierarchy is handled by FLASHHierarchy, and data IO is also logically handled there, although via a FLASHIOHandler instance. The IOHandler is defined using a bit of metaclass magic via the io_registry, which is set up in the IOHandler base class in yt/utilities/io_handler.py. If you look in yt/frontends/flash/io.py, you'll find the definition for FLASHIOHandler, which implements all the IO for hydro data. Since it uses h5py, this all happens via a dict-like interface.</div>
<div><br></div><div>I'm not sure if there is an exhaustive list of things that a StaticOutput object needs to implement. There is probably a minimal number of parameters that the rest of the code needs to know about and that list of things should probably be written down somewhere. I would look at a few of the frontends (FLASH and Orion are simple, relatively clean examples; the enzo frontend is unfortunately a bit messy, there's been a lot of work recently on the Ramses, SPH, ART, and ARTIO frontends) and try to implement functionality that is common between the codes. A couple examples spring to mind. All StaticOutput sublcasses need to define a parameters dict and a minimal set of parameters needed elsewhere. Similarly, there needs to be a units dict which defines unit conversions.</div>
<div><br></div><div>Frontends also define a fields.py file. Right now, frontends needs to implement fields with enzo field names (so, 'Density', 'Pressure', 'x-velocity', 'GasEnergy', 'TotalEnergy', etc.) because universal_fields.py assumes these field names are defined and in CGS units. There's an effort to make this situation better, both by providing an updated list of field names, and a unit system, but that's currently on the backburner while Matt pushes ahead on necessary yt-3.0 infrastructure work. </div>
<div><br></div><div>You should also supply on-disk fields without conversions to CGS and with native field names - this is how we distinguish between 'yt' fields, which always have units and enzo field names, and on-disk fields, which are always in code units. As a curious aside, this ad hoc system is broken for enzo, making it a little non-trivial to access on-disk enzo data. Like I said, the units refactor will make this better :)</div>
<div><br></div><div>I'll let Matt explain the distinction between known fields and unknown fields... ;)</div><div><br></div><div>Anyway, like Matt said on IRC, please let us know about sticking points in this process. We really should have better documentation for how to write a frontend and it would be cool if we could streamline the process or simplify by refactoring and cleaning up the frontend code.</div>
<div><br></div><div>HTH,</div><div><br></div><div>-Nathan</div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div class="h5">On Fri, Aug 23, 2013 at 4:37 PM, Stuart Mumford <span dir="ltr"><<a href="mailto:stuart@mumford.me.uk" target="_blank">stuart@mumford.me.uk</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5"><div dir="ltr"><div><div>Hello,<br><br></div>First an introduction, I am Stuart Mumford I am doing a solar physics PhD at Sheffield University in the UK. I work primarily on MHD wave simulations of the solar atmosphere studying energy propagation and wave excitation. The code SAC (<a href="http://www.aanda.org/10.1051/0004-6361:200809800" target="_blank">http://www.aanda.org/10.1051/0004-6361:200809800</a>) solves for perturbations ontop of a fixed background to deal with the very wide range of the parameters e.g. density and temperature in the solar atmosphere. I also attended SciPy this year, where I met some of you and discovered yt, mainly as a consequence of my contributions to SunPy (<a href="http://www.sunpy.org" target="_blank">www.sunpy.org</a>) which is a solar physics data analysis library.<br>
<br></div><div>So that's that over with, now having been playing with yt on and off since SciPy I started using it properly this week, as the need arose. I am very impressed with yt's capabilities and I want to be able to make better use of it. To that end I am trying to write a frontend for SAC, as I have been speaking to some of you about in IRC (I am Cadair in IRC and on GH).<br>
<br></div><div>I have some questions regarding this, as far as I can ascertain a frontend provides a mapper between the stored data and a abstracted format that yt understands? I have started looking in the code for FLASH and Stream to try and get an idea where to start and not got very far. So some questions to start:<br>
<br>Where should the file handling be done?<br>Where is the file opened?<br>What things should the StaticOutput subclass define?<br><br></div><div>Thanks once again for your help on IRC over the last few days and I hope you can point me in the right direction.<span><font color="#888888"><br>
<br></font></span></div><span><font color="#888888"><div>Stuart<br></div></font></span></div>
<br></div></div>_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org" target="_blank">yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
<br></blockquote></div><br></div></div>
<br>_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
<br></blockquote></div><br></div>