[yt-dev] Consolidating Boxlib frontends

Thu May 16 11:52:45 PDT 2013

In the plotfile's main header, called Header, yes -- for each level there
should be a single "filename" listed such as "Level_0/Cell" or
"Level_1/Cell", etc...

I think there are some cases (depending on what actually generated the
plotfile) where the "Cell" part of the "filename" will be something else,
such as "MF" for Multifab.  In all cases, there should be a single
"filename" terminating a block of descriptions for a level in the Header
file.

Chris

On Thu, May 16, 2013 at 11:47 AM, Chris Malone <chris.m.malone at gmail.com>wrote:

> Yes, the comoving bit is not in the Castro anymore - or at least no one
> uses it that I'm aware of.
>
> The number of Level_*/Cell_D_* files for each level is actually based on
> the number of processors set to do IO.  This can be changed with a runtime
> parameter; Maestro also does this.  The simple problems for which you have
> Castro data (like the one I just uploaded to dickenson) were run on a
> single core, so there is only one Cell_D file per level.
>
> Chris
>
>
>
> On Thu, May 16, 2013 at 11:38 AM, Casey W. Stark <caseywstark at gmail.com>wrote:
>
>> Hi Matt.
>>
>> I think castro.use_comoving was removed once Nyx was ripped out.
>>
>> It sounds like there are more inconsistencies... Nyx datasets always have
>> the structure plt%05i/Level_%i, then in each level directory, the Cell_H
>> file and 64 data files Cell_D_%04d from 0000 to 0063.
>>
>>
>> On Thu, May 16, 2013 at 11:29 AM, Matthew Turk <matthewturk at gmail.com>wrote:
>>
>>> On Thu, May 16, 2013 at 10:06 AM, Michael Zingale
>>> <michael.zingale at stonybrook.edu> wrote:
>>> > Yes, that is correct.
>>> >
>>> > Also, orion is C++ Box lib, not Fortran.
>>>
>>> Oops, thanks.
>>>
>>> >
>>> > Finally, because of how C++ boxlib works, it isn't easy to store
>>> runtime
>>> > parameters if the default was unchanged.  In Fortran box lib, a python
>>> > script parses parameter files and writes f90 code at compile time,
>>> making it
>>> > easy.
>>>
>>> Ah, okay, great.  That is helpful, and something we can do.
>>>
>>> >
>>> > So for Castro and Nyx, if you tell me exactly what info you might need
>>> from
>>> > the runtime paramters, I can add a section to job_info that provides
>>> the
>>> > info
>>>
>>> I will look into this, but I think right now it comes down to:
>>>
>>> amr.n_cell (We might be able to get this in the Header?)
>>> amr.ref_ratio (which I think we can get elsewhere?)
>>> Prob.lo_bc
>>> castro.use_comoving
>>> comoving_OmL
>>> comoving_OmM
>>> comoving_h
>>> comoving_a (Which is currently in a different  file)
>>>
>>> One thing I'm also wondering about is looking at the Header file, it
>>> seems that all of the examples I have only have a single data file for
>>> each level, of the form:
>>>
>>> Level_*/Cell
>>>
>>> with a corresponding Cell_H that contains the necessary header info.
>>> Is this how it always is -- as in, only one data file per Header?  Is
>>> there a way to read, from the header, how many types of data files
>>> there will be?  It would make things considerably easier if I knew how
>>> many types of data files.
>>>
>>> >
>>> > On May 16, 2013 9:53 AM, "Matthew Turk" <matthewturk at gmail.com> wrote:
>>> >>
>>> >> As a quick followup, I have noticed that in many cases we default to
>>> >> using the "inputs" values over the values in the Header file.  For
>>> >> instance, the domain edges and the index space.  I'd rather use the
>>> >> Header file since it's a fundamental component, whereas the inputs
>>> >> should not strictly be necessary to orient the data, if I understand
>>> >> Boxlib correctly.
>>> >>
>>> >> On Thu, May 16, 2013 at 9:18 AM, Matthew Turk <matthewturk at gmail.com>
>>> >> wrote:
>>> >> > Hi all,
>>> >> >
>>> >> > Okay, from reading this I think I see a few things to address:
>>> >> >
>>> >> >  * We should certainly ensure that the Fortran/C++ frontends both
>>> >> > work.  I believe -- but may be wrong -- that Maestro and Orion are
>>> the
>>> >> > "Boxlib Fortran" frontends, and Nyx and Castro are the "Boxlib C++"
>>> >> > frontends.  The differences seem to be in how level files are stored
>>> >> > and a bit about the parameters.
>>> >> >  * A clear message: make as many of the inputs files optional as
>>> >> > possible, and once the job_info changes have been deployed, that
>>> will
>>> >> > only leave Orion without job_info.
>>> >> >  * Particles should be delegated to subclasses.
>>> >> >  * Almost none of the Enzo-isms are necessary anymore, and we can
>>> work
>>> >> > around the gamma/refine_by stuff.
>>> >> >  * Get rid of paranoia mode
>>> >> >  * Put off changes to the Pluto/Chombo frontends
>>> >> >
>>> >> > For what it's worth, I've gotten a "boxlib" frontend that can load
>>> up
>>> >> > the example Orion data as well as a Maestro dataset that Mike
>>> provided
>>> >> > me.  The biggest issue was with the fortran-formatted numbers we
>>> >> > corrected for previously, but I think I've got a workaround.
>>> >> >
>>> >> > In digging through the code I also found a number of routines that
>>> >> > have been largely unchanged since their first implementation some
>>> >> > years ago, which I'm going to try to clean up.
>>> >> >
>>> >> > Thanks everyone.  I've been using the bookmark "boxlib" in my repo
>>> at
>>> >> > https://bitbucket.org/MatthewTurk/yt .
>>> >> >
>>> >> > -Matt
>>> >> >
>>> >> >
>>> >> >
>>> >> > On Wed, May 15, 2013 at 8:36 PM, Michael Zingale
>>> >> > <michael.zingale at stonybrook.edu> wrote:
>>> >> >> Maestro particles are also their own thing, and we shouldn't worry
>>> >> >> about
>>> >> >> them.  We use them for postprocessing, and not really
>>> visualization.
>>> >> >>
>>> >> >> Looking in BoxLib, it is the case that the F90 BoxLib
>>> >> >> (Src/F_BaseLib/fabio.f90) specifies Level_XX instead of Level_X
>>> names
>>> >> >> for
>>> >> >> the directory.
>>> >> >>
>>> >> >>
>>> >> >> On Wed, May 15, 2013 at 7:30 PM, Andrew Myers <atmyers2 at gmail.com>
>>> >> >> wrote:
>>> >> >>>
>>> >> >>> Hi Folks,
>>> >> >>>
>>> >> >>> The Orion particles are completely independent of BoxLib. I don't
>>> know
>>> >> >>> if
>>> >> >>> new BoxLib specifies a way for particle data to be stored, or if
>>> all
>>> >> >>> the
>>> >> >>> Boxlib codes just do their own thing, so I'm not sure whether:
>>> >> >>>
>>> >> >>> 1. The base class should be agnostic about particles and particle
>>> >> >>> readers
>>> >> >>> get implemented in each subclass, or
>>> >> >>> 2. The base class should implement the Nyx-style particles and the
>>> >> >>> Orion
>>> >> >>> subclass should overwrite this.
>>> >> >>>
>>> >> >>> Matt, on your specific questions:
>>> >> >>>
>>> >> >>>
>>> >> >>>  * Is it okay to get rid of the old enzo parameter mappings?
>>> >> >>>
>>> >> >>> If I'm reading the code correctly, this is currently needed for
>>> >> >>> "Gamma"
>>> >> >>> and "Refineby" to get set correctly, but nothing else.
>>> >> >>>
>>> >> >>>  * Do we need to have all of the paranoid IO still?
>>> >> >>>
>>> >> >>> I'm not 100% sure what this does, but currently paranoid_read is
>>> False
>>> >> >>> by
>>> >> >>> default and I've never noticed a problem with that.
>>> >> >>>
>>> >> >>>  * Do you mind if I reword some of the names?
>>> >> >>>
>>> >> >>> Not at all.
>>> >> >>>
>>> >> >>> -Andrew
>>> >> >>>
>>> >> >>> On Wed, May 15, 2013 at 3:32 PM, Casey W. Stark
>>> >> >>> <caseywstark at gmail.com>
>>> >> >>> wrote:
>>> >> >>>>
>>> >> >>>> Hi Matt.
>>> >> >>>>
>>> >> >>>> That all sounds right.
>>> >> >>>>
>>> >> >>>> Really unifying the BoxLib codes sounds like tricky business to
>>> me. I
>>> >> >>>> think the current boxlib codes castro, maestro, and nyx will be
>>> using
>>> >> >>>> basically the same versions of boxlib, but orion could be
>>> different
>>> >> >>>> altogether. I haven't seen the code, so I can't say, but I bet
>>> this
>>> >> >>>> is the
>>> >> >>>> reason for the particle difference.
>>> >> >>>>
>>> >> >>>> > * Castro was originally going to be a frontend
>>> >> >>>>
>>> >> >>>> Nyx was originally part of Castro, but we ripped it out around
>>> the
>>> >> >>>> same
>>> >> >>>> time. Is anyone using yt with Castro data now?
>>> >> >>>>
>>> >> >>>> >  * Some weird differences like Level_%i and Level_%02i for
>>> level
>>> >> >>>> > filenames.
>>> >> >>>>
>>> >> >>>> Hm, I don't know about this one since we never use more than a
>>> few
>>> >> >>>> levels, and that's rare.
>>> >> >>>>
>>> >> >>>> >  * Is it okay to get rid of the old enzo parameter mappings?
>>> >> >>>> > * Do we need to have all of the paranoid IO still?
>>> >> >>>>
>>> >> >>>> I'm not sure what these were for... If things like parameter
>>> names
>>> >> >>>> are
>>> >> >>>> fine without the remapping, then great. The IO seems fine, so I
>>> doubt
>>> >> >>>> we
>>> >> >>>> need a paranoid option.
>>> >> >>>>
>>> >> >>>> >  * Do you mind if I reword some of the names?
>>> >> >>>>
>>> >> >>>> Go for it.
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> >> >>>> On Wed, May 15, 2013 at 11:50 AM, Matthew Turk
>>> >> >>>> <matthewturk at gmail.com>
>>> >> >>>> wrote:
>>> >> >>>>>
>>> >> >>>>> Hi all,
>>> >> >>>>>
>>> >> >>>>> I'm spending a bit of time today looking over the Boxlib
>>> frontends
>>> >> >>>>> and
>>> >> >>>>> identifying commonalities and differences.  I wanted to write
>>> with
>>> >> >>>>> some of my initial findings as well as some specific questions.
>>> >> >>>>>
>>> >> >>>>> Casey, Andrew, and Jeff I would particularly like your
>>> feedback, but
>>> >> >>>>> I
>>> >> >>>>> also intend to submit this for detailed review and
>>> collaboration.
>>> >> >>>>> My
>>> >> >>>>> general plan is to consolidate the data_structures, IO and
>>> >> >>>>> definitions
>>> >> >>>>> into a single Boxlib frontend, with subclasses that cover each
>>> of
>>> >> >>>>> the
>>> >> >>>>> specific differences.
>>> >> >>>>>
>>> >> >>>>> The main differences I am seeing:
>>> >> >>>>>
>>> >> >>>>>  * Castro was originally going to be a frontend, but morphed
>>> into
>>> >> >>>>> the
>>> >> >>>>> Nyx frontend, and I don't believe anything Castro-specific is
>>> left
>>> >> >>>>> anymore.  I am inclined to remove it and in the future add on
>>> any
>>> >> >>>>> subclass-specific items when Castro comes back into the fold.
>>> >> >>>>>  * Nyx and Orion handle particles completely differently.
>>> >> >>>>>  * Nyx has optimized IO, particularly for fluids, which we
>>> should
>>> >> >>>>> use
>>> >> >>>>> everywhere.
>>> >> >>>>>  * Some weird differences like Level_%i and Level_%02i for level
>>> >> >>>>> filenames.
>>> >> >>>>>  * Units will need to be set individually.
>>> >> >>>>>  * Parsing the Fortran and non-Fortran parameter files (i.e.,
>>> >> >>>>> Maestro
>>> >> >>>>> and everyone else) will be tricky.
>>> >> >>>>>
>>> >> >>>>> There were a few other things I wanted to get feed back on.
>>> >> >>>>>
>>> >> >>>>>  * Is it okay to get rid of the old enzo parameter mappings?
>>> >> >>>>>  * Do we need to have all of the paranoid IO still?
>>> >> >>>>>  * Do you mind if I reword some of the names?
>>> >> >>>>>
>>> >> >>>>> What do you all think?  If anybody wants to help out, I'll be
>>> in IRC
>>> >> >>>>> working on this for a while today, but I'll also try to submit
>>> a PR
>>> >> >>>>> as
>>> >> >>>>> early as possible to iterate on and get feedback from.
>>> >> >>>>>
>>> >> >>>>> -Matt
>>> >> >>>>> _______________________________________________
>>> >> >>>>> yt-dev mailing list
>>> >> >>>>> yt-dev at lists.spacepope.org
>>> >> >>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> >> >>>> _______________________________________________
>>> >> >>>> yt-dev mailing list
>>> >> >>>> yt-dev at lists.spacepope.org
>>> >> >>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>> >> >>>>
>>> >> >>>
>>> >> >>
>>> >> >>
>>> >> >>
>>> >> >> --
>>> >> >> Michael Zingale
>>> >> >> Associate Professor
>>> >> >>
>>> >> >> Dept. of Physics & Astronomy • Stony Brook University • Stony
>>> Brook, NY
>>> >> >> 11794-3800
>>> >> >> phone:  631-632-8225
>>> >> >> e-mail: Michael.Zingale at stonybrook.edu
>>> >> >> web: http://www.astro.sunysb.edu/mzingale
>>> _______________________________________________
>>> yt-dev mailing list
>>> yt-dev at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>
>>
>>
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130516/1903a9c3/attachment.htm>