[yt-dev] Consolidating Boxlib frontends

Matthew Turk matthewturk at gmail.com
Thu May 16 11:56:32 PDT 2013 

Thanks, everybody!

On Thu, May 16, 2013 at 2:55 PM, Michael Zingale
<michael.zingale at stonybrook.edu> wrote:
> just to be clear, that is the number of MPI tasks, not the number of
> processors (# pros = # MPI * OMP_NUM_THREADS)
>
> I just looked at a big run I did for a paper on > 10,000 cores, and there is
> just a single line:
>
> Level_00/Cell
>
>
>
> On Thu, May 16, 2013 at 2:52 PM, Matthew Turk <matthewturk at gmail.com> wrote:
>>
>> On Thu, May 16, 2013 at 2:47 PM, Chris Malone <chris.m.malone at gmail.com>
>> wrote:
>> > Yes, the comoving bit is not in the Castro anymore - or at least no one
>> > uses
>> > it that I'm aware of.
>> >
>> > The number of Level_*/Cell_D_* files for each level is actually based on
>> > the
>> > number of processors set to do IO.  This can be changed with a runtime
>> > parameter; Maestro also does this.  The simple problems for which you
>> > have
>> > Castro data (like the one I just uploaded to dickenson) were run on a
>> > single
>> > core, so there is only one Cell_D file per level.
>>
>> The number of processors is also in the Cell_H file on the third line,
>> to my understanding of the format.  But I think I'm still not making
>> myself clear: will there ever be a data file that isn't
>> Cell_Something.  Let me give an example.  In RadTube's global Header
>> file we have:
>>
>> 0 8 7.26154130223839759212753081768e-07
>> 500
>> 0 16
>> 0 16
>> 0 16
>> 112 128
>> 0 16
>> 0 16
>> 96 112
>> 0 16
>> 0 16
>> 80 96
>> 0 16
>> 0 16
>> 64 80
>> 0 16
>> 0 16
>> 48 64
>> 0 16
>> 0 16
>> 32 48
>> 0 16
>> 0 16
>> 16 32
>> 0 16
>> 0 16
>> Level_0/Cell
>> 1 36 7.26154130223839970970989895344e-07
>>
>> I am currently parsing this by getting each lo/hi/dimension pair, then
>> assuming there is a *single* line following the set that describes the
>> prefix for the level header, in this case Level_0/Cell , which is then
>> followed by the *next level*.  The original frontend code was much
>> more lenient, and in fact allowed for *multiple* filenames to be
>> displayed there -- for instance, it would allow something like:
>>
>> Level_0/Cell
>> Level_0/Dinosaurs
>> Level_0/BlahBlah
>>
>> where it expects Level_0/Dinosaurs_H and Level_0/BlahBlah_H as well as
>> their corresponding processor files.  *Is this real* or can I just
>> assume there will only be one type of data file, in this case
>> Level_0/Cell and its corresponding data?
>>
>> >
>> > Chris
>> >
>> >
>> >
>> > On Thu, May 16, 2013 at 11:38 AM, Casey W. Stark <caseywstark at gmail.com>
>> > wrote:
>> >>
>> >> Hi Matt.
>> >>
>> >> I think castro.use_comoving was removed once Nyx was ripped out.
>> >>
>> >> It sounds like there are more inconsistencies... Nyx datasets always
>> >> have
>> >> the structure plt%05i/Level_%i, then in each level directory, the
>> >> Cell_H
>> >> file and 64 data files Cell_D_%04d from 0000 to 0063.
>> >>
>> >>
>> >> On Thu, May 16, 2013 at 11:29 AM, Matthew Turk <matthewturk at gmail.com>
>> >> wrote:
>> >>>
>> >>> On Thu, May 16, 2013 at 10:06 AM, Michael Zingale
>> >>> <michael.zingale at stonybrook.edu> wrote:
>> >>> > Yes, that is correct.
>> >>> >
>> >>> > Also, orion is C++ Box lib, not Fortran.
>> >>>
>> >>> Oops, thanks.
>> >>>
>> >>> >
>> >>> > Finally, because of how C++ boxlib works, it isn't easy to store
>> >>> > runtime
>> >>> > parameters if the default was unchanged.  In Fortran box lib, a
>> >>> > python
>> >>> > script parses parameter files and writes f90 code at compile time,
>> >>> > making it
>> >>> > easy.
>> >>>
>> >>> Ah, okay, great.  That is helpful, and something we can do.
>> >>>
>> >>> >
>> >>> > So for Castro and Nyx, if you tell me exactly what info you might
>> >>> > need
>> >>> > from
>> >>> > the runtime paramters, I can add a section to job_info that provides
>> >>> > the
>> >>> > info
>> >>>
>> >>> I will look into this, but I think right now it comes down to:
>> >>>
>> >>> amr.n_cell (We might be able to get this in the Header?)
>> >>> amr.ref_ratio (which I think we can get elsewhere?)
>> >>> Prob.lo_bc
>> >>> castro.use_comoving
>> >>> comoving_OmL
>> >>> comoving_OmM
>> >>> comoving_h
>> >>> comoving_a (Which is currently in a different  file)
>> >>>
>> >>> One thing I'm also wondering about is looking at the Header file, it
>> >>> seems that all of the examples I have only have a single data file for
>> >>> each level, of the form:
>> >>>
>> >>> Level_*/Cell
>> >>>
>> >>> with a corresponding Cell_H that contains the necessary header info.
>> >>> Is this how it always is -- as in, only one data file per Header?  Is
>> >>> there a way to read, from the header, how many types of data files
>> >>> there will be?  It would make things considerably easier if I knew how
>> >>> many types of data files.
>> >>>
>> >>> >
>> >>> > On May 16, 2013 9:53 AM, "Matthew Turk" <matthewturk at gmail.com>
>> >>> > wrote:
>> >>> >>
>> >>> >> As a quick followup, I have noticed that in many cases we default
>> >>> >> to
>> >>> >> using the "inputs" values over the values in the Header file.  For
>> >>> >> instance, the domain edges and the index space.  I'd rather use the
>> >>> >> Header file since it's a fundamental component, whereas the inputs
>> >>> >> should not strictly be necessary to orient the data, if I
>> >>> >> understand
>> >>> >> Boxlib correctly.
>> >>> >>
>> >>> >> On Thu, May 16, 2013 at 9:18 AM, Matthew Turk
>> >>> >> <matthewturk at gmail.com>
>> >>> >> wrote:
>> >>> >> > Hi all,
>> >>> >> >
>> >>> >> > Okay, from reading this I think I see a few things to address:
>> >>> >> >
>> >>> >> >  * We should certainly ensure that the Fortran/C++ frontends both
>> >>> >> > work.  I believe -- but may be wrong -- that Maestro and Orion
>> >>> >> > are
>> >>> >> > the
>> >>> >> > "Boxlib Fortran" frontends, and Nyx and Castro are the "Boxlib
>> >>> >> > C++"
>> >>> >> > frontends.  The differences seem to be in how level files are
>> >>> >> > stored
>> >>> >> > and a bit about the parameters.
>> >>> >> >  * A clear message: make as many of the inputs files optional as
>> >>> >> > possible, and once the job_info changes have been deployed, that
>> >>> >> > will
>> >>> >> > only leave Orion without job_info.
>> >>> >> >  * Particles should be delegated to subclasses.
>> >>> >> >  * Almost none of the Enzo-isms are necessary anymore, and we can
>> >>> >> > work
>> >>> >> > around the gamma/refine_by stuff.
>> >>> >> >  * Get rid of paranoia mode
>> >>> >> >  * Put off changes to the Pluto/Chombo frontends
>> >>> >> >
>> >>> >> > For what it's worth, I've gotten a "boxlib" frontend that can
>> >>> >> > load
>> >>> >> > up
>> >>> >> > the example Orion data as well as a Maestro dataset that Mike
>> >>> >> > provided
>> >>> >> > me.  The biggest issue was with the fortran-formatted numbers we
>> >>> >> > corrected for previously, but I think I've got a workaround.
>> >>> >> >
>> >>> >> > In digging through the code I also found a number of routines
>> >>> >> > that
>> >>> >> > have been largely unchanged since their first implementation some
>> >>> >> > years ago, which I'm going to try to clean up.
>> >>> >> >
>> >>> >> > Thanks everyone.  I've been using the bookmark "boxlib" in my
>> >>> >> > repo
>> >>> >> > at
>> >>> >> > https://bitbucket.org/MatthewTurk/yt .
>> >>> >> >
>> >>> >> > -Matt
>> >>> >> >
>> >>> >> >
>> >>> >> >
>> >>> >> > On Wed, May 15, 2013 at 8:36 PM, Michael Zingale
>> >>> >> > <michael.zingale at stonybrook.edu> wrote:
>> >>> >> >> Maestro particles are also their own thing, and we shouldn't
>> >>> >> >> worry
>> >>> >> >> about
>> >>> >> >> them.  We use them for postprocessing, and not really
>> >>> >> >> visualization.
>> >>> >> >>
>> >>> >> >> Looking in BoxLib, it is the case that the F90 BoxLib
>> >>> >> >> (Src/F_BaseLib/fabio.f90) specifies Level_XX instead of Level_X
>> >>> >> >> names
>> >>> >> >> for
>> >>> >> >> the directory.
>> >>> >> >>
>> >>> >> >>
>> >>> >> >> On Wed, May 15, 2013 at 7:30 PM, Andrew Myers
>> >>> >> >> <atmyers2 at gmail.com>
>> >>> >> >> wrote:
>> >>> >> >>>
>> >>> >> >>> Hi Folks,
>> >>> >> >>>
>> >>> >> >>> The Orion particles are completely independent of BoxLib. I
>> >>> >> >>> don't
>> >>> >> >>> know
>> >>> >> >>> if
>> >>> >> >>> new BoxLib specifies a way for particle data to be stored, or
>> >>> >> >>> if
>> >>> >> >>> all
>> >>> >> >>> the
>> >>> >> >>> Boxlib codes just do their own thing, so I'm not sure whether:
>> >>> >> >>>
>> >>> >> >>> 1. The base class should be agnostic about particles and
>> >>> >> >>> particle
>> >>> >> >>> readers
>> >>> >> >>> get implemented in each subclass, or
>> >>> >> >>> 2. The base class should implement the Nyx-style particles and
>> >>> >> >>> the
>> >>> >> >>> Orion
>> >>> >> >>> subclass should overwrite this.
>> >>> >> >>>
>> >>> >> >>> Matt, on your specific questions:
>> >>> >> >>>
>> >>> >> >>>
>> >>> >> >>>  * Is it okay to get rid of the old enzo parameter mappings?
>> >>> >> >>>
>> >>> >> >>> If I'm reading the code correctly, this is currently needed for
>> >>> >> >>> "Gamma"
>> >>> >> >>> and "Refineby" to get set correctly, but nothing else.
>> >>> >> >>>
>> >>> >> >>>  * Do we need to have all of the paranoid IO still?
>> >>> >> >>>
>> >>> >> >>> I'm not 100% sure what this does, but currently paranoid_read
>> >>> >> >>> is
>> >>> >> >>> False
>> >>> >> >>> by
>> >>> >> >>> default and I've never noticed a problem with that.
>> >>> >> >>>
>> >>> >> >>>  * Do you mind if I reword some of the names?
>> >>> >> >>>
>> >>> >> >>> Not at all.
>> >>> >> >>>
>> >>> >> >>> -Andrew
>> >>> >> >>>
>> >>> >> >>> On Wed, May 15, 2013 at 3:32 PM, Casey W. Stark
>> >>> >> >>> <caseywstark at gmail.com>
>> >>> >> >>> wrote:
>> >>> >> >>>>
>> >>> >> >>>> Hi Matt.
>> >>> >> >>>>
>> >>> >> >>>> That all sounds right.
>> >>> >> >>>>
>> >>> >> >>>> Really unifying the BoxLib codes sounds like tricky business
>> >>> >> >>>> to
>> >>> >> >>>> me. I
>> >>> >> >>>> think the current boxlib codes castro, maestro, and nyx will
>> >>> >> >>>> be
>> >>> >> >>>> using
>> >>> >> >>>> basically the same versions of boxlib, but orion could be
>> >>> >> >>>> different
>> >>> >> >>>> altogether. I haven't seen the code, so I can't say, but I bet
>> >>> >> >>>> this
>> >>> >> >>>> is the
>> >>> >> >>>> reason for the particle difference.
>> >>> >> >>>>
>> >>> >> >>>> > * Castro was originally going to be a frontend
>> >>> >> >>>>
>> >>> >> >>>> Nyx was originally part of Castro, but we ripped it out around
>> >>> >> >>>> the
>> >>> >> >>>> same
>> >>> >> >>>> time. Is anyone using yt with Castro data now?
>> >>> >> >>>>
>> >>> >> >>>> >  * Some weird differences like Level_%i and Level_%02i for
>> >>> >> >>>> > level
>> >>> >> >>>> > filenames.
>> >>> >> >>>>
>> >>> >> >>>> Hm, I don't know about this one since we never use more than a
>> >>> >> >>>> few
>> >>> >> >>>> levels, and that's rare.
>> >>> >> >>>>
>> >>> >> >>>> >  * Is it okay to get rid of the old enzo parameter mappings?
>> >>> >> >>>> > * Do we need to have all of the paranoid IO still?
>> >>> >> >>>>
>> >>> >> >>>> I'm not sure what these were for... If things like parameter
>> >>> >> >>>> names
>> >>> >> >>>> are
>> >>> >> >>>> fine without the remapping, then great. The IO seems fine, so
>> >>> >> >>>> I
>> >>> >> >>>> doubt
>> >>> >> >>>> we
>> >>> >> >>>> need a paranoid option.
>> >>> >> >>>>
>> >>> >> >>>> >  * Do you mind if I reword some of the names?
>> >>> >> >>>>
>> >>> >> >>>> Go for it.
>> >>> >> >>>>
>> >>> >> >>>>
>> >>> >> >>>>
>> >>> >> >>>> On Wed, May 15, 2013 at 11:50 AM, Matthew Turk
>> >>> >> >>>> <matthewturk at gmail.com>
>> >>> >> >>>> wrote:
>> >>> >> >>>>>
>> >>> >> >>>>> Hi all,
>> >>> >> >>>>>
>> >>> >> >>>>> I'm spending a bit of time today looking over the Boxlib
>> >>> >> >>>>> frontends
>> >>> >> >>>>> and
>> >>> >> >>>>> identifying commonalities and differences.  I wanted to write
>> >>> >> >>>>> with
>> >>> >> >>>>> some of my initial findings as well as some specific
>> >>> >> >>>>> questions.
>> >>> >> >>>>>
>> >>> >> >>>>> Casey, Andrew, and Jeff I would particularly like your
>> >>> >> >>>>> feedback,
>> >>> >> >>>>> but
>> >>> >> >>>>> I
>> >>> >> >>>>> also intend to submit this for detailed review and
>> >>> >> >>>>> collaboration.
>> >>> >> >>>>> My
>> >>> >> >>>>> general plan is to consolidate the data_structures, IO and
>> >>> >> >>>>> definitions
>> >>> >> >>>>> into a single Boxlib frontend, with subclasses that cover
>> >>> >> >>>>> each
>> >>> >> >>>>> of
>> >>> >> >>>>> the
>> >>> >> >>>>> specific differences.
>> >>> >> >>>>>
>> >>> >> >>>>> The main differences I am seeing:
>> >>> >> >>>>>
>> >>> >> >>>>>  * Castro was originally going to be a frontend, but morphed
>> >>> >> >>>>> into
>> >>> >> >>>>> the
>> >>> >> >>>>> Nyx frontend, and I don't believe anything Castro-specific is
>> >>> >> >>>>> left
>> >>> >> >>>>> anymore.  I am inclined to remove it and in the future add on
>> >>> >> >>>>> any
>> >>> >> >>>>> subclass-specific items when Castro comes back into the fold.
>> >>> >> >>>>>  * Nyx and Orion handle particles completely differently.
>> >>> >> >>>>>  * Nyx has optimized IO, particularly for fluids, which we
>> >>> >> >>>>> should
>> >>> >> >>>>> use
>> >>> >> >>>>> everywhere.
>> >>> >> >>>>>  * Some weird differences like Level_%i and Level_%02i for
>> >>> >> >>>>> level
>> >>> >> >>>>> filenames.
>> >>> >> >>>>>  * Units will need to be set individually.
>> >>> >> >>>>>  * Parsing the Fortran and non-Fortran parameter files (i.e.,
>> >>> >> >>>>> Maestro
>> >>> >> >>>>> and everyone else) will be tricky.
>> >>> >> >>>>>
>> >>> >> >>>>> There were a few other things I wanted to get feed back on.
>> >>> >> >>>>>
>> >>> >> >>>>>  * Is it okay to get rid of the old enzo parameter mappings?
>> >>> >> >>>>>  * Do we need to have all of the paranoid IO still?
>> >>> >> >>>>>  * Do you mind if I reword some of the names?
>> >>> >> >>>>>
>> >>> >> >>>>> What do you all think?  If anybody wants to help out, I'll be
>> >>> >> >>>>> in
>> >>> >> >>>>> IRC
>> >>> >> >>>>> working on this for a while today, but I'll also try to
>> >>> >> >>>>> submit a
>> >>> >> >>>>> PR
>> >>> >> >>>>> as
>> >>> >> >>>>> early as possible to iterate on and get feedback from.
>> >>> >> >>>>>
>> >>> >> >>>>> -Matt
>> >>> >> >>>>> _______________________________________________
>> >>> >> >>>>> yt-dev mailing list
>> >>> >> >>>>> yt-dev at lists.spacepope.org
>> >>> >> >>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>> >> >>>>
>> >>> >> >>>>
>> >>> >> >>>>
>> >>> >> >>>> _______________________________________________
>> >>> >> >>>> yt-dev mailing list
>> >>> >> >>>> yt-dev at lists.spacepope.org
>> >>> >> >>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>> >> >>>>
>> >>> >> >>>
>> >>> >> >>
>> >>> >> >>
>> >>> >> >>
>> >>> >> >> --
>> >>> >> >> Michael Zingale
>> >>> >> >> Associate Professor
>> >>> >> >>
>> >>> >> >> Dept. of Physics & Astronomy • Stony Brook University • Stony
>> >>> >> >> Brook, NY
>> >>> >> >> 11794-3800
>> >>> >> >> phone:  631-632-8225
>> >>> >> >> e-mail: Michael.Zingale at stonybrook.edu
>> >>> >> >> web: http://www.astro.sunysb.edu/mzingale
>> >>> _______________________________________________
>> >>> yt-dev mailing list
>> >>> yt-dev at lists.spacepope.org
>> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> yt-dev mailing list
>> >> yt-dev at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >>
>> >
>> >
>> > _______________________________________________
>> > yt-dev mailing list
>> > yt-dev at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>
>
>
>
> --
> Michael Zingale
> Associate Professor
>
> Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY
> 11794-3800
> phone:  631-632-8225
> e-mail: Michael.Zingale at stonybrook.edu
> web: http://www.astro.sunysb.edu/mzingale

More information about the yt-dev mailing list