[yt-dev] Consolidating Boxlib frontends
Michael Zingale
michael.zingale at stonybrook.edu
Thu May 16 11:59:07 PDT 2013
not sure if this helps of not, but this is from plotfile.f90 in BoxLib:
! NavierStokes-V1.1 Plotfile Formats
! Record
! : c : NavierStokes-V1.1/HyperClaw-V1.1
! : c : Numbers of fields = n
! n: i : Field Names
! : i : Dimension = dm
! : r : Time
! : i : Number of Levels - 1 : nl
! : r : Physical domain lo end [1:dm]
! : r : Physical domain hi end [1:dm]
! : i : Refinement Ratios [1:nl-1]
! : b : Prob domains per level [1:nl]
! : i : unused [1:nl]
! nl: r : grid spacing, per level, [1:dm]
! : i : unused :
! : i : unused
! For each level
! [
! : iiri : dummy, nboxes, dummy, dummy
! For each box, j
! [
! : r : plo[1:dm,j], phi[1:dm, j]
! ]
! : c : level directory
! ]
! Close Header File
! For each level
! [
! Open Header of sub-directory
! : iiii: dummy, dummy, ncomponents, dummy
! : i ; '(', nboxes dummy
! For each box, j
! [
! : b : bx[j]
! ]
! : : ')'
! For each box, j
! [
! : ci : 'FabOnDisk: ' Filename[j], Offset[j]
! ]
! : i : nboxes, ncomponents
! For each box, j
! [
! : r : min[j]
! ]
! : i : nboxes, ncomponents
! For each box, j
! [
! : r : man[j]
! ]
! Close subgrid file
! ]
On Thu, May 16, 2013 at 2:56 PM, Matthew Turk <matthewturk at gmail.com> wrote:
> Thanks, everybody!
>
> On Thu, May 16, 2013 at 2:55 PM, Michael Zingale
> <michael.zingale at stonybrook.edu> wrote:
> > just to be clear, that is the number of MPI tasks, not the number of
> > processors (# pros = # MPI * OMP_NUM_THREADS)
> >
> > I just looked at a big run I did for a paper on > 10,000 cores, and
> there is
> > just a single line:
> >
> > Level_00/Cell
> >
> >
> >
> > On Thu, May 16, 2013 at 2:52 PM, Matthew Turk <matthewturk at gmail.com>
> wrote:
> >>
> >> On Thu, May 16, 2013 at 2:47 PM, Chris Malone <chris.m.malone at gmail.com
> >
> >> wrote:
> >> > Yes, the comoving bit is not in the Castro anymore - or at least no
> one
> >> > uses
> >> > it that I'm aware of.
> >> >
> >> > The number of Level_*/Cell_D_* files for each level is actually based
> on
> >> > the
> >> > number of processors set to do IO. This can be changed with a runtime
> >> > parameter; Maestro also does this. The simple problems for which you
> >> > have
> >> > Castro data (like the one I just uploaded to dickenson) were run on a
> >> > single
> >> > core, so there is only one Cell_D file per level.
> >>
> >> The number of processors is also in the Cell_H file on the third line,
> >> to my understanding of the format. But I think I'm still not making
> >> myself clear: will there ever be a data file that isn't
> >> Cell_Something. Let me give an example. In RadTube's global Header
> >> file we have:
> >>
> >> 0 8 7.26154130223839759212753081768e-07
> >> 500
> >> 0 16
> >> 0 16
> >> 0 16
> >> 112 128
> >> 0 16
> >> 0 16
> >> 96 112
> >> 0 16
> >> 0 16
> >> 80 96
> >> 0 16
> >> 0 16
> >> 64 80
> >> 0 16
> >> 0 16
> >> 48 64
> >> 0 16
> >> 0 16
> >> 32 48
> >> 0 16
> >> 0 16
> >> 16 32
> >> 0 16
> >> 0 16
> >> Level_0/Cell
> >> 1 36 7.26154130223839970970989895344e-07
> >>
> >> I am currently parsing this by getting each lo/hi/dimension pair, then
> >> assuming there is a *single* line following the set that describes the
> >> prefix for the level header, in this case Level_0/Cell , which is then
> >> followed by the *next level*. The original frontend code was much
> >> more lenient, and in fact allowed for *multiple* filenames to be
> >> displayed there -- for instance, it would allow something like:
> >>
> >> Level_0/Cell
> >> Level_0/Dinosaurs
> >> Level_0/BlahBlah
> >>
> >> where it expects Level_0/Dinosaurs_H and Level_0/BlahBlah_H as well as
> >> their corresponding processor files. *Is this real* or can I just
> >> assume there will only be one type of data file, in this case
> >> Level_0/Cell and its corresponding data?
> >>
> >> >
> >> > Chris
> >> >
> >> >
> >> >
> >> > On Thu, May 16, 2013 at 11:38 AM, Casey W. Stark <
> caseywstark at gmail.com>
> >> > wrote:
> >> >>
> >> >> Hi Matt.
> >> >>
> >> >> I think castro.use_comoving was removed once Nyx was ripped out.
> >> >>
> >> >> It sounds like there are more inconsistencies... Nyx datasets always
> >> >> have
> >> >> the structure plt%05i/Level_%i, then in each level directory, the
> >> >> Cell_H
> >> >> file and 64 data files Cell_D_%04d from 0000 to 0063.
> >> >>
> >> >>
> >> >> On Thu, May 16, 2013 at 11:29 AM, Matthew Turk <
> matthewturk at gmail.com>
> >> >> wrote:
> >> >>>
> >> >>> On Thu, May 16, 2013 at 10:06 AM, Michael Zingale
> >> >>> <michael.zingale at stonybrook.edu> wrote:
> >> >>> > Yes, that is correct.
> >> >>> >
> >> >>> > Also, orion is C++ Box lib, not Fortran.
> >> >>>
> >> >>> Oops, thanks.
> >> >>>
> >> >>> >
> >> >>> > Finally, because of how C++ boxlib works, it isn't easy to store
> >> >>> > runtime
> >> >>> > parameters if the default was unchanged. In Fortran box lib, a
> >> >>> > python
> >> >>> > script parses parameter files and writes f90 code at compile time,
> >> >>> > making it
> >> >>> > easy.
> >> >>>
> >> >>> Ah, okay, great. That is helpful, and something we can do.
> >> >>>
> >> >>> >
> >> >>> > So for Castro and Nyx, if you tell me exactly what info you might
> >> >>> > need
> >> >>> > from
> >> >>> > the runtime paramters, I can add a section to job_info that
> provides
> >> >>> > the
> >> >>> > info
> >> >>>
> >> >>> I will look into this, but I think right now it comes down to:
> >> >>>
> >> >>> amr.n_cell (We might be able to get this in the Header?)
> >> >>> amr.ref_ratio (which I think we can get elsewhere?)
> >> >>> Prob.lo_bc
> >> >>> castro.use_comoving
> >> >>> comoving_OmL
> >> >>> comoving_OmM
> >> >>> comoving_h
> >> >>> comoving_a (Which is currently in a different file)
> >> >>>
> >> >>> One thing I'm also wondering about is looking at the Header file, it
> >> >>> seems that all of the examples I have only have a single data file
> for
> >> >>> each level, of the form:
> >> >>>
> >> >>> Level_*/Cell
> >> >>>
> >> >>> with a corresponding Cell_H that contains the necessary header info.
> >> >>> Is this how it always is -- as in, only one data file per Header?
> Is
> >> >>> there a way to read, from the header, how many types of data files
> >> >>> there will be? It would make things considerably easier if I knew
> how
> >> >>> many types of data files.
> >> >>>
> >> >>> >
> >> >>> > On May 16, 2013 9:53 AM, "Matthew Turk" <matthewturk at gmail.com>
> >> >>> > wrote:
> >> >>> >>
> >> >>> >> As a quick followup, I have noticed that in many cases we default
> >> >>> >> to
> >> >>> >> using the "inputs" values over the values in the Header file.
> For
> >> >>> >> instance, the domain edges and the index space. I'd rather use
> the
> >> >>> >> Header file since it's a fundamental component, whereas the
> inputs
> >> >>> >> should not strictly be necessary to orient the data, if I
> >> >>> >> understand
> >> >>> >> Boxlib correctly.
> >> >>> >>
> >> >>> >> On Thu, May 16, 2013 at 9:18 AM, Matthew Turk
> >> >>> >> <matthewturk at gmail.com>
> >> >>> >> wrote:
> >> >>> >> > Hi all,
> >> >>> >> >
> >> >>> >> > Okay, from reading this I think I see a few things to address:
> >> >>> >> >
> >> >>> >> > * We should certainly ensure that the Fortran/C++ frontends
> both
> >> >>> >> > work. I believe -- but may be wrong -- that Maestro and Orion
> >> >>> >> > are
> >> >>> >> > the
> >> >>> >> > "Boxlib Fortran" frontends, and Nyx and Castro are the "Boxlib
> >> >>> >> > C++"
> >> >>> >> > frontends. The differences seem to be in how level files are
> >> >>> >> > stored
> >> >>> >> > and a bit about the parameters.
> >> >>> >> > * A clear message: make as many of the inputs files optional
> as
> >> >>> >> > possible, and once the job_info changes have been deployed,
> that
> >> >>> >> > will
> >> >>> >> > only leave Orion without job_info.
> >> >>> >> > * Particles should be delegated to subclasses.
> >> >>> >> > * Almost none of the Enzo-isms are necessary anymore, and we
> can
> >> >>> >> > work
> >> >>> >> > around the gamma/refine_by stuff.
> >> >>> >> > * Get rid of paranoia mode
> >> >>> >> > * Put off changes to the Pluto/Chombo frontends
> >> >>> >> >
> >> >>> >> > For what it's worth, I've gotten a "boxlib" frontend that can
> >> >>> >> > load
> >> >>> >> > up
> >> >>> >> > the example Orion data as well as a Maestro dataset that Mike
> >> >>> >> > provided
> >> >>> >> > me. The biggest issue was with the fortran-formatted numbers
> we
> >> >>> >> > corrected for previously, but I think I've got a workaround.
> >> >>> >> >
> >> >>> >> > In digging through the code I also found a number of routines
> >> >>> >> > that
> >> >>> >> > have been largely unchanged since their first implementation
> some
> >> >>> >> > years ago, which I'm going to try to clean up.
> >> >>> >> >
> >> >>> >> > Thanks everyone. I've been using the bookmark "boxlib" in my
> >> >>> >> > repo
> >> >>> >> > at
> >> >>> >> > https://bitbucket.org/MatthewTurk/yt .
> >> >>> >> >
> >> >>> >> > -Matt
> >> >>> >> >
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > On Wed, May 15, 2013 at 8:36 PM, Michael Zingale
> >> >>> >> > <michael.zingale at stonybrook.edu> wrote:
> >> >>> >> >> Maestro particles are also their own thing, and we shouldn't
> >> >>> >> >> worry
> >> >>> >> >> about
> >> >>> >> >> them. We use them for postprocessing, and not really
> >> >>> >> >> visualization.
> >> >>> >> >>
> >> >>> >> >> Looking in BoxLib, it is the case that the F90 BoxLib
> >> >>> >> >> (Src/F_BaseLib/fabio.f90) specifies Level_XX instead of
> Level_X
> >> >>> >> >> names
> >> >>> >> >> for
> >> >>> >> >> the directory.
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >> On Wed, May 15, 2013 at 7:30 PM, Andrew Myers
> >> >>> >> >> <atmyers2 at gmail.com>
> >> >>> >> >> wrote:
> >> >>> >> >>>
> >> >>> >> >>> Hi Folks,
> >> >>> >> >>>
> >> >>> >> >>> The Orion particles are completely independent of BoxLib. I
> >> >>> >> >>> don't
> >> >>> >> >>> know
> >> >>> >> >>> if
> >> >>> >> >>> new BoxLib specifies a way for particle data to be stored, or
> >> >>> >> >>> if
> >> >>> >> >>> all
> >> >>> >> >>> the
> >> >>> >> >>> Boxlib codes just do their own thing, so I'm not sure
> whether:
> >> >>> >> >>>
> >> >>> >> >>> 1. The base class should be agnostic about particles and
> >> >>> >> >>> particle
> >> >>> >> >>> readers
> >> >>> >> >>> get implemented in each subclass, or
> >> >>> >> >>> 2. The base class should implement the Nyx-style particles
> and
> >> >>> >> >>> the
> >> >>> >> >>> Orion
> >> >>> >> >>> subclass should overwrite this.
> >> >>> >> >>>
> >> >>> >> >>> Matt, on your specific questions:
> >> >>> >> >>>
> >> >>> >> >>>
> >> >>> >> >>> * Is it okay to get rid of the old enzo parameter mappings?
> >> >>> >> >>>
> >> >>> >> >>> If I'm reading the code correctly, this is currently needed
> for
> >> >>> >> >>> "Gamma"
> >> >>> >> >>> and "Refineby" to get set correctly, but nothing else.
> >> >>> >> >>>
> >> >>> >> >>> * Do we need to have all of the paranoid IO still?
> >> >>> >> >>>
> >> >>> >> >>> I'm not 100% sure what this does, but currently paranoid_read
> >> >>> >> >>> is
> >> >>> >> >>> False
> >> >>> >> >>> by
> >> >>> >> >>> default and I've never noticed a problem with that.
> >> >>> >> >>>
> >> >>> >> >>> * Do you mind if I reword some of the names?
> >> >>> >> >>>
> >> >>> >> >>> Not at all.
> >> >>> >> >>>
> >> >>> >> >>> -Andrew
> >> >>> >> >>>
> >> >>> >> >>> On Wed, May 15, 2013 at 3:32 PM, Casey W. Stark
> >> >>> >> >>> <caseywstark at gmail.com>
> >> >>> >> >>> wrote:
> >> >>> >> >>>>
> >> >>> >> >>>> Hi Matt.
> >> >>> >> >>>>
> >> >>> >> >>>> That all sounds right.
> >> >>> >> >>>>
> >> >>> >> >>>> Really unifying the BoxLib codes sounds like tricky business
> >> >>> >> >>>> to
> >> >>> >> >>>> me. I
> >> >>> >> >>>> think the current boxlib codes castro, maestro, and nyx will
> >> >>> >> >>>> be
> >> >>> >> >>>> using
> >> >>> >> >>>> basically the same versions of boxlib, but orion could be
> >> >>> >> >>>> different
> >> >>> >> >>>> altogether. I haven't seen the code, so I can't say, but I
> bet
> >> >>> >> >>>> this
> >> >>> >> >>>> is the
> >> >>> >> >>>> reason for the particle difference.
> >> >>> >> >>>>
> >> >>> >> >>>> > * Castro was originally going to be a frontend
> >> >>> >> >>>>
> >> >>> >> >>>> Nyx was originally part of Castro, but we ripped it out
> around
> >> >>> >> >>>> the
> >> >>> >> >>>> same
> >> >>> >> >>>> time. Is anyone using yt with Castro data now?
> >> >>> >> >>>>
> >> >>> >> >>>> > * Some weird differences like Level_%i and Level_%02i for
> >> >>> >> >>>> > level
> >> >>> >> >>>> > filenames.
> >> >>> >> >>>>
> >> >>> >> >>>> Hm, I don't know about this one since we never use more
> than a
> >> >>> >> >>>> few
> >> >>> >> >>>> levels, and that's rare.
> >> >>> >> >>>>
> >> >>> >> >>>> > * Is it okay to get rid of the old enzo parameter
> mappings?
> >> >>> >> >>>> > * Do we need to have all of the paranoid IO still?
> >> >>> >> >>>>
> >> >>> >> >>>> I'm not sure what these were for... If things like parameter
> >> >>> >> >>>> names
> >> >>> >> >>>> are
> >> >>> >> >>>> fine without the remapping, then great. The IO seems fine,
> so
> >> >>> >> >>>> I
> >> >>> >> >>>> doubt
> >> >>> >> >>>> we
> >> >>> >> >>>> need a paranoid option.
> >> >>> >> >>>>
> >> >>> >> >>>> > * Do you mind if I reword some of the names?
> >> >>> >> >>>>
> >> >>> >> >>>> Go for it.
> >> >>> >> >>>>
> >> >>> >> >>>>
> >> >>> >> >>>>
> >> >>> >> >>>> On Wed, May 15, 2013 at 11:50 AM, Matthew Turk
> >> >>> >> >>>> <matthewturk at gmail.com>
> >> >>> >> >>>> wrote:
> >> >>> >> >>>>>
> >> >>> >> >>>>> Hi all,
> >> >>> >> >>>>>
> >> >>> >> >>>>> I'm spending a bit of time today looking over the Boxlib
> >> >>> >> >>>>> frontends
> >> >>> >> >>>>> and
> >> >>> >> >>>>> identifying commonalities and differences. I wanted to
> write
> >> >>> >> >>>>> with
> >> >>> >> >>>>> some of my initial findings as well as some specific
> >> >>> >> >>>>> questions.
> >> >>> >> >>>>>
> >> >>> >> >>>>> Casey, Andrew, and Jeff I would particularly like your
> >> >>> >> >>>>> feedback,
> >> >>> >> >>>>> but
> >> >>> >> >>>>> I
> >> >>> >> >>>>> also intend to submit this for detailed review and
> >> >>> >> >>>>> collaboration.
> >> >>> >> >>>>> My
> >> >>> >> >>>>> general plan is to consolidate the data_structures, IO and
> >> >>> >> >>>>> definitions
> >> >>> >> >>>>> into a single Boxlib frontend, with subclasses that cover
> >> >>> >> >>>>> each
> >> >>> >> >>>>> of
> >> >>> >> >>>>> the
> >> >>> >> >>>>> specific differences.
> >> >>> >> >>>>>
> >> >>> >> >>>>> The main differences I am seeing:
> >> >>> >> >>>>>
> >> >>> >> >>>>> * Castro was originally going to be a frontend, but
> morphed
> >> >>> >> >>>>> into
> >> >>> >> >>>>> the
> >> >>> >> >>>>> Nyx frontend, and I don't believe anything Castro-specific
> is
> >> >>> >> >>>>> left
> >> >>> >> >>>>> anymore. I am inclined to remove it and in the future add
> on
> >> >>> >> >>>>> any
> >> >>> >> >>>>> subclass-specific items when Castro comes back into the
> fold.
> >> >>> >> >>>>> * Nyx and Orion handle particles completely differently.
> >> >>> >> >>>>> * Nyx has optimized IO, particularly for fluids, which we
> >> >>> >> >>>>> should
> >> >>> >> >>>>> use
> >> >>> >> >>>>> everywhere.
> >> >>> >> >>>>> * Some weird differences like Level_%i and Level_%02i for
> >> >>> >> >>>>> level
> >> >>> >> >>>>> filenames.
> >> >>> >> >>>>> * Units will need to be set individually.
> >> >>> >> >>>>> * Parsing the Fortran and non-Fortran parameter files
> (i.e.,
> >> >>> >> >>>>> Maestro
> >> >>> >> >>>>> and everyone else) will be tricky.
> >> >>> >> >>>>>
> >> >>> >> >>>>> There were a few other things I wanted to get feed back on.
> >> >>> >> >>>>>
> >> >>> >> >>>>> * Is it okay to get rid of the old enzo parameter
> mappings?
> >> >>> >> >>>>> * Do we need to have all of the paranoid IO still?
> >> >>> >> >>>>> * Do you mind if I reword some of the names?
> >> >>> >> >>>>>
> >> >>> >> >>>>> What do you all think? If anybody wants to help out, I'll
> be
> >> >>> >> >>>>> in
> >> >>> >> >>>>> IRC
> >> >>> >> >>>>> working on this for a while today, but I'll also try to
> >> >>> >> >>>>> submit a
> >> >>> >> >>>>> PR
> >> >>> >> >>>>> as
> >> >>> >> >>>>> early as possible to iterate on and get feedback from.
> >> >>> >> >>>>>
> >> >>> >> >>>>> -Matt
> >> >>> >> >>>>> _______________________________________________
> >> >>> >> >>>>> yt-dev mailing list
> >> >>> >> >>>>> yt-dev at lists.spacepope.org
> >> >>> >> >>>>>
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >> >>>>
> >> >>> >> >>>>
> >> >>> >> >>>>
> >> >>> >> >>>> _______________________________________________
> >> >>> >> >>>> yt-dev mailing list
> >> >>> >> >>>> yt-dev at lists.spacepope.org
> >> >>> >> >>>>
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>> >> >>>>
> >> >>> >> >>>
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >>
> >> >>> >> >> --
> >> >>> >> >> Michael Zingale
> >> >>> >> >> Associate Professor
> >> >>> >> >>
> >> >>> >> >> Dept. of Physics & Astronomy • Stony Brook University • Stony
> >> >>> >> >> Brook, NY
> >> >>> >> >> 11794-3800
> >> >>> >> >> phone: 631-632-8225
> >> >>> >> >> e-mail: Michael.Zingale at stonybrook.edu
> >> >>> >> >> web: http://www.astro.sunysb.edu/mzingale
> >> >>> _______________________________________________
> >> >>> yt-dev mailing list
> >> >>> yt-dev at lists.spacepope.org
> >> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>
> >> >>
> >> >>
> >> >> _______________________________________________
> >> >> yt-dev mailing list
> >> >> yt-dev at lists.spacepope.org
> >> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>
> >> >
> >> >
> >> > _______________________________________________
> >> > yt-dev mailing list
> >> > yt-dev at lists.spacepope.org
> >> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >
> >
> >
> >
> >
> > --
> > Michael Zingale
> > Associate Professor
> >
> > Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY
> > 11794-3800
> > phone: 631-632-8225
> > e-mail: Michael.Zingale at stonybrook.edu
> > web: http://www.astro.sunysb.edu/mzingale
>
--
Michael Zingale
Associate Professor
Dept. of Physics & Astronomy • Stony Brook University • Stony Brook, NY
11794-3800
*phone*: 631-632-8225
*e-mail*: Michael.Zingale at stonybrook.edu
*web*: http://www.astro.sunysb.edu/mzingale
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20130516/c0874a77/attachment.htm>
More information about the yt-dev
mailing list