[Yt-dev] yt dataset database screencast

Stephen Skory s at skory.us
Fri Sep 9 07:49:52 PDT 2011


Hi Cameron,

> 1) It might be nice to have a function which instantiates the datadumps
> into the Amazon DB without having to actually load them in yt (to save
> the time of actually loading them one-by-one).

This is something I've already thought of but haven't implemented.
Something like 'yt touch DD????/DD????' or some similarly descriptive
verb of what's going on. I'd like to have the SQLite local database
stuff working before trying to set that up (with Matt's help).

> 2) It might also be nice to have a keyword / description that one could
> set in the PF or perhaps in the DB when loading the DDs, for
> identification purposes (e.g. This is the run I did to look at metal
> cooling with feedback, etc.).  Perhaps that is something that should be
> built into enzo's PF though.  Hmm...

I think there are few ways that this could be done. It would not be an
awful thing if there was a "Description" parameter added to Enzo that
was attached to every restart file. This could then be included with
the data added to the database. Or, a description field could be added
to SimpleDB that is editable through the web script. I think in any
case, the second option is probably a good one, but the first one
might be worth thinking about.

> 3) Is this functionality specific to enzo datafiles, or is it universal
> to anything yt accesses?  I see the "ProblemType" header, which suggests
> that it may be unique to enzo.  Or is it just pulling from a variety of
> different headers which are defined in each datafile, regardless of
> simulation-code origin?  It would be good for this to keep with yt's
> goals of universal acceptance of other sim codes.

I would like to make things universal, and the nature of SimpleDB is
flexible enough that it's not a problem on that end of things. Unlike
a traditional SQL database, items stored don't all have to have the
same columns. So yes, your suspicion is mostly correct - the column
headers are created based on the data returned. But the search input
boxes at the bottom are done by hand, since there is no obvious way to
do them automatically (I think...). I think that when this is all
(SQLite and SimpleDB) working for enzo datasets, which it is not
currently, it won't be too difficult to get it to work for other
simulation datasets.

Thanks for the comments!

-- 
Stephen Skory
s at skory.us
http://stephenskory.com/
510.621.3687 (google voice)



More information about the yt-dev mailing list