[yt-svn] commit/yt-doc: caseywstark: Dropping the new frontend wiki page into the docs, updating link context in the developing docs page.
Bitbucket
commits-noreply at bitbucket.org
Sat Mar 3 04:59:15 PST 2012
1 new commit in yt-doc:
https://bitbucket.org/yt_analysis/yt-doc/changeset/45a9e83d961a/
changeset: 45a9e83d961a
user: caseywstark
date: 2012-03-03 02:24:48
summary: Dropping the new frontend wiki page into the docs, updating link context in the developing docs page.
affected #: 2 files
diff -r c4f4cbbb167da2be7779965ba0e24af774aa9d97 -r 45a9e83d961a1addf095dc5abaadcc8a92962338 source/advanced/creating_frontend.rst
--- /dev/null
+++ b/source/advanced/creating_frontend.rst
@@ -0,0 +1,143 @@
+.. _creating_frontend:
+
+Creating A New Code Frontend
+============================
+
+``yt`` is designed to support analysis and visualization of data from multiple
+different simulation codes, although it has so far been most successfully
+applied to Adaptive Mesh Refinement (AMR) data. For a list of codes and the
+level of support they enjoy, we've created a handy [[CodeSupportLevels|table]].
+We'd like to support a broad range of codes, both AMR-based and otherwise. To
+add support for a new code, a few things need to be put into place. These
+necessary structures can be classified into a couple categories:
+
+ * Data meaning: This is the set of parameters that convert the data into
+ physically relevant units; things like spatial and mass conversions, time
+ units, and so on.
+ * Data localization: These are structures that help make a "first pass" at data
+ loading. Essentially, we need to be able to make a first pass at guessing
+ where data in a given physical region would be located on disk. With AMR
+ data, this is typically quite easy: the grid patches are the "first pass" at
+ localization.
+ * Data reading: This is the set of routines that actually perform a read of
+ either all data in a region or a subset of that data.
+
+Data Meaning Structures
+-----------------------
+
+If you are interested in adding a new code, be sure to drop us a line on
+`yt-dev <http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org>`_!
+
+To get started, make a new directory in ``yt/frontends`` with the name of your
+code -- you can start by copying into it the contents of the ``stream``
+directory, which is a pretty empty format. You'll then have to create a subclass
+of ``StaticOutput``. This subclass will need to handle conversion between the
+different physical units and the code units; for the most part, the examples of
+``OrionStaticOutput`` and ``EnzoStaticOutput`` should be followed, but
+``ChomboStaticOutput``, as a slightly newer addition, can also be used as an
+instructive example -- be sure to add an ``_is_valid`` classmethod that will
+verify if a filename is valid for that output type, as that is how "load" works.
+
+A new set of fields must be added in the file ``fields.py`` in that directory.
+For the most part this means subclassing ``CodeFieldInfoContainer`` and adding
+the necessary fields specific to that code. Here is the Chombo field container:
+
+.. code-block:: python
+
+ from UniversalFields import *
+ class ChomboFieldContainer(CodeFieldInfoContainer):
+ _shared_state = {}
+ _field_list = {}
+ ChomboFieldInfo = ChomboFieldContainer()
+ add_chombo_field = ChomboFieldInfo.add_field
+
+The field container is a shared state object, which is why we explicitly set
+``_shared_state`` equal to a mutable.
+
+Data Localization Structures
+----------------------------
+
+As of right now, the "grid patch" mechanism is going to remain in yt, however in
+the future that may change. As such, some other output formats -- like Gadget --
+may be shoe-horned in, slightly.
+
+Hierarchy
+^^^^^^^^^
+
+To set up data localization, an ``AMRHierarchy`` subclass must be added in the
+file ``data_structures.py``. The hierarchy object must override the following
+methods:
+
+ * ``_detect_fields``: ``self.field_list`` must be populated as a list of
+ strings corresponding to "native" fields in the data files.
+ * ``_setup_classes``: it's probably safe to crib this from one of the other
+ ``AMRHierarchy`` subclasses.
+ * ``_count_grids``: this must set self.num_grids to be the total number of
+ grids in the simulation.
+ * ``_parse_hierarchy``: this must fill in ``grid_left_edge``,
+ ``grid_right_edge``, ``grid_particle_count``, ``grid_dimensions`` and
+ ``grid_levels`` with the appropriate information. Additionally, ``grids``
+ must be an array of grid objects that already know their IDs.
+ * ``_populate_grid_objects``: this initializes the grids by calling
+ ``_prepare_grid`` and ``_setup_dx`` on all of them. Additionally, it should
+ set up ``Children`` and ``Parent`` lists on each grid object.
+ * ``_setup_unknown_fields``: If a field is in the data file that yt doesn't
+ already know, this is where you make a guess at it.
+ * ``_setup_derived_fields``: ``self.derived_field_list`` needs to be made a
+ list of strings that correspond to all derived fields valid for this
+ hierarchy.
+
+For the most part, the ``ChomboHierarchy`` should be the first place to look for
+hints on how to do this; ``EnzoHierarchy`` is also instructive.
+
+Grids
+^^^^^
+
+A new grid object, subclassing ``AMRGridPatch``, will also have to be added.
+This should go in ``data_structures.py``. For the most part, this may be all
+that is needed:
+
+.. code-block:: python
+
+ class ChomboGrid(AMRGridPatch):
+ _id_offset = 0
+ __slots__ = ["_level_id"]
+ def __init__(self, id, hierarchy, level = -1):
+ AMRGridPatch.__init__(self, id, filename = hierarchy.hierarchy_filename,
+ hierarchy = hierarchy)
+ self.Parent = []
+ self.Children = []
+ self.Level = level
+
+
+Even the most complex grid object, ``OrionGrid``, is still relatively simple.
+
+Data Reading Functions
+----------------------
+
+In ``io.py``, there are a number of IO handlers that handle the mechanisms by
+which data is read off disk. To implement a new data reader, you must subclass
+``BaseIOHandler`` and override the following methods:
+
+ * ``_read_field_names``: this routine accepts a grid object and must return all
+ the fields in the data file affiliated with that grid. It is used at the
+ initialization of the ``AMRHierarchy`` but likely not later.
+ * ``modify``: This accepts a field from a data file and returns it ready to be
+ used by yt. This is used in Enzo data for preloading.
+ * ``_read_data_set``: This accepts a grid object and a field name and must
+ return that field, ready to be used by yt as a NumPy array. Note that this
+ presupposes that any actions done in ``modify`` (above) have been executed.
+ * ``_read_data_slice``: This accepts a grid object, a field name, an axis and
+ an (integer) coordinate, and it must return a slice through the array at that
+ value.
+ * ``preload``: (optional) This accepts a list of grids and a list of datasets
+ and it populates ``self.queue`` (a dict keyed by grid id) with dicts of
+ datasets.
+ * ``_read_exception``: (property) This is a tuple of exceptions that can be
+ raised by the data reading to indicate a field does not exist in the file.
+
+
+And that just about covers it. Please feel free to email
+`yt-users <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>`_ or
+`yt-dev <http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org>`_ with
+any questions, or to let us know you're thinking about adding a new code to yt.
diff -r c4f4cbbb167da2be7779965ba0e24af774aa9d97 -r 45a9e83d961a1addf095dc5abaadcc8a92962338 source/advanced/developing.rst
--- a/source/advanced/developing.rst
+++ b/source/advanced/developing.rst
@@ -63,7 +63,7 @@
``yt/utilities/command_line.py`` in the function ``do_bootstrap``.
Here is the list of items that the script will attempt to accomplish, along
-with a brief motivation of each.
+with a brief motivation of each.
#. **Ensure that the yt-supplemental repository is checked out into
``YT_DEST``**. To make sure that the extensions we're going to use to
@@ -107,7 +107,7 @@
start developing yt efficiently.
.. _included-hg-extensions:
-
+
Included hg Extensions
^^^^^^^^^^^^^^^^^^^^^^
@@ -160,18 +160,18 @@
If you just want to *look* at the source code, you already have it on your
computer. Go to the directory where you ran the install_script.sh, then
-go to ``$YT_DEST/src/yt-hg`` . In this directory are a number of
+go to ``$YT_DEST/src/yt-hg`` . In this directory are a number of
subdirectories with different components of the code, although most of them
are in the yt subdirectory. Feel free to explore here. If you're looking
-for a specific file or function in the yt source code, use the unix find
+for a specific file or function in the yt source code, use the unix find
command:
.. code-block:: bash
$ find <DIRECTORY_TREE_TO_SEARCH> -name '<FILENAME>'
-The above command will find the FILENAME in any subdirectory in the
-DIRECTORY_TREE_TO_SEARCH. Alternatively, if you're looking for a function
+The above command will find the FILENAME in any subdirectory in the
+DIRECTORY_TREE_TO_SEARCH. Alternatively, if you're looking for a function
call or a keyword in an unknown file in a directory tree, try:
.. code-block:: bash
@@ -180,8 +180,8 @@
This can be very useful for tracking down functions in the yt source.
-While you can edit this source code and execute it on your local machine,
-you will be unable to share your work with others in the community (or
+While you can edit this source code and execute it on your local machine,
+you will be unable to share your work with others in the community (or
get feedback on your work). If you want to submit your modifications to the
yt project, follow the directions below.
@@ -191,32 +191,32 @@
yt is hosted on BitBucket, and you can see all of the yt repositories at
http://hg.yt-project.org/ . With the yt installation script you should have a
copy of Mercurial for checking out pieces of code. Make sure you have followed
-the steps above for bootstrapping your development (to assure you have a
+the steps above for bootstrapping your development (to assure you have a
bitbucket account, etc.)
In order to access the source code for yt, we ask that you make a "fork" of
the main yt repository on bitbucket. A fork is simply an exact copy of the
main repository (along with its history) that you will now own and can make
-modifications as you please. You can create a personal fork by visiting the yt
+modifications as you please. You can create a personal fork by visiting the yt
bitbucket webpage at https://bitbucket.org/yt_analysis/yt/wiki/Home . After
-logging in, you should see an option near the top right labeled "fork".
+logging in, you should see an option near the top right labeled "fork".
Click this option, and then click the fork repository button on the subsequent
page. You now have a forked copy of the yt repository for your own personal
use.
This forked copy exists on the bitbucket repository, so in order to access
-it locally, follow the instructions at the top of that webpage for that
+it locally, follow the instructions at the top of that webpage for that
forked repository, namely run at a local command line:
.. code-block:: bash
$ hg clone http://bitbucket.org/<USER>/<REPOSITORY_NAME>
-This downloads that new forked repository to your local machine, so that you
+This downloads that new forked repository to your local machine, so that you
can access it, read it, make modifications, etc. It will put the repository in
-a local directory of the same name as the repository in the current working
+a local directory of the same name as the repository in the current working
directory. You can see any past state of the code by using the hg log command.
-For example, the following command would show you the last 5 changesets
+For example, the following command would show you the last 5 changesets
(modifications to the code) that were submitted to that repository.
.. code-block:: bash
@@ -224,8 +224,8 @@
$ cd <REPOSITORY_NAME>
$ hg log -l 5
-Using the revision specifier (the number or hash identifier next to each
-changeset), you can update the local repository to any past state of the
+Using the revision specifier (the number or hash identifier next to each
+changeset), you can update the local repository to any past state of the
code (a previous changeset or version) by executing the command:
.. code-block:: bash
@@ -234,9 +234,9 @@
Lastly, if you want to use this new downloaded version of your yt repository
as the *active* version of yt on your computer (i.e. the one which is executed
-when you run yt from the command line or ``from yt.mods import *``),
-then you must "activate" it using the following commands from within the
-repository directory.
+when you run yt from the command line or ``from yt.mods import *``),
+then you must "activate" it using the following commands from within the
+repository directory.
In order to do this for the first time with a new repository, you have to
copy some config files over from your yt installation directory (where yt
@@ -266,8 +266,8 @@
to the repository, but they need to be reviewed/tested by other users of
the code before they're pulled into the main repository.
-When you're ready to submit them to the main repository, simply go to the
-bitbucket page for your personal fork of the yt-analysis yt repository,
+When you're ready to submit them to the main repository, simply go to the
+bitbucket page for your personal fork of the yt-analysis yt repository,
and click the button to issue a pull request (at top right):
Make sure you notify ``yt_analysis`` and put in a little description. That'll
@@ -315,7 +315,7 @@
.. code-block:: bash
- $ ``cp yt-old/src/yt-hg/*.cfg yt-testing``
+ $ ``cp yt-old/src/yt-hg/*.cfg yt-testing``
$ cd yt-testing
$ python setup.py develop
@@ -330,7 +330,7 @@
$ cd yt-old/src/yt-hg
$ python setup.py develop
-If you want to accept the changeset or reject it (if you have sufficient
+If you want to accept the changeset or reject it (if you have sufficient
priveleges) or comment on it, you can do so from its pull request webpage.
How To Read The Source Code
@@ -512,14 +512,13 @@
``yt`` strives to be a general-purpose analysis tool for astrophysical data.
To that end, we'd like to short up our support for codes besides Enzo, as well
-as ensure that the other codes we support -- Orion, Tiger, etc -- are
+as ensure that the other codes we support -- Orion, Tiger, etc. -- are
well-supported.
-`A page has been set up <http://yt-project.org/wiki/AddingSupportForANewCode>`_
-on the Trac site to describe the method of adding support for a new code to
-``yt``. Please feel free to use it as a reference, but if you would like some
-assistance, drop a line to one of the mailing lists (see :ref:`mailing-list`)
-for more help.
+The :ref:`creating_frontend` page describes the process of adding support for a
+new code to ``yt``. Please feel free to use it as a reference, but if you would
+like some assistance, drop a line to one of the mailing lists (see
+:ref:`mailing-list`) for more help.
GUIs and Interactive Exploration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Repository URL: https://bitbucket.org/yt_analysis/yt-doc/
--
This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.
More information about the yt-svn
mailing list