[Yt-dev] Tsz Ka's merger tree problem

[Matthew Turk]

Hi Stephen et al,

For everyone else, what is going on is that the reorganization of yt
has required more imports to populate a self-registering plugin system
for output files.  Each StaticOutput subclass defines an _is_valid()
function, which means "This is the type of data you have!"  This is
used by the load() command, so if the subclass is not imported (i.e.,
defined in the registry) the _is_valid function can't be found and it
can't be a target for load().

If you do:

from yt.analysis_mods.... import something

but don't import the right frontend, then it can't be found.  yt.mods
imports all the frontends, so normal usage is not affected by this.

So on one hand, this is operating exactly as intended.  The
output_type_registry is filled in by a metaclass, whenever a new class
is created; this is the self-registering mechanism.  It means that
frontends for codes are 100% contained within their directory, and
there's no pollution -- if, for instance, the ramses++ headers
interfere with static linking, you can avoid loading them by not
loading that frontend.

This is why I recommend importing from yt.mods in all the recipes;
this imports most or all of the frontends, populates the registry, and
makes them all available to load.  It's also why I added the amods
object to yt.mods, which is an object that on-demand imports analysis
modules; this helps us avoid having to import everything all at once
(adds substantial time to loading, can also present problems that
would otherwise be non-fatal, etc etc) but also makes available all
the analysis modules.  So you could for instance:

from yt.mods import *
pf = load("...")
amods.halo_finding.HaloFinder(pf)

Anyway, I think that if we want to preserve on-demand importing, which
I very much would like to preserve, we will have to have two levels of
fallback for the _is_valid function.  The first, and default, would be
for any subclasses of StaticOutput that define something; this is what
we already do.

The second would be to provide a set of loader functions:

def enzo_is_valid(...)
    if ...:
        import yt.frontends.enzo.api
        return "enzo"

or something like that.  We would then define or copy these right from
the existing definitions, but perhaps only for the most common codes
(Enzo, Orion, FLASH.)  If a dataset is not recognized, it could
iterate over all the known is_valid functions that exist independently
of subclasses of StaticOutput.

Anyway, I guess the tl;dr summary is: this is not something I thought
was a problem, it's not a problem if you use yt.mods and the amods
object, but there are ways to get around it.  Is this worth
implementing?

-Matt

On Tue, Feb 15, 2011 at 10:21 PM, Stephen Skory <stephenskory at yahoo.com> wrote:
> Hi Matt,
>
> What I discovered is that without calling "from yt.mods import *", the class StaticOutput() was not being initialized in yt/data_objects/static_output.py, so the output_type_registry dict was empty in convenience.py. Should this concern us?
>
>
>  Stephen Skory
> stephenskory at yahoo.com
> http://stephenskory.com/
> 510.621.3687 (google voice)
> _______________________________________________
> Yt-dev mailing list
> Yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>