[yt-users] Best way to load in halos?

[Matthew Turk]

Hi all,

tl;dr: Mass column sounds fine, as long as we recognize that it's not
going to agree with what yt would calculate from the other info.

Christine, you're absolutely not "butting in" -- thank you for your thoughts.

Initially when I saw your exchange with Stephen, my thought was that
the radius is the more fundamental component when dealing with halos,
since one could in principle extract back out the mass.  This operates
under the assumption -- which you note -- that the radii of the
particle that's furthest from the center governs the halos.  As
Geoffrey pointed out, this is not a terribly good assumption.  That
being said, I still consider the radius to be more fundamental, from
the perspective of reconstructing a useful halo object.

For the purposes of visualization, it's fine.  All of our viz that
uses baryon quantities will not be bothered by this.  But for things
that look at baryonic quantities, we might not really want the halo
(which is triaxially) to be defined only by a single quantity.  So,
reconstructing a halo from a radius is probably sufficient for some
tasks, but it is lossy, and poor.

Really, though, the question we're dancing around here is somewhat
more intrusive.

The point of finding halos and loading them into yt is not so that we
can *have* halos in yt, it's so that we can do something with them.
That something could be making visualizations, tracking particles,
analyzing baryonic quantities and so on.  Some of these need the exact
particles that belonged to a halo, others don't.  For instance, right
now identify which fluid cells/parcels "belong" to a given halo is
somewhat ill-defined.  We use the radius to figure out which cells
belong to the halo, and we use spheres.  Hopefully in the future we
can use Geoffrey's ellipsoidal stuff, but that's not fully in place
yet.  So when we do any kind of analysis, now, on fluid that's
affiliated with particle collections, we have to make assumptions
about which fluid parcels contribute.

Britton's points are also quite good, in that one can provide a
secondary calibration of the halo size based on virial arguments.

So perhaps it's not necessarily interesting to talk about loading in
halos unless they explicitly come with lists of particles, and even
then saying which cells (in a mesh code) belong to a given halo is not
necessarily completely well-defined.  For my purposes, I think I'm
okay with loading in spheres as "halo" objects, getting the particles
from the file, and then applying traditional "halo" analysis to that.

But, we're kind of at a funny spot here.  (And maybe this should be
moved over to yt-dev.)  I think there is a place for loading in some
kind of information from an external source, and using that to guide
subsequent analysis.  In this case, the source was a halo finder, and
I wanted to apply analysis of things like examining ionization
fraction and SFR.  In fact, I think I *prefer* that to the idea that
yt needs to have every possible halo finder, every possible [thing] in
it, and that all analysis has to happen internally.  That kind of
philosophy doesn't scale, and isn't terribly inclusive.

And, really, I think we open up a lot of really interesting
collaborations between yt and (for instance) SAM codes if we start
thinking about halo objects not necessarily as collections of
particles, but also semantically meaningful objects.  Something to
think about when discussing halo finding in yt, I suppose.

Well, anyway, this has kind of diverged.  I'm okay with having a mass
column, but I want to point out that the mass column won't agree with
the mass calculated by yt; the only way for that to happen is to have
the halo catalog also contain particle information.  And the only way
to take particle information and get back those specific particles is
to do search through particle IDs to match them up.  This should be
fast, but acceptable.

Anyway, thanks to everyone for your thoughts -- and to Stephen for the
implementation!

-Matt

On Mon, Jul 9, 2012 at 8:07 PM, Geoffrey So <gsiisg at gmail.com> wrote:
> Getting back to "getting the halo", I have a dumb thought.  I was thinking
> since the maximum radius is known, YT can extract the particles/fields in
> the sphere and call that a new halo, but you can also do a halo finding in
> that sphere and just take the biggest halo as the halo that was meant by the
> center and max radius right?  But this of course requires you to know how
> the original haloes were generated (specifically which halo finder it used),
> and if an matching method exists in YT....  Come to think of it, if you know
> the original halo finding method and it exists in YT, you can just run the
> halo finder in YT and use .dump() to save the info, the only reason not to
> do that is to save computation time, but those are cheap nowadays.
>
> From
> G.S.
>
> On Mon, Jul 9, 2012 at 5:52 PM, Britton Smith <brittonsmith at gmail.com>
> wrote:
>>
>> Hi everyone,
>>
>> The radius, even if somewhat arbitrary is a very useful quantity to have
>> on hand for a halo.  With the halo profiler, for example, the halo radius is
>> used to determine the maximum radius out to which radial profiling should be
>> done.  Without that, there's no good way to decide how far out radial
>> profiles should be taking.  Setting a constant value is generally not a good
>> idea, since it can vary a lot from situation to situation.  With the halo
>> profiler, you can then calculate virial mass and radius in a way that also
>> includes the baryons, something that the halo finder cannot do.  In
>> conclusion, I strongly support reading in a radius from a halo catalog.
>>
>> Britton
>>
>>
>> On Mon, Jul 9, 2012 at 7:06 PM, Christine Simpson
>> <csimpson at astro.columbia.edu> wrote:
>>>
>>> Hi Stephen,
>>>
>>> I didn't mean to butt into your discussion with Matt, sorry.  I don't
>>> want to make more work for you and I totally understand if this
>>> suggestion doesn't fit in with your vision of the yt halo routines.
>>>
>>> I will say, however, that I actually have somewhat of the opposite
>>> opinion on the issue that you raise.  To me the radius is an arbitrarily
>>> calculated value and the mass that the halo finder returns is what is
>>> more fundamental.  Of course, yt can't know the particles that an
>>> external halo finder identified as belonging to a halo unless that
>>> information is provided, but I would argue that having a 'radius'
>>> doesn't give you that information either.
>>>
>>> It's been a while since I've looked into the halo finding and handling
>>> routines in yt, so it is very likely that my information is outdated (I
>>> apologize if this is the case), but I recall that the default for the
>>> halo 'radius' was the radius of the furthest group particle.  For a halo
>>> finder like hop that can return very irregular halos, this makes little
>>> physical sense in my opinion.  I believe there was at one time an option
>>> to use what the docs called the 'r200' radius, but it was defined
>>> relative to the mean density of the universe.  I guess it is okay to
>>> define it this way, but it is rather non-standard (I would define it
>>> relative to the critical density of the universe).  Either way, this
>>> radius is defined by the mass of the halo (and the redshift).  I
>>> understand that cosmology is messy, but I think it is better to use that
>>> radius than the distance of the furthest particle, which could be quite
>>> arbitrary in many respects.
>>>
>>> I guess I would just say that for me, I'd rather know what total mass
>>> the halo finder identified and then work from there (i.e. define a
>>> radius and then a sphere object).  I can see that others may have other
>>> opinions and halo finders that may operate in other ways.  I guess my
>>> one piece of advice would be to try to make routines as general as
>>> possible.  I sometimes feel in yt that there is an unnecessary degree of
>>> specialization.  Of course, I love yt and I say this from a place of
>>> sincere appreciation for the package and its developers.
>>>
>>> Christine
>>>
>>> On Mon, 2012-07-09 at 16:23 -0600, Stephen Skory wrote:
>>> > Christine,
>>> >
>>> > > A suggestion, if it isn't too much trouble.  Could you add a column
>>> > > for
>>> > > mass?
>>> >
>>> > Actually, just as I sent my last reply I thought a bit harder, and I
>>> > realized that this raises a prickly issue. Let me explain.
>>> >
>>> > When you run a halo finder in yt, and save your results with .dump(),
>>> > you can use LoadHaloes() to recover carbon-copies of what you had
>>> > before. Any operation you ran on the halos before .dump() will give
>>> > you the same answer with them loaded later on. This is because with
>>> > .dump() everything that makes up the halos, in particular the full
>>> > particle assignments, is saved to disk.
>>> >
>>> > However, with this text loading method, we don't know anything about
>>> > what particles are in the halo when the halo finder was run. This
>>> > means that all we can do (and it's what I'm doing in this new method)
>>> > is simply define the halo as a sphere centered on a point. If I ask
>>> > for particle information from this halo, it will use the sphere.
>>> > Typically, halos found by halo finders are not spheres.
>>> >
>>> > Now, if we read in mass from a text file as you suggest, what mass is
>>> > this? If it's the mass of the particles as the halo finder you used
>>> > calculated, what does that mean in yt? It can't reproduce this mass on
>>> > its own, and this mass doesn't correspond to anything yt knows about,
>>> > namely, a nondescript sphere.
>>> >
>>> > I'll have to think about this some more, but the TL;DR is that
>>> > specifying a mass like this from a text file introduces some
>>> > ambiguity.
>>> >
>>> > I'm still open to other column suggestions.
>>> >
>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>