[yt-users] Light Cone Projection

[Pengfei Chen]

OK. Thank you, Britton!

Pengfei


On Sat, Mar 1, 2014 at 8:35 AM, Britton Smith <brittonsmith at gmail.com>wrote:

> Hi Pengfei,
>
> This can happen sometime just due to roundoff error or error that crops up
> when converting back and forth between redshift and time.  Since the box
> fraction is only marginally greater than 1 for that segment of the light
> ray, it shouldn't really be a problem to use these datasets.  You should
> still be able to make the light ray and just ignore the error message.
>
> Britton
>
>
> On Thu, Feb 27, 2014 at 8:20 AM, Pengfei Chen <madcpf at gmail.com> wrote:
>
>> Hi Britton,
>>
>> Last time when I asked you about minimum_coherent_box_fraction in
>> light_ray in this email, you told me that "one should stick with the option
>> of using the smallest number of datasets as possible to traverse a redshift
>> interval." Now I have some datasets expanding from redshift 5 to 3, which
>> have enough outputs but didn't use the subroutine plan_cosmology_splice to
>> plan outputs. Now when I use the subroutine Lightray I got some problems:
>>
>> If I set use_minimum_datasets=True, it would have errors like this:
>>
>> P014 yt : [ERROR    ] 2014-02-27 01:24:21,405 Warning: box fraction
>> required to go from z = 3.684226 to 3.659368 is 1.001284
>> P013 yt : [ERROR    ] 2014-02-27 01:24:21,405 Full box delta z is
>> 0.024826, but it is 0.024857 to the next data dump.
>>
>> I don't know why there are such errors given that I have enough datasets.
>> The generated light solution file is like this:
>> http://paste.yt-project.org/show/4335/
>> But I'm not sure if this lightray is good to use with the above error
>> message.
>>
>> If I set use_minimum_datasets=False, all the datasets would be used and
>> no errors are shown. The output light solution file is like this:
>> http://paste.yt-project.org/show/4334/
>> Also I'm not sure if I should use this with the potential problem about
>> "minimum_coherent_box_fraction".
>>
>> Another thing I could do is to keep use_minimum_datasets=False but reduce
>> the datasets used manually until the minimum number of datasets are used
>> with no error messages. Could you please give me some advice about what I
>> should do?
>>
>> Thank you!
>> Pengfei
>>
>>
>>
>> On Mon, Sep 30, 2013 at 4:36 AM, Britton Smith <brittonsmith at gmail.com>wrote:
>>
>>> Hi Pengfei,
>>>
>>> To answer your first question, the minimum_coherent_box_fraction option
>>> is a bit of a failed experiment in which one is using as many datasets as
>>> possible in the light cone, instead of using as few as possible.  In this
>>> case, we use the same ray trajectory through multiple datasets until the
>>> ray has extended the length of one box.  This was to avoid making cuts in
>>> halos or filaments by using thin slices of each dataset.  Functionally,
>>> this works by waiting to re-randomize the ray trajectory until you've gone
>>> through a full box length.  In practice, doing this produced a number of
>>> artifacts in the light cone projections so should probably be used with
>>> caution in the light rays.  I think in reality, one should stick with the
>>> option of using the smallest number of datasets as possible to traverse a
>>> redshift interval.
>>>
>>> On your second question, the problem is actually that, if you start from
>>> a random position in a periodic cube of box length L, you only return to
>>> that position after moving a distance of L if you move exactly along the
>>> axis directions.  As another example, if you started in the corner of the
>>> box and went toward the opposite corner (which is the same point since it's
>>> periodic) you would have to travel a distance of sqrt(3) * L to get there.
>>> In reality, though, you probably do not want to be starting from the same
>>> point in each dataset, since you will then just be resampling the same
>>> structure over and over.
>>>
>>> Other than that, the code itself looks just fine.  Please, let me know
>>> if I haven't answered your questions.
>>>
>>> Britton
>>>
>>>
>>> On Wed, Sep 25, 2013 at 12:19 PM, Pengfei Chen <madcpf at gmail.com> wrote:
>>>
>>>> Hi Britton,
>>>>
>>>> Thank you very much for your detailed instructions. I've added two
>>>> parameters in _calculate_light_ray_solution in light_ray.py. They are
>>>> 'start_point' and 'start_direction', each of which is supposed to be a
>>>> 3-element list/array. Their default values are None and they can be used
>>>> together or individually. The code is attached and it works. But I still
>>>> have two problems:
>>>>
>>>> (1) I am confused about the parameter "minimum_coherent_box_fraction",
>>>> and what the following statement is trying to do:
>>>>
>>>>             if (q == 0) or (self.minimum_coherent_box_fraction == 0) or
>>>> \
>>>>                     (box_fraction_used >
>>>>                      self.minimum_coherent_box_fraction) or \
>>>>                     (box_fraction_used +
>>>>
>>>>  self.light_ray_solution[q]['traversal_box_fraction'] > 1.0):
>>>>
>>>> What does "coherent large scale structure" and "box coherence" mean?
>>>>
>>>> (2) When I am using minimum datasets the start point of each dataset is
>>>> not same as the end point of the last dataset even if I
>>>> set minimum_coherent_box_fraction = 1. Because
>>>>
>>>>                      box_fraction_used +
>>>>
>>>>  self.light_ray_solution[q]['traversal_box_fraction'] > 1.0
>>>>
>>>> is always satisfied when using minimum datasets. I don't know the right
>>>> way to get start point to be the same as the end point of the last dataset,
>>>> so I tried to change the statements in (1) to
>>>>            if (q == 0):
>>>> I find it works for(parameters when call make_light_ray):
>>>>                   start_point=None,
>>>>                   start_direction=None,
>>>> and
>>>>                   start_point=None,
>>>>                   start_direction=[1,1,0],
>>>> and
>>>>                   start_point=[0.5,0.5,0.5],
>>>>                   start_direction=[1,1,0],
>>>> But not
>>>>                   start_point=[0.5,0.5,0.5],
>>>>                   start_direction=None,
>>>>
>>>> For the last case it shows the following error:
>>>>
>>>> yt : [INFO     ] 2013-09-25 04:07:18,162 Getting the binary hierarchy
>>>> Parsing Hierarchy100%
>>>> |||||||||||||||||||||||||||||||||||||||||||||||||||||||||| Time: 00:00:00
>>>> yt : [INFO     ] 2013-09-25 04:07:18,185 Gathering a field list (this
>>>> may take a moment.)
>>>> yt : [INFO     ] 2013-09-25 04:07:18,443 Getting field t from 0
>>>> Traceback (most recent call last):
>>>>   File "make_light_ray.py", line 68, in <module>
>>>>     njobs=-1)
>>>>   File
>>>> "/Users/madcpf/Documents/yt/yt/analysis_modules/cosmological_observation/light_ray/light_ray.py",
>>>> line 388, in make_light_ray
>>>>     (sub_ray['dts'] *
>>>>   File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py",
>>>> line 318, in __getitem__
>>>>     self.get_data(key)
>>>>   File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py",
>>>> line 525, in get_data
>>>>     for grid in self._grids])
>>>> ValueError: need at least one array to concatenate
>>>>
>>>> Could you please take a look when you have time?
>>>>
>>>> Thank you so much!
>>>>
>>>> Pengfei
>>>>
>>>>
>>>>
>>>>
>>>> 2013/9/11 Britton Smith <brittonsmith at gmail.com>
>>>>
>>>>> Hi Pengfei,
>>>>>
>>>>> That is right.  Using plan_cosmology_splice will allow you to minimize
>>>>> the number of outputs needed to traverse a given redshift interval.  Note,
>>>>> that since the start and stop positions in each dataset are random, there
>>>>> is no guarantee that you will not pass through the same structure more than
>>>>> once, although the odds are quite low.
>>>>>
>>>>> I'm very glad to hear you're interested in working on this.  For
>>>>> starters, you might want to read our documentation page on developing for
>>>>> yt, which is here:
>>>>> http://yt-project.org/doc/advanced/developing.html
>>>>> This will explain some basics about how to work in a way that will
>>>>> make it easiest for your contributions to be pulled into the main
>>>>> repository.
>>>>>
>>>>> As for specifics on what you want to do, you should check out the yt
>>>>> source file:
>>>>> yt/analysis_modules/cosmological_observation/light_ray/light_ray.py
>>>>>
>>>>> Have a look at the routine called _calculate_light_ray_solution.  This
>>>>> routine is called by make_light_ray, and is responsible for creating the
>>>>> list of starting and ending positions for each ray segment.  Starting on
>>>>> line 165, you can see where the random initial position for a segment is
>>>>> calculated.  Here is where you would want to add the ability to set a fixed
>>>>> position.  A few lines down, a random trajectory is calculated, which is
>>>>> used along with the length of the ray segment to determine the end point.
>>>>>
>>>>> This should get you started.  Please, let us know if you have any more
>>>>> questions.  Additionally, you might consider joining the developers mailing
>>>>> list if you think you would like to continue working on yt, which would be
>>>>> great!  Good luck!
>>>>>
>>>>> Britton
>>>>>
>>>>>
>>>>> On Wed, Sep 11, 2013 at 12:59 AM, Pengfei Chen <madcpf at gmail.com>wrote:
>>>>>
>>>>>> Hi Britton,
>>>>>>
>>>>>> Thank you so much for your reply! I think I understand how LightRay
>>>>>> works. What plan_cosmology_splice() does is to choose output redshifts so
>>>>>> that comoving distance between them is approximately equal to but less than
>>>>>> the comoving box size. So the light ray would not pass through the same
>>>>>> structure in one dataset no matter how it travels. Is it right?
>>>>>>
>>>>>> Also I would like to add support to the LightRay object for setting a
>>>>>> fixed initial position. Since I never modified yt or any other large code
>>>>>> before, could you please give me some advice? Is there any documentation or
>>>>>> tutorials I could read?
>>>>>>
>>>>>> Have a nice day!
>>>>>>
>>>>>> Pengfei
>>>>>>
>>>>>>
>>>>>> 2013/9/9 Britton Smith <brittonsmith at gmail.com>
>>>>>>
>>>>>>> Hi Pengfei,
>>>>>>>
>>>>>>> Thank you for catching all of those errors in the source and
>>>>>>> documentation.  As of now, they have all been fixed and if you do "yt
>>>>>>> update" the recipe should run as normal without the need to alter anything.
>>>>>>>
>>>>>>> I can answer your additional questions here.
>>>>>>>
>>>>>>> (1) What's the initial spectrum before any absorption?
>>>>>>>>
>>>>>>>
>>>>>>> The spectrum generator only creates absorption features.
>>>>>>> Essentially, it assumes that you have already subtracted off the continuum,
>>>>>>> so effectively the input spectrum is just f(lambda) = 1.0.
>>>>>>>
>>>>>>>
>>>>>>>> (2) Where does the light ray originate from? Is it shoot from a
>>>>>>>> random position in the first dataset? Can we set it manually like on the
>>>>>>>> surface of the box? Does it go out of the box and back into it until the
>>>>>>>> next output redshift?
>>>>>>>>
>>>>>>>
>>>>>>> The light ray originates in a random position inside the first
>>>>>>> dataset with a random trajectory.  This code assumes a periodic domain, so
>>>>>>> the ray will wrap around if it leaves the box.  Once it has traversed a
>>>>>>> distance equivalent to the comoving distance to the next dataset, it
>>>>>>> terminates and begins a new ray there.  Currently, you can not manually set
>>>>>>> the initial position of a LightRay, but you can also using the ray object (
>>>>>>> http://yt-project.org/doc/analyzing/objects.html#available-objects).
>>>>>>> If you are interested in adding support to the LightRay object for setting
>>>>>>> a fixed initial position (and presumably a fixed end position), let me know
>>>>>>> and I can help you get started.  If you were to do this, we would be very
>>>>>>> pleased do have you contribute this code back to the main code base.
>>>>>>>
>>>>>>>
>>>>>>>> (3) In the output file lightraysolution.txt, what do the
>>>>>>>> parameters(dl/box, Start x/y/z, End x/y/z) mean?
>>>>>>>>
>>>>>>>
>>>>>>> Since the start and stop positions are randomly chosen, this file
>>>>>>> shows what those random positions are for each dataset, in case you want to
>>>>>>> know that.  dl/box is the fraction of the box length that was used for a
>>>>>>> ray in that dataset.  For more information on this, I would suggest you
>>>>>>> check out this page:
>>>>>>>
>>>>>>> http://yt-project.org/doc/analysis_modules/light_ray_generator.html#light-ray-generator
>>>>>>>
>>>>>>> Britton
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Sorry that I have so many questions... I wonder if there is some
>>>>>>>> existing documents or paper that have a more detailed description.
>>>>>>>>
>>>>>>>> Thank you again for your help! Have a nice weekend!
>>>>>>>>
>>>>>>>> Pengfei
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2013/9/6 Britton Smith <brittonsmith at gmail.com>
>>>>>>>>
>>>>>>>>> Hi Pengfei,
>>>>>>>>>
>>>>>>>>> The near_redshift and far_redshift are free parameters for you to
>>>>>>>>> set, and just control the redshift interval over which to make the light
>>>>>>>>> cone projection.  You might also want to look at the narrative
>>>>>>>>> documentation on the light cone generator, which you can find here:
>>>>>>>>>
>>>>>>>>> http://yt-project.org/doc/analysis_modules/light_cone_generator.html
>>>>>>>>> Note, that you need to use this with a simulation that made enough
>>>>>>>>> data outputs to connect the redshift interval you specify.  The
>>>>>>>>> "enzo_tiny_cosmology" dataset available at
>>>>>>>>> http://yt-project.org/data/ was configured to work with the light
>>>>>>>>> cone recipe in the cookbook, so I would suggest downloading that.
>>>>>>>>>
>>>>>>>>> As for the error, this is happening because there is a line in
>>>>>>>>> your enzo parameter file that contains more than one "=" sign.  I have just
>>>>>>>>> submitted a pull request that fixes it, so this should be fixed soon.  In
>>>>>>>>> the mean time, you can simply comment out that line in your parameter file
>>>>>>>>> and everything should work fine.
>>>>>>>>>
>>>>>>>>> Britton
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, Sep 6, 2013 at 3:02 AM, Pengfei Chen <madcpf at gmail.com>wrote:
>>>>>>>>>
>>>>>>>>>> Hi all,
>>>>>>>>>>
>>>>>>>>>> I downloaded the example script of light cone projectoin(
>>>>>>>>>> http://yt-project.org/doc/cookbook/cosmological_analysis.html#light-cone-projection)
>>>>>>>>>> to my laptop and changed the Enzo simulation parameter file and redshifts.
>>>>>>>>>> But when I ran it the following error occurred:
>>>>>>>>>>
>>>>>>>>>>   File "light_cone_projection.py", line 19, in <module>
>>>>>>>>>>     find_outputs=False)
>>>>>>>>>>   File
>>>>>>>>>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/light_cone/light_cone.py",
>>>>>>>>>> line 158, in __init__
>>>>>>>>>>     find_outputs=find_outputs)
>>>>>>>>>>   File
>>>>>>>>>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/cosmology_splice.py",
>>>>>>>>>> line 44, in __init__
>>>>>>>>>>     find_outputs=find_outputs)
>>>>>>>>>>   File
>>>>>>>>>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/convenience.py", line 131, in
>>>>>>>>>> simulation
>>>>>>>>>>     find_outputs=find_outputs)
>>>>>>>>>>   File
>>>>>>>>>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py",
>>>>>>>>>> line 81, in __init__
>>>>>>>>>>     find_outputs=find_outputs)
>>>>>>>>>>   File
>>>>>>>>>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/data_objects/time_series.py",
>>>>>>>>>> line 343, in __init__
>>>>>>>>>>     self._parse_parameter_file()
>>>>>>>>>>   File
>>>>>>>>>> "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py",
>>>>>>>>>> line 273, in _parse_parameter_file
>>>>>>>>>>     param, vals = (i.strip() for i in line.split("="))
>>>>>>>>>> ValueError: need more than 1 value to unpack
>>>>>>>>>>
>>>>>>>>>> Does the parameters "near_redshift" and "far_redshift" in
>>>>>>>>>> LightCone need to be the same with "CosmologyFinalRedshift" and
>>>>>>>>>> "CosmologyInitialRedshift" in enzo parameter file?
>>>>>>>>>> Does anyone know how to solve this?
>>>>>>>>>>
>>>>>>>>>> Thank you all!
>>>>>>>>>>
>>>>>>>>>> Pengfei
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> yt-users mailing list
>>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-users mailing list
>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> yt-users mailing list
>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20140301/29df9d41/attachment.htm>