<div dir="ltr">OK. Thank you, Britton!<div><br></div><div>Pengfei</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Mar 1, 2014 at 8:35 AM, Britton Smith <span dir="ltr"><<a href="mailto:brittonsmith@gmail.com" target="_blank">brittonsmith@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Pengfei,<div><br></div><div>This can happen sometime just due to roundoff error or error that crops up when converting back and forth between redshift and time. Since the box fraction is only marginally greater than 1 for that segment of the light ray, it shouldn't really be a problem to use these datasets. You should still be able to make the light ray and just ignore the error message.</div>
<span class="HOEnZb"><font color="#888888">
<div><br></div><div>Britton</div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Feb 27, 2014 at 8:20 AM, Pengfei Chen <span dir="ltr"><<a href="mailto:madcpf@gmail.com" target="_blank">madcpf@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Britton,<div><br></div>Last time when I asked you about minimum_coherent_box_fraction in light_ray in this email, you told me that "one should stick with the option of using the smallest number of datasets as possible to traverse a redshift interval." Now I have some datasets expanding from redshift 5 to 3, which have enough outputs but didn't use the subroutine plan_cosmology_splice to plan outputs. Now when I use the subroutine Lightray I got some problems:<div>
<br></div><div>If I set use_minimum_datasets=True, it would have errors like this:</div><div><br></div><div><div>P014 yt : [ERROR ] 2014-02-27 01:24:21,405 Warning: box fraction required to go from z = 3.684226 to 3.659368 is 1.001284</div>
<div>P013 yt : [ERROR ] 2014-02-27 01:24:21,405 Full box delta z is 0.024826, but it is 0.024857 to the next data dump.</div><div><br></div><div>I don't know why there are such errors given that I have enough datasets. The generated light solution file is like this:</div>
<div><div><a href="http://paste.yt-project.org/show/4335/" target="_blank">http://paste.yt-project.org/show/4335/</a></div></div><div>But I'm not sure if this lightray is good to use with the above error message.</div>
<div><br></div>
<div>If I set use_minimum_datasets=False, all the datasets would be used and no errors are shown. The output light solution file is like this:</div><div><a href="http://paste.yt-project.org/show/4334/" target="_blank">http://paste.yt-project.org/show/4334/</a><br>
</div><div>Also I'm not sure if I should use this with the potential problem about "minimum_coherent_box_fraction".</div><div><br></div><div>Another thing I could do is to keep use_minimum_datasets=False but reduce the datasets used manually until the minimum number of datasets are used with no error messages. Could you please give me some advice about what I should do?</div>
</div><div><br></div><div>Thank you!</div><span><font color="#888888"><div>Pengfei</div><div><br></div></font></span></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">
On Mon, Sep 30, 2013 at 4:36 AM, Britton Smith <span dir="ltr"><<a href="mailto:brittonsmith@gmail.com" target="_blank">brittonsmith@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>Hi Pengfei,<br><br></div>To answer your first question, the minimum_coherent_box_fraction option is a bit of a failed experiment in which one is using as many datasets as possible in the light cone, instead of using as few as possible. In this case, we use the same ray trajectory through multiple datasets until the ray has extended the length of one box. This was to avoid making cuts in halos or filaments by using thin slices of each dataset. Functionally, this works by waiting to re-randomize the ray trajectory until you've gone through a full box length. In practice, doing this produced a number of artifacts in the light cone projections so should probably be used with caution in the light rays. I think in reality, one should stick with the option of using the smallest number of datasets as possible to traverse a redshift interval.<br>
<br></div>On your second question, the problem is actually that, if you start from a random position in a periodic cube of box length L, you only return to that position after moving a distance of L if you move exactly along the axis directions. As another example, if you started in the corner of the box and went toward the opposite corner (which is the same point since it's periodic) you would have to travel a distance of sqrt(3) * L to get there. In reality, though, you probably do not want to be starting from the same point in each dataset, since you will then just be resampling the same structure over and over.<br>
<br></div>Other than that, the code itself looks just fine. Please, let me know if I haven't answered your questions.<span><font color="#888888"><br><br>Britton<br></font></span></div><div>
<div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Sep 25, 2013 at 12:19 PM, Pengfei Chen <span dir="ltr"><<a href="mailto:madcpf@gmail.com" target="_blank">madcpf@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Britton,<div><br></div><div>Thank you very much for your detailed instructions. I've added two parameters in _calculate_light_ray_solution in light_ray.py. They are 'start_point' and 'start_direction', each of which is supposed to be a 3-element list/array. Their default values are None and they can be used together or individually. The code is attached and it works. But I still have two problems:</div>
<div><br></div><div>(1) I am confused about the parameter "minimum_coherent_box_fraction", and what the following statement is trying to do:</div><div><br></div><div><div> if (q == 0) or (self.minimum_coherent_box_fraction == 0) or \</div>
<div> (box_fraction_used ></div><div> self.minimum_coherent_box_fraction) or \</div><div> (box_fraction_used +</div><div> self.light_ray_solution[q]['traversal_box_fraction'] > 1.0):</div>
</div><div><br></div><div>What does "coherent large scale structure" and "box coherence" mean?</div><div><br></div><div>(2) When I am using minimum datasets the start point of each dataset is not same as the end point of the last dataset even if I set minimum_coherent_box_fraction = 1. Because</div>
<div><div><br></div><div> box_fraction_used +</div><div> self.light_ray_solution[q]['traversal_box_fraction'] > 1.0</div></div><div><br></div><div>is always satisfied when using minimum datasets. I don't know the right way to get start point to be the same as the end point of the last dataset, so I tried to change the statements in (1) to </div>
<div> if (q == 0):</div><div>I find it works for(parameters when call make_light_ray):</div><div><div> start_point=None,</div><div> start_direction=None, </div></div><div>and </div>
<div><div> start_point=None,</div><div> start_direction=[1,1,0], </div></div><div>and</div><div><div> start_point=[0.5,0.5,0.5],</div><div> start_direction=[1,1,0], </div>
</div><div>But not</div><div><div><div> start_point=[0.5,0.5,0.5],</div><div> start_direction=None, </div></div></div><div><br></div><div>For the last case it shows the following error:</div>
<div><br></div><div><div>yt : [INFO ] 2013-09-25 04:07:18,162 Getting the binary hierarchy</div><div>Parsing Hierarchy100% |||||||||||||||||||||||||||||||||||||||||||||||||||||||||| Time: 00:00:00 </div><div>yt : [INFO ] 2013-09-25 04:07:18,185 Gathering a field list (this may take a moment.)</div>
<div>yt : [INFO ] 2013-09-25 04:07:18,443 Getting field t from 0</div><div><div>Traceback (most recent call last):</div></div><div> File "make_light_ray.py", line 68, in <module></div><div>
njobs=-1)</div><div>
File "/Users/madcpf/Documents/yt/yt/analysis_modules/cosmological_observation/light_ray/light_ray.py", line 388, in make_light_ray</div><div> (sub_ray['dts'] *</div><div> File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py", line 318, in __getitem__</div>
<div> self.get_data(key)</div><div> File "/Users/madcpf/Documents/yt/yt/data_objects/data_containers.py", line 525, in get_data</div><div> for grid in self._grids])</div><div>ValueError: need at least one array to concatenate</div>
</div><div><br></div><div>Could you please take a look when you have time?</div><div><br></div><div>Thank you so much!</div><span><font color="#888888"><div><br></div><div>Pengfei</div><div><br></div><div>
<br></div>
</font></span></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">2013/9/11 Britton Smith <span dir="ltr"><<a href="mailto:brittonsmith@gmail.com" target="_blank">brittonsmith@gmail.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><div><div><div><div><div>Hi Pengfei,<br><br>That is right. Using plan_cosmology_splice will allow you to minimize the number of outputs needed to traverse a given redshift interval. Note, that since the start and stop positions in each dataset are random, there is no guarantee that you will not pass through the same structure more than once, although the odds are quite low.<br>
<br></div>I'm very glad to hear you're interested in working on this. For starters, you might want to read our documentation page on developing for yt, which is here:<br><a href="http://yt-project.org/doc/advanced/developing.html" target="_blank">http://yt-project.org/doc/advanced/developing.html</a><br>
</div>This will explain some basics about how to work in a way that will make it easiest for your contributions to be pulled into the main repository.<br><br></div>As for specifics on what you want to do, you should check out the yt source file:<br>
yt/analysis_modules/cosmological_observation/light_ray/light_ray.py<br><br></div>Have a look at the routine called _calculate_light_ray_solution. This routine is called by make_light_ray, and is responsible for creating the list of starting and ending positions for each ray segment. Starting on line 165, you can see where the random initial position for a segment is calculated. Here is where you would want to add the ability to set a fixed position. A few lines down, a random trajectory is calculated, which is used along with the length of the ray segment to determine the end point.<br>
<br></div>This should get you started. Please, let us know if you have any more questions. Additionally, you might consider joining the developers mailing list if you think you would like to continue working on yt, which would be great! Good luck!<span><font color="#888888"><br>
<br>Britton<br></font></span></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Sep 11, 2013 at 12:59 AM, Pengfei Chen <span dir="ltr"><<a href="mailto:madcpf@gmail.com" target="_blank">madcpf@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Britton,<br><br>Thank you so much for your reply! I think I understand how LightRay works. What plan_cosmology_splice() does is to choose output redshifts so that comoving distance between them is approximately equal to but less than the comoving box size. So the light ray would not pass through the same structure in one dataset no matter how it travels. Is it right?<br>
<br>Also I would like to add support to the LightRay object for setting a fixed initial position. Since I never modified yt or any other large code before, could you please give me some advice? Is there any documentation or tutorials I could read?<br>
<br>Have a nice day!<span><font color="#888888"><br><br>Pengfei<br></font></span></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">2013/9/9 Britton Smith <span dir="ltr"><<a href="mailto:brittonsmith@gmail.com" target="_blank">brittonsmith@gmail.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div>Hi Pengfei,<br><br></div>Thank you for catching all of those errors in the source and documentation. As of now, they have all been fixed and if you do "yt update" the recipe should run as normal without the need to alter anything.<br>
<br></div>I can answer your additional questions here.<br><div class="gmail_extra"><br><div class="gmail_quote"><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr"><div>(1) What's the initial spectrum before any absorption? </div></div></blockquote><div><br></div></div><div>The spectrum generator only creates absorption features. Essentially, it assumes that you have already subtracted off the continuum, so effectively the input spectrum is just f(lambda) = 1.0.<br>
</div><div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>(2) Where does the light ray originate from? Is it shoot from a random position in the first dataset? Can we set it manually like on the surface of the box? Does it go out of the box and back into it until the next output redshift?</div>
</div></blockquote><div><br></div></div><div>The light ray originates in a random position inside the first dataset with a random trajectory. This code assumes a periodic domain, so the ray will wrap around if it leaves the box. Once it has traversed a distance equivalent to the comoving distance to the next dataset, it terminates and begins a new ray there. Currently, you can not manually set the initial position of a LightRay, but you can also using the ray object (<a href="http://yt-project.org/doc/analyzing/objects.html#available-objects" target="_blank">http://yt-project.org/doc/analyzing/objects.html#available-objects</a>). If you are interested in adding support to the LightRay object for setting a fixed initial position (and presumably a fixed end position), let me know and I can help you get started. If you were to do this, we would be very pleased do have you contribute this code back to the main code base.<br>
</div><div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">
<div>(3) In the output file lightraysolution.txt, what do the parameters(dl/box, Start x/y/z, End x/y/z) mean?</div></div></blockquote><div><br></div></div><div>Since the start and stop positions are randomly chosen, this file shows what those random positions are for each dataset, in case you want to know that. dl/box is the fraction of the box length that was used for a ray in that dataset. For more information on this, I would suggest you check out this page: <br>
<a href="http://yt-project.org/doc/analysis_modules/light_ray_generator.html#light-ray-generator" target="_blank">http://yt-project.org/doc/analysis_modules/light_ray_generator.html#light-ray-generator</a><span><font color="#888888"><br>
</font></span></div><span><font color="#888888"><div><br></div><div>Britton<br>
</div></font></span><div><div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>Sorry that I have so many questions... I wonder if there is some existing documents or paper that have a more detailed description. </div>
<div><br></div><div>Thank you again for your help! Have a nice weekend!</div><span><font color="#888888"><div><br></div><div>Pengfei</div><div><br></div></font></span></div><div><div><div class="gmail_extra">
<br><br><div class="gmail_quote">2013/9/6 Britton Smith <span dir="ltr"><<a href="mailto:brittonsmith@gmail.com" target="_blank">brittonsmith@gmail.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div><div>Hi Pengfei,<br><br></div>The near_redshift and far_redshift are free parameters for you to set, and just control the redshift interval over which to make the light cone projection. You might also want to look at the narrative documentation on the light cone generator, which you can find here:<br>
<a href="http://yt-project.org/doc/analysis_modules/light_cone_generator.html" target="_blank">http://yt-project.org/doc/analysis_modules/light_cone_generator.html</a><br></div>Note, that you need to use this with a simulation that made enough data outputs to connect the redshift interval you specify. The "enzo_tiny_cosmology" dataset available at <a href="http://yt-project.org/data/" target="_blank">http://yt-project.org/data/</a> was configured to work with the light cone recipe in the cookbook, so I would suggest downloading that.<br>
<br></div>As for the error, this is happening because there is a line in your enzo parameter file that contains more than one "=" sign. I have just submitted a pull request that fixes it, so this should be fixed soon. In the mean time, you can simply comment out that line in your parameter file and everything should work fine.<br>
<br></div>Britton<br></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>On Fri, Sep 6, 2013 at 3:02 AM, Pengfei Chen <span dir="ltr"><<a href="mailto:madcpf@gmail.com" target="_blank">madcpf@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div><div dir="ltr">Hi all,<br><br>I downloaded the example script of light cone projectoin(<a href="http://yt-project.org/doc/cookbook/cosmological_analysis.html#light-cone-projection" target="_blank">http://yt-project.org/doc/cookbook/cosmological_analysis.html#light-cone-projection</a>) to my laptop and changed the Enzo simulation parameter file and redshifts. But when I ran it the following error occurred:<br>
<br> File "light_cone_projection.py", line 19, in <module><br> find_outputs=False)<br> File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/light_cone/light_cone.py", line 158, in __init__<br>
find_outputs=find_outputs)<br> File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/analysis_modules/cosmological_observation/cosmology_splice.py", line 44, in __init__<br> find_outputs=find_outputs)<br> File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/convenience.py", line 131, in simulation<br>
find_outputs=find_outputs)<br> File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 81, in __init__<br> find_outputs=find_outputs)<br> File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/data_objects/time_series.py", line 343, in __init__<br>
self._parse_parameter_file()<br> File "/Users/madcpf/Documents/dev_yt/src/yt-hg/yt/frontends/enzo/simulation_handling.py", line 273, in _parse_parameter_file<br> param, vals = (i.strip() for i in line.split("="))<br>
ValueError: need more than 1 value to unpack<br><br>Does the parameters "near_redshift" and "far_redshift" in LightCone need to be the same with "CosmologyFinalRedshift" and "CosmologyInitialRedshift" in enzo parameter file?<br>
Does anyone know how to solve this?<br><br>Thank you all!<span><font color="#888888"><br><br>Pengfei<br></font></span></div>
<br></div></div>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div><br></div>
<br>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div></div></div><br></div></div>
<br>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org" target="_blank">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div><br></div>
</div></div><br>_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
<br></blockquote></div><br></div>