<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Apr 27, 2016 at 9:12 PM, Hsi-Yu Schive <span dir="ltr"><<a href="mailto:hyschive@gmail.com" target="_blank">hyschive@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div dir="ltr">Hi all,<br><br>I just finish the GAMER frontend for YT, and below I want to summarize some issues and things that might be confusing in the current skeleton frontend. I appreciate if you could help validate these issues and/or clarify some points in the section "Things to be clarified". Hopefully it will make frontend implementation easier in the future. I'll collect all comments and issue a PR for the revised skeleton. It will be complementary to the recent PR #2130 by Jonah Miller, which provides an extremely helpful example frontend.<br></div></div></blockquote><div><br></div><div>Thanks so much for doing this. Future frontend authors are indebted to you. I've replied inline where you had questions. </div><div><br></div><div>I think all of the places you're noticing inconsistencies or where the skeleton frontend does things that aren't necessary anymore, you're probably right, and you should go ahead and make the necessary changes to update it. Matt and others might also want to chime in if I get anything wrong...</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><br><br><b><font color="#0000ff">Things to be clarified:</font><br></b><font color="#0000ff">============================</font><br>1. Difference between <b>FieldInfoContainer.known_other_fields</b> and <b>GridIndex._detect_output_field</b>:<br> => known_other_fields contains fields that *might* be in an output, and lists their units and aliases. _detect_output_fields finds the fields that are defined in the currently loaded dataset.<br></div></div></blockquote><div><br></div><div>This is correct. known_particle_fields is the same, but for particle fields.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><br>2. <b>Dataset.unique_identifier</b>:<div> => ??<br><br></div></div></div></blockquote><div><br></div><div>This is supposed to an identifier that uniquely identifies the dataset being read in. Some frontends use a string, others use an integer. Some frontends will check if there is some metadata that can be read in from the output file to use for this, but fall back to the creation time of the file. This code from the Enzo frontend might be illustrative:</div><div><br></div><div><div> if "MetaDataDatasetUUID" in self.parameters:</div><div> self.unique_identifier = self.parameters["MetaDataDatasetUUID"]</div><div> elif "CurrentTimeIdentifier" in self.parameters:</div><div> self.unique_identifier = self.parameters["CurrentTimeIdentifier"]</div><div> else:</div><div> self.unique_identifier = \</div><div> int(os.stat(self.parameter_filename)[stat.ST_CTIME])</div></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div>3. Set <b>AMRGridPatch.Parent = None</b> for grids without parents, and <b>AMRGridPatch.Children = []</b> for grids without children</div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div><br>4. Field array returned by <b>BaseIOHandler._read_fluid_selection</b> should be contiguous along the z instead of x direction. Therefore, for C-like array with the dimension [x][y][z] and for Fortran-like array with the the dimension (z,y,x), a matrix transpose is required (e.g., using np_array.transpose() or np_array.swapaxes(0,2))<br><br>5. <b>start_index, stop_index, </b>and <b>ActiveDimension</b>s in the AMRGridPatch objects</div><div> => ??<br> (1) It looks like stop_index is not used anymore<br></div></div></div></blockquote><div><br></div><div>Just grepping from stop_index, I think I agree. We should probably remove it from the frontends where it's defined.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div> (2) ActiveDimensions will be set by AMRGridPatch._prepare_grid. So perhaps frontend does not need to set it explicitly anymore </div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div> (3) It seems that start_index is also calculated automatically </div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div>6. <b>chunk, selector, </b>and <b>_read_chunk_data</b> in io.py.</div><div> => ??</div><div> Also the following comment about caching is confusing:<br> def _read_chunk_data(self, chunk, fields):<br> # This reads the data from a single chunk, and is only used for caching.<br></div></div></div></blockquote><div><br></div><div>Not sure what's up with this.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div><br>7. float type:<br> (1) <b>GridIndex.float_type</b> is the float type for left and right simulation edges and must be float64 now<br> (2) <b>BaseIOHandler._read_fluid_selection</b> should always return float64 now even if the on-disk data are stored in single precision<br><br>8. Difference between <b>add_output_field </b>and <b>add_field </b>used in FieldInfoContainer.setup_fluid_fields:<br><br>9. Dataset._parse_parameter_file: what does the following comment mean: "Note that these are all assumed to be in code units; domain_left_edge and domain_right_edge will be updated to be in code units at a later time"?<br> => Perhaps it means that "domain_left_edge and domain_right_edge will be converted to YTArray automatically at a later time"?<br></div></div></div></blockquote><div><br></div><div>Yes, that is a better way of phrasing what happens. <br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div><br><b><font color="#0000ff">Missing procedure</font><br></b><span style="color:rgb(0,0,255)">============================</span><br>1. Add the new frontend in <b>frontends/api.py</b><br>2. Set <b>GridIndex.max_level</b><br>3. Set <b>Dataset.refine_by</b><br><br><br><font color="#0000ff"><b>Bugin the current skeleton frontend</b></font></div><div><span style="color:rgb(0,0,255)">============================</span><br>1. data_structures.py: <font color="#000000"><b>self.dataset</b></font> is not set in GridIndex.__init__ when calling self.index_filename = self.dataset.parameter_filename<br>2. data_structures.py: replace self.Parent=[] by <b>self.Parent = None</b><br>3. fields.py: the <b>field_list </b>argument is missing in FieldInfoContainer.__init__. This issue has been fixed in Jonah's PR.<br> def __init__(self, ds, field_list):<br> super(SkeletonFieldInfo, self).__init__(ds, field_list)</div><div><br></div></div></div></blockquote><div><br></div><div>Looking forward to a pull request with fixes!</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div></div><div><br></div><div>Sincerely,</div><div>Hsi-Yu (Justin)</div></div></div>
<br>_______________________________________________<br>
yt-dev mailing list<br>
<a href="mailto:yt-dev@lists.spacepope.org">yt-dev@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org" rel="noreferrer" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org</a><br>
<br></blockquote></div><br></div></div>