[yt-users] Reading Array Data

Thu Jan 11 12:11:45 PST 2018

We support array particle fields already, we just needed to teach the Enzo
frontend that some active particle fields can be array fields and how to
detect that.

On Thu, Jan 11, 2018 at 2:10 PM, John Regan <johnanthonyregan at gmail.com>
wrote:

> Top man! I'll pull that down in the morning. What was the issue?
>
> On Thu, Jan 11, 2018 at 7:57 PM, Nathan Goldbaum <nathan12343 at gmail.com>
> wrote:
>
>> I've opened a pull request with a fix:
>>
>> https://github.com/yt-project/yt/pull/1665
>>
>> On Thu, Jan 11, 2018 at 1:29 PM, John Regan <johnanthonyregan at gmail.com>
>> wrote:
>>
>>> No worries at all. Really appreciate you looking at this.
>>>
>>> On Thu, Jan 11, 2018 at 7:26 PM, Nathan Goldbaum <nathan12343 at gmail.com>
>>> wrote:
>>>
>>>> Ah, apologies, missed that in your first e-mail.
>>>>
>>>> On Thu, Jan 11, 2018 at 1:24 PM, John Regan <johnanthonyregan at gmail.com
>>>> > wrote:
>>>>
>>>>> Hi Nathan,
>>>>>
>>>>> The 12 GB folder should be available at the link I sent in the first
>>>>> email. Here it is: https://www.dropbox.com/sh/o6a
>>>>> cuvia1sqyp7y/AAC0lEI5lZ6bGa0tP2-G1jLea?dl=0
>>>>>
>>>>> If you run the script I sent it will fail pretty quickly with a
>>>>> traceback. Let me know if that works.
>>>>>
>>>>>
>>>>> On Thu, Jan 11, 2018 at 6:50 PM, Nathan Goldbaum <
>>>>> nathan12343 at gmail.com> wrote:
>>>>>
>>>>>> It would probably be best to share the full output. I might just need
>>>>>> to see the traceback too, it's hard to say what will help without actually
>>>>>> being able to see it :)
>>>>>>
>>>>>> Is there any chance you can make a smaller output with similar active
>>>>>> particle fields? I can also download the 12 GB if you make it publicly
>>>>>> visible online somewhere.
>>>>>>
>>>>>>
>>>>>> On Thu, Jan 11, 2018 at 12:42 PM, John Regan <
>>>>>> johnanthonyregan at gmail.com> wrote:
>>>>>>
>>>>>>> Hi Nathan ,
>>>>>>>
>>>>>>> Thanks for the email. If I sent you the exact hdf5 file (i.e. the
>>>>>>> .cpuXXXX file) would that would and the traceback? You could see the format
>>>>>>> the array gets written then....
>>>>>>>
>>>>>>> It wasn't clear to me from reading through the YT reader would work.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> John
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jan 11, 2018 at 3:43 PM, Nathan Goldbaum <
>>>>>>> nathan12343 at gmail.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jan 11, 2018 at 3:43 AM, John Regan <
>>>>>>>> johnanthonyregan at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> I'm trying to get YT to read a vector which is stored as part of
>>>>>>>>> particle structure in Enzo (a vector in the active particle structure). The
>>>>>>>>> data looks OK when I view the raw HDF5 data using hdfview. I find that in
>>>>>>>>> some snapshots YT can read the data but not in others. If there is more
>>>>>>>>> than one particle things get worse for sure.
>>>>>>>>>
>>>>>>>>> It could be that when writing to the HDF5 file I haven't put the
>>>>>>>>> data in a format that is compatible with YT.
>>>>>>>>>
>>>>>>>>> I've attached a simple script that will fall over when trying to
>>>>>>>>> read the data and a link to the enzo output (size = 12 GBs). The active
>>>>>>>>> particles are in files DD0120.cpu0022 (Grid 3072), DD0120.cpu0023 (Grid
>>>>>>>>> 3076) and DD0014 (Grid 3080). YT falls over in the io_handler but it's not
>>>>>>>>> obvious to me whether I need to modify that code or the format that I write
>>>>>>>>> the data to the HDF5 file in.
>>>>>>>>>
>>>>>>>>
>>>>>>>> It's hard to say if there's an issue on the Enzo side or the yt
>>>>>>>> side without a copy of the traceback you are seeing or a way to reproduce
>>>>>>>> the issue. I'm also not sure what your data format looks like offhand. If I
>>>>>>>> remember correctly, you added support for vector active particle attributes
>>>>>>>> to your fork of Enzo and it's entirely possible that there needs to be
>>>>>>>> updates on the yt side to support reading the data from your fork.
>>>>>>>>
>>>>>>>> I suspect the place you need to modify in yt is here:
>>>>>>>>
>>>>>>>> https://github.com/yt-project/yt/blob/eb3358c622bc047200b9bc
>>>>>>>> 6a954ddee8f59a0e07/yt/frontends/enzo/io.py#L92-L113
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Any help/pointers would be much appreciated.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> John
>>>>>>>>>
>>>>>>>>> Link to dataset:
>>>>>>>>> https://www.dropbox.com/sh/o6acuvia1sqyp7y/AAC0lEI5lZ6bGa0tP
>>>>>>>>> 2-G1jLea?dl=0
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> yt-users mailing list
>>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> yt-users mailing list
>>>>>>>> yt-users at lists.spacepope.org
>>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> yt-users mailing list
>>>>>>> yt-users at lists.spacepope.org
>>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> yt-users mailing list
>>>>>> yt-users at lists.spacepope.org
>>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> yt-users mailing list
>>>>> yt-users at lists.spacepope.org
>>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>>>
>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>>
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20180111/42fdb5da/attachment-0001.html>