[yt-users] Problem loading FLASH2.5 data

Wed Oct 19 09:05:25 PDT 2016

Dear Kacper,

sure you can add those files to the official testing suite!

Dear Alex,

I installed yt on a “clean” machine using pip and I still get the error that also Dominik confirmed.

Thank you and best regards,
Philipp

> On 19 Oct 2016, at 17:33, Kacper Kowalik <xarthisius.kk at gmail.com> wrote:
> 
> On 10/18/2016 09:11 AM, Philipp wrote:
>> Hi Nathan,
>> 
>> thank you for the fast reply. Sorry, I didn’t check the sample data
>> for F2.5 files. I uploaded a small test file here:
>> http://use.yt/upload/b895e885 <http://use.yt/upload/b895e885> which
>> is at time 0 (maybe not a very clever idea if the time is read
>> incorrectly) and another file (http://use.yt/upload/1106d6c4
>> <http://use.yt/upload/1106d6c4>) at time 23.8 kyr. The block size is
>> [8,8,8].
>> 
>> With best regards, 
>> Philipp
> 
> Hi Phillip,
> can I add those datasets to yt's testing suite? That mandates having
> them publicly available at http://yt-project.org/data/
> 
> I've also opened PR 2421 [1] that hopefully will fix the issue you're
> seeing.
> 
> Cheers,
> Kacper
> 
> [1] https://bitbucket.org/yt_analysis/yt/pull-requests/2421
> 
> 
>>> On 18 Oct 2016, at 15:37, Nathan Goldbaum <nathan12343 at gmail.com>
>>> wrote:
>>> 
>>> Hi Phillip,
>>> 
>>> I don't think we have any Flash2.5 test data on yt-project.org/data
>>> <http://yt-project.org/data>. We use the datasets there in our test
>>> suite, so if we don't have any such data that means that our
>>> support for a data format isn't tested and is subject to breakage.
>>> 
>>> Would you be willing to share some test data that the yt project
>>> can use for debugging purposes? It should be relatively small,
>>> should trigger the error you're seeing, and should be ok to host
>>> publicly on yt-project.org/data <http://yt-project.org/data> and
>>> use in our test suite. If so, we can look into restoring support
>>> and making sure we don't regress again.
>>> 
>>> You can share the data using e.g. Dropbox, google drive, or by
>>> using the yt curldrop:
>>> 
>>> https://docs.hub.yt/services.html#curldrop
>>> <https://docs.hub.yt/services.html#curldrop>
>>> 
>>> Nathan
>>> 
>>> On Tuesday, October 18, 2016, Philipp Girichidis <keule-phg at gmx.de
>>> <mailto:keule-phg at gmx.de>> wrote: Dear YT users,
>>> 
>>> I upgraded my version of YT from an older version that still
>>> installed the libraries on its own (version downloaded 2016/06/21)
>>> to a more recent version using yt-conda. With the new version I
>>> cannot read old FLASH2.5 data any more. The previous version of yt
>>> gave me the correct output
>>> 
>>> yt : [INFO     ] 2016-09-09 16:37:59,285 Parameters: current_time
>>> = 1260000000.0 yt : [INFO     ] 2016-09-09 16:37:59,286 Parameters:
>>> domain_dimensions         = [8 8 8] yt : [INFO     ] 2016-09-09
>>> 16:37:59,289 Parameters: domain_left_edge          = [
>>> -4.00000000e+17  -4.00000000e+17  -4.00000000e+17] yt : [INFO     ]
>>> 2016-09-09 16:37:59,290 Parameters: domain_right_edge         = [
>>> 4.00000000e+17   4.00000000e+17   4.00000000e+17] yt : [INFO     ]
>>> 2016-09-09 16:37:59,291 Parameters: cosmological_simulation   =
>>> 0.0
>>> 
>>> the new version does not read some of the numbers properly,
>>> 
>>> yt : [INFO     ] 2016-09-09 16:39:11,692 Parameters: current_time
>>> = 1.06099789548e-313 yt : [INFO     ] 2016-09-09 16:39:11,692
>>> Parameters: domain_dimensions         = [1104332420          0
>>> 1102957925] yt : [INFO     ] 2016-09-09 16:39:11,693 Parameters:
>>> domain_left_edge          = [ -4.00000000e+17  -4.00000000e+17
>>> -4.00000000e+17] yt : [INFO     ] 2016-09-09 16:39:11,694
>>> Parameters: domain_right_edge         = [  4.00000000e+17
>>> 4.00000000e+17   4.00000000e+17] yt : [INFO     ] 2016-09-09
>>> 16:39:11,695 Parameters: cosmological_simulation   = 0.0
>>> 
>>> The time value as well as the dimensions are messed up and thus yt
>>> crashes with a memory error at some point. To me it looks like a
>>> conversion problem between big and little endian or some similar
>>> conversion problem between single and double precision. I tried
>>> various combinations of MAC and linux and the problem does not seem
>>> to operating system dependent.
>>> 
>>> Quite a few functions have changed in terms of reading particle
>>> data, but the simulation does not contain particles. I tried to
>>> find the part of the code that reads the domain_dimensions and
>>> looked at file 
>>> yt-conda/lib/python2.7/site-packages/yt/frontends/flash/data_structures.py
>>> 
>>> 
> where in line 99ff the block dimensions are read. But this part of the
> code is not executed (checked with print statements in iyt). Loading the
> hdf5 file with h5py and using the code in line 104 gives me the correct
> numbers, but I guess yt reads the data from somewhere else?
>>> 
>>> Any hints on how the hierarchy of reading data is organized or
>>> where to search? I know FLASH2.5 is very old but we still have a
>>> lot of (quite expensive) F2.5 data.
>>> 
>>> With best regards, Philipp Girichidis 
>>> _______________________________________________ yt-users mailing
>>> list yt-users at lists.spacepope.org <javascript:;> 
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>> <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org> 
>>> _______________________________________________ yt-users mailing
>>> list yt-users at lists.spacepope.org 
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> 
>> 
>> 
>> 
>> _______________________________________________ yt-users mailing
>> list yt-users at lists.spacepope.org 
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> 
> 
> 
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org