[yt-users] Problem loading FLASH2.5 data

Nathan Goldbaum nathan12343 at gmail.com
Wed Oct 19 09:11:07 PDT 2016 

So it turns out this is due to a regression in the h5py library:

https://github.com/h5py/h5py/issues/764

If you'd like a quick workaround without Kacper's pull request, you just
need to forcibly downgrade h5py:

     $ conda uninstall h5py
     $ conda install h5py==2.5.0

or if you use pip:

     $ pip uninstall h5py
     $ pip install h5py==2.5.0

Hopefully Kacper's fix will be in by the time we release yt 3.3.2 and
you'll be able to upgrade h5py once again and hopefully the h5py bug will
be fixed by the time the next version fo h5py comes out.

-Nathan

On Wed, Oct 19, 2016 at 11:05 AM, Philipp <keule-phg at gmx.de> wrote:

> Dear Kacper,
>
> sure you can add those files to the official testing suite!
>
>
> Dear Alex,
>
> I installed yt on a “clean” machine using pip and I still get the error
> that also Dominik confirmed.
>
> Thank you and best regards,
> Philipp
>
>
> > On 19 Oct 2016, at 17:33, Kacper Kowalik <xarthisius.kk at gmail.com>
> wrote:
> >
> > On 10/18/2016 09:11 AM, Philipp wrote:
> >> Hi Nathan,
> >>
> >> thank you for the fast reply. Sorry, I didn’t check the sample data
> >> for F2.5 files. I uploaded a small test file here:
> >> http://use.yt/upload/b895e885 <http://use.yt/upload/b895e885> which
> >> is at time 0 (maybe not a very clever idea if the time is read
> >> incorrectly) and another file (http://use.yt/upload/1106d6c4
> >> <http://use.yt/upload/1106d6c4>) at time 23.8 kyr. The block size is
> >> [8,8,8].
> >>
> >> With best regards,
> >> Philipp
> >
> > Hi Phillip,
> > can I add those datasets to yt's testing suite? That mandates having
> > them publicly available at http://yt-project.org/data/
> >
> > I've also opened PR 2421 [1] that hopefully will fix the issue you're
> > seeing.
> >
> > Cheers,
> > Kacper
> >
> > [1] https://bitbucket.org/yt_analysis/yt/pull-requests/2421
> >
> >
> >>> On 18 Oct 2016, at 15:37, Nathan Goldbaum <nathan12343 at gmail.com>
> >>> wrote:
> >>>
> >>> Hi Phillip,
> >>>
> >>> I don't think we have any Flash2.5 test data on yt-project.org/data
> >>> <http://yt-project.org/data>. We use the datasets there in our test
> >>> suite, so if we don't have any such data that means that our
> >>> support for a data format isn't tested and is subject to breakage.
> >>>
> >>> Would you be willing to share some test data that the yt project
> >>> can use for debugging purposes? It should be relatively small,
> >>> should trigger the error you're seeing, and should be ok to host
> >>> publicly on yt-project.org/data <http://yt-project.org/data> and
> >>> use in our test suite. If so, we can look into restoring support
> >>> and making sure we don't regress again.
> >>>
> >>> You can share the data using e.g. Dropbox, google drive, or by
> >>> using the yt curldrop:
> >>>
> >>> https://docs.hub.yt/services.html#curldrop
> >>> <https://docs.hub.yt/services.html#curldrop>
> >>>
> >>> Nathan
> >>>
> >>> On Tuesday, October 18, 2016, Philipp Girichidis <keule-phg at gmx.de
> >>> <mailto:keule-phg at gmx.de>> wrote: Dear YT users,
> >>>
> >>> I upgraded my version of YT from an older version that still
> >>> installed the libraries on its own (version downloaded 2016/06/21)
> >>> to a more recent version using yt-conda. With the new version I
> >>> cannot read old FLASH2.5 data any more. The previous version of yt
> >>> gave me the correct output
> >>>
> >>> yt : [INFO     ] 2016-09-09 16:37:59,285 Parameters: current_time
> >>> = 1260000000.0 yt : [INFO     ] 2016-09-09 16:37:59,286 Parameters:
> >>> domain_dimensions         = [8 8 8] yt : [INFO     ] 2016-09-09
> >>> 16:37:59,289 Parameters: domain_left_edge          = [
> >>> -4.00000000e+17  -4.00000000e+17  -4.00000000e+17] yt : [INFO     ]
> >>> 2016-09-09 16:37:59,290 Parameters: domain_right_edge         = [
> >>> 4.00000000e+17   4.00000000e+17   4.00000000e+17] yt : [INFO     ]
> >>> 2016-09-09 16:37:59,291 Parameters: cosmological_simulation   =
> >>> 0.0
> >>>
> >>> the new version does not read some of the numbers properly,
> >>>
> >>> yt : [INFO     ] 2016-09-09 16:39:11,692 Parameters: current_time
> >>> = 1.06099789548e-313 yt : [INFO     ] 2016-09-09 16:39:11,692
> >>> Parameters: domain_dimensions         = [1104332420          0
> >>> 1102957925] yt : [INFO     ] 2016-09-09 16:39:11,693 Parameters:
> >>> domain_left_edge          = [ -4.00000000e+17  -4.00000000e+17
> >>> -4.00000000e+17] yt : [INFO     ] 2016-09-09 16:39:11,694
> >>> Parameters: domain_right_edge         = [  4.00000000e+17
> >>> 4.00000000e+17   4.00000000e+17] yt : [INFO     ] 2016-09-09
> >>> 16:39:11,695 Parameters: cosmological_simulation   = 0.0
> >>>
> >>> The time value as well as the dimensions are messed up and thus yt
> >>> crashes with a memory error at some point. To me it looks like a
> >>> conversion problem between big and little endian or some similar
> >>> conversion problem between single and double precision. I tried
> >>> various combinations of MAC and linux and the problem does not seem
> >>> to operating system dependent.
> >>>
> >>> Quite a few functions have changed in terms of reading particle
> >>> data, but the simulation does not contain particles. I tried to
> >>> find the part of the code that reads the domain_dimensions and
> >>> looked at file
> >>> yt-conda/lib/python2.7/site-packages/yt/frontends/flash/
> data_structures.py
> >>>
> >>>
> > where in line 99ff the block dimensions are read. But this part of the
> > code is not executed (checked with print statements in iyt). Loading the
> > hdf5 file with h5py and using the code in line 104 gives me the correct
> > numbers, but I guess yt reads the data from somewhere else?
> >>>
> >>> Any hints on how the hierarchy of reading data is organized or
> >>> where to search? I know FLASH2.5 is very old but we still have a
> >>> lot of (quite expensive) F2.5 data.
> >>>
> >>> With best regards, Philipp Girichidis
> >>> _______________________________________________ yt-users mailing
> >>> list yt-users at lists.spacepope.org <javascript:;>
> >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >>> <http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org>
> >>> _______________________________________________ yt-users mailing
> >>> list yt-users at lists.spacepope.org
> >>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >>
> >>
> >>
> >>
> >> _______________________________________________ yt-users mailing
> >> list yt-users at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >>
> >
> >
> > _______________________________________________
> > yt-users mailing list
> > yt-users at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20161019/9ae5953e/attachment.html>

More information about the yt-users mailing list