[yt-dev] Tipsy _is_valid will deadlock loading FLASH datasets

[Nathan Goldbaum]

Hi Ben,

There are some sizable test FLASH datasets here: http://yt-project.org/data/

Can you see if you can reproduce it?

I think this error will happen for any datasets where yt loads the actual
output file rather than a short parameter file.  For FLASH there is only
one monolithic hdf5 file.

I don't think any FLASH-specific fixes will be sufficient.

-Nathan


On Thu, Apr 10, 2014 at 4:32 PM, B.W. Keller <kellerbw at mcmaster.ca> wrote:

> Oh my.  Sorry that I have introduced this, unfortunately there is no way
> to detect tipsy files other than actually reading the entire file from
> disk.  Perhaps the way to fix this would be to drop the priority of tipsy
> datasets to the bottom, so that a valid FLASH dataset will be detected
> prior to the Tipsy check?
>
>
> On Thu, Apr 10, 2014 at 7:15 PM, Nathan Goldbaum <nathan12343 at gmail.com>wrote:
>
>> Hi all,
>>
>> We just had a bug report from Aaron Smith at UT Austin.  The symptom is
>> that the "load" comman was taking 30 seconds to complete on his FLASH
>> dataset, which should never happen for FLASH.
>>
>> After asking him to profile the code, he produced the following profile:
>>
>> http://ngoldbaum.net/yt-load/
>>
>> It seems that the recent changes to the Tipsy frontend which allow it to
>> autodetect binary outputs have made it so in some cases non-tipsy data is
>> loaded off disk.
>>
>> I'm not sure about the best way to handle this, which is why I'm writing
>> to the list rather than issuing a PR.
>>
>> -Nathan
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20140410/286e7ad9/attachment.html>