<div dir="ltr">Are you sure that the files are the same on your laptop and on Pleiades? Are you sure that you're running the same version of yt on your laptop and pleiades?<div><br></div><div>A way to check the former would be to make a tarball containing the dataset and then find the tarball's SHA hash on your laptop and on Pleiades, e.g.:</div><div><br></div><div># on linux</div><div>$ sha256sum my_data.tar</div><div><br></div><div># on MacOS</div><div>$ shasum -a 256 my_data.tar</div><div><br></div><div>If those commands don't print the same answer on your laptop and on Pleiades, that indicates that the files are not identical and perhaps there has been some corruption. That might be the source of the issue.</div><div><br></div><div>You can check to make sure the yt versions are the same on your laptop and on Pleiades by running "yt version" on the bash command line.<br><div><br></div><div>What's happening in the error you're running that yt is trying to parse the octree hierarchy in your data by reading some bytes from the data file. It's dying because it's getting back a piece of data that's not the shape it's expecting based on what we know about the ART data format. It's very odd that this works on your laptop using the exact same data file, because yt is making the same assumptions about the structure of the data file on Pleiades and your laptop.</div><div><br></div><div>Finally, if none of the diagnostic suggestions I made above help, it would help to make further progress on this issue if you can share the data file that's triggering the problem so that one of us can reproduce the issue and debug locally, although I totally understand if there are possible concerns about sharing data.</div><div><br></div><div>-Nathan</div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jul 25, 2017 at 3:05 PM, Clayton Strawn <span dir="ltr"><<a href="mailto:cjstrawn@ucsc.edu" target="_blank">cjstrawn@ucsc.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"></blockquote>Hello,<br><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"></blockquote><br>I have a request for help. <div><br><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"></blockquote>I am trying to run some analysis code remotely through ssh in yt on NASA Pleiades. I have been able to install yt and trident and am running in a virtualenv, but something is having a very deeply buried error that I don't understand and fatally kills any task I send to Pleiades.<div><br></div><div>The same code works fine on my home laptop with the same input files, so I assume the problem is something to do with the yt distribution on Pleiades, but I used "pip install -U yt" so it should be up-to-date. The error code comes from the fortran_utils file, and it says "<span style="font-family:monospace,monospace">unpack requires a string argument of length 4"</span><br><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><font face="monospace, monospace"><br></font><font face="monospace, monospace">(myenv) cstrawn@pfe27:~/quasarlines/<wbr>quasarlines> python quasar_scan.py VELA_v2_08 10MpcBox_csf512_a0.330.d "[H I,H II,O IV,O V,O VI,Si II,Si III,Si IV,C I,C II,C III,C IV,Mg II,Mg III,Mg IV]"<br></font><font face="monospace, monospace">yt : [INFO ] 2017-07-25 12:11:26,454 discovered particle_data:/home1/cstrawn/<wbr>quasarlines/quasarlines/<wbr>PMcrs0a0.330.DAT<br></font><font face="monospace, monospace">yt : [INFO ] 2017-07-25 12:11:26,454 discovered particle_header:/home1/<wbr>cstrawn/quasarlines/<wbr>quasarlines/PMcrda0.330.DAT<br></font><font face="monospace, monospace">yt : [INFO ] 2017-07-25 12:11:26,462 Using root level of 14<br></font><font face="monospace, monospace">yt : [INFO ] 2017-07-25 12:11:26,463 Discovered 7 species of particles<br></font><font face="monospace, monospace">yt : [INFO ] 2017-07-25 12:11:26,463 Particle populations: 28016640 4263936 752640 123904 19552 2089406 85309<br></font><font face="monospace, monospace">yt : [INFO ] 2017-07-25 12:11:26,658 Max level is 12<br></font><font face="monospace, monospace">yt : [INFO ] 2017-07-25 12:11:26,682 Parameters: current_time = 3.350407555 Gyr<br></font><font face="monospace, monospace">yt : [INFO ] 2017-07-25 12:11:26,682 Parameters: domain_dimensions = [128 128 128]<br></font><font face="monospace, monospace">yt : [INFO ] 2017-07-25 12:11:26,683 Parameters: domain_left_edge = [ 0. 0. 0.]<br></font><font face="monospace, monospace">yt : [INFO ] 2017-07-25 12:11:26,683 Parameters: domain_right_edge = [ 1. 1. 1.]<br></font><font face="monospace, monospace">yt : [INFO ] 2017-07-25 12:11:26,683 Parameters: cosmological_simulation = True<br></font><font face="monospace, monospace">yt : [INFO ] 2017-07-25 12:11:26,683 Parameters: current_redshift = 2.02694081565<br></font><font face="monospace, monospace">yt : [INFO ] 2017-07-25 12:11:26,683 Parameters: omega_lambda = 0.730000019073<br></font><font face="monospace, monospace">yt : [INFO ] 2017-07-25 12:11:26,684 Parameters: omega_matter = 0.270000010729<br></font><font face="monospace, monospace">yt : [INFO ] 2017-07-25 12:11:26,684 Parameters: hubble_constant = 0.699999988079<br></font><font face="monospace, monospace">18426 LOSs to scan.<br></font><font face="monospace, monospace">Is this an appropriate number? (0 for no)</font><font face="monospace, monospace"><br></font><font face="monospace, monospace">1<br></font><font face="monospace, monospace">1/18426<br></font><font face="monospace, monospace">Traceback (most recent call last):<br></font><font face="monospace, monospace"> File "quasar_scan.py", line 375, in <module><br></font><font face="monospace, monospace"> q.get_coldens(save = 10)<br></font><font face="monospace, monospace"> File "quasar_scan.py", line 148, in get_coldens<br></font><font face="monospace, monospace"> ftype='gas')<br></font><font face="monospace, monospace"> File "/u/cstrawn/myenv/lib/python2.<wbr>7/site-packages/trident/ray_<wbr>generator.py", line 239, in make_simple_ray<br></font><font face="monospace, monospace"> particle_type = _determine_particle_type_from_<wbr>ftype(ds, ftype)<br></font><font face="monospace, monospace"> File "/u/cstrawn/myenv/lib/python2.<wbr>7/site-packages/trident/ray_<wbr>generator.py", line 590, in _determine_particle_type_from_<wbr>ftype<br></font><font face="monospace, monospace"> field_list_arr = np.asarray(ds.derived_field_<wbr>list)<br></font><font face="monospace, monospace"> File "/u/cstrawn/myenv/lib/python2.<wbr>7/site-packages/yt/data_<wbr>objects/static_output.py", line 164, in ireq<br></font><font face="monospace, monospace"> self.index<br></font><font face="monospace, monospace"> File "/u/cstrawn/myenv/lib/python2.<wbr>7/site-packages/yt/data_<wbr>objects/static_output.py", line 419, in index<br></font><font face="monospace, monospace"> self, dataset_type=self.dataset_<wbr>type)<br></font><font face="monospace, monospace"> File "/u/cstrawn/myenv/lib/python2.<wbr>7/site-packages/yt/frontends/<wbr>art/data_structures.py", line 65, in __init__<br></font><font face="monospace, monospace"> super(ARTIndex, self).__init__(ds, dataset_type)<br></font><font face="monospace, monospace"> File "/u/cstrawn/myenv/lib/python2.<wbr>7/site-packages/yt/geometry/<wbr>geometry_handler.py", line 50, in __init__<br></font><font face="monospace, monospace"> self._setup_geometry()<br></font><font face="monospace, monospace"> File "/u/cstrawn/myenv/lib/python2.<wbr>7/site-packages/yt/geometry/<wbr>oct_geometry_handler.py", line 25, in _setup_geometry<br></font><font face="monospace, monospace"> self._initialize_oct_handler()<br></font><font face="monospace, monospace"> File "/u/cstrawn/myenv/lib/python2.<wbr>7/site-packages/yt/frontends/<wbr>art/data_structures.py", line 91, in _initialize_oct_handler<br></font><font face="monospace, monospace"> self.domains]<br></font><font face="monospace, monospace"> File "/u/cstrawn/myenv/lib/python2.<wbr>7/site-packages/yt/frontends/<wbr>art/data_structures.py", line 761, in level_count<br></font><font face="monospace, monospace"> self.level_offsets<br></font><font face="monospace, monospace"> File "/u/cstrawn/myenv/lib/python2.<wbr>7/site-packages/yt/frontends/<wbr>art/data_structures.py", line 782, in level_offsets<br></font><font face="monospace, monospace"> self.ds.min_level, self.ds.max_level)<br></font><font face="monospace, monospace"> File "/u/cstrawn/myenv/lib/python2.<wbr>7/site-packages/yt/frontends/<wbr>art/data_structures.py", line 820, in _count_art_octs<br></font><font face="monospace, monospace"> ns = fpu.peek_record_size(f, endian='>')<br></font><font face="monospace, monospace"> File "/u/cstrawn/myenv/lib/python2.<wbr>7/site-packages/yt/utilities/<wbr>fortran_utils.py", line 250, in peek_record_size<br></font><font face="monospace, monospace"> s = struct.unpack('>i', f.read(struct.calcsize('>i')))<br></font><span style="font-family:monospace,monospace">struct.error: unpack requires a string argument of length 4<br><br><br></span></blockquote><div>Please let me know if this is a known problem or if you have any idea what I should do to avoid running into it. I don't really understand fortran enough to know what is even supposed to be happening in fortran_utils.</div><div><br></div>Thanks so much,</div><div>Clayton Strawn<br><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"></blockquote></blockquote><span style="font-family:monospace,monospace"><br></span></div></div></div>
<br>______________________________<wbr>_________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" rel="noreferrer" target="_blank">http://lists.spacepope.org/<wbr>listinfo.cgi/yt-users-<wbr>spacepope.org</a><br>
<br></blockquote></div><br></div>