[yt-users] Problems with rockstar halo finding
Jullian Santos
jullianhbsantos at gmail.com
Mon Jun 13 11:35:52 PDT 2016
Hi,
I forgot to say that I had installed yt once before and I don't delete this
old version. If you look at the directories where I'm running yt you will
see that the yt is called from a hidden file. I run with the activate
script, but I'm not sure that he is calling the correct library, Is the
path correct too (for a All-in-One Installation)? Can I delete the .local
files and try to rebuild?
2016-06-13 15:19 GMT-03:00 Jullian Santos <jullianhbsantos at gmail.com>:
> Hi,
>
> I tried with python 2 and I received:
>
>
> File
> "/home/jullian/.local/lib/python2.7/site-packages/yt-3.2.3-py2.7-linux-x86_64.egg/yt/__init__.py",
> line 108, in <module>
> from yt.fields.api import \
> File
> "/home/jullian/.local/lib/python2.7/site-packages/yt-3.2.3-py2.7-linux-x86_64.egg/yt/fields/api.py",
> line 19, in <module>
> from . import angular_momentum
> File
> "/home/jullian/.local/lib/python2.7/site-packages/yt-3.2.3-py2.7-linux-x86_64.egg/yt/fields/angular_momentum.py",
> line 31, in <module>
> from .vector_operations import \
> File
> "/home/jullian/.local/lib/python2.7/site-packages/yt-3.2.3-py2.7-linux-x86_64.egg/yt/fields/vector_operations.py",
> line 31, in <module>
> from yt.utilities.lib.misc_utilities import obtain_rv_vec
> ImportError:
> /home/jullian/.local/lib/python2.7/site-packages/yt-3.2.3-py2.7-linux-x86_64.egg/yt/utilities/lib/misc_utilities.so:
> undefined symbol: PyUnicodeUCS4_DecodeUTF8
> Traceback (most recent call last):
> File "/home/jullian/ABinitio/Out/DD0050/halo_rockstar.py", line 12, in
> <module>
> import yt
> File
> "/home/jullian/.local/lib/python2.7/site-packages/yt-3.2.3-py2.7-linux-x86_64.egg/yt/__init__.py",
> line 108, in <module>
> from yt.fields.api import \
> File
> "/home/jullian/.local/lib/python2.7/site-packages/yt-3.2.3-py2.7-linux-x86_64.egg/yt/fields/api.py",
> line 19, in <module>
> from . import angular_momentum
> File
> "/home/jullian/.local/lib/python2.7/site-packages/yt-3.2.3-py2.7-linux-x86_64.egg/yt/fields/angular_momentum.py",
> line 31, in <module>
> from .vector_operations import \
> File
> "/home/jullian/.local/lib/python2.7/site-packages/yt-3.2.3-py2.7-linux-x86_64.egg/yt/fields/vector_operations.py",
> line 31, in <module>
> from yt.utilities.lib.misc_utilities import obtain_rv_vec
> ImportError:
> /home/jullian/.local/lib/python2.7/site-packages/yt-3.2.3-py2.7-linux-x86_64.egg/yt/utilities/lib/misc_utilities.so:
> undefined symbol: PyUnicodeUCS4_DecodeUTF8
>
>
> I installed yt-x86-64 by the All-in-One Installation Script and I don't
> know how to change PyUnicode and rebuild. Do you have some guess to me?
>
> Thanks
>
>
> 2016-06-09 12:02 GMT-03:00 Matthew Turk <matthewturk at gmail.com>:
>
>> Hi Jullian,
>>
>> My guess from reading is that there's a python3 incompatibility
>> specifically in the Rockstar code interface that yt provides. I
>> believe that it should be fixable, but right now I don't have the
>> setup to check it. If you file a bug we will mark it as a blocker and
>> then return to it before the impending 3.3 release of yt; I'm sorry to
>> not have better news right now. It should work on Python2, but for
>> Python3 there is some location where the unicode/non-unicode decoding
>> needs to happen to move between python strings and char*.
>>
>> -Matt
>>
>> On Tue, Jun 7, 2016 at 11:35 AM, Jullian Santos
>> <jullianhbsantos at gmail.com> wrote:
>> > Hello,
>> > My name is Jullian Santos, I am a Ph.D. student of the University of São
>> > Paulo, Brazil, and I am having some troubles with the Rockstar halo
>> finding
>> > function.
>> >
>> >
>> > I tried to run a test analyses given by yt-project page (the link is
>> below,
>> > I only modified the file to be opened) but the job always finish in
>> error. I
>> > submitted the job in my own machine with the command:
>> >
>> >
>> > #PBS -S /bin/bash
>> > #PBS -l nodes=4
>> > #PBS -e /sto/home/jullian/M2.e.txt
>> > #PBS -o /sto/home/jullian/M2.o.txt
>> > #PBS -V
>> >
>> >
>> > mpirun -n 4 $PBS_NODEFILE python3
>> > "/home/jullian/ABinitio/Out/DD0050/halo_rockstar.py"
>> >
>> > and the program returned:
>> >
>> > writers
>> > P001 yt : [WARNING ] 2016-06-07 14:46:19,708 Total Particle Count:
>> > 1.125e+06
>> > File "/home/jullian/ABinitio/Out/DD0050/halo_rockstar.py", line 62, in
>> > <module>
>> > hc.create()
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
>> > line 293, in create
>> > self._run(save_halos, save_catalog, njobs=njobs, dynamic=dynamic)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/utilities/parallel_tools/parallel_analysis_interface.py",
>> > line 305, in barrierize
>> > retval = func(*args, **kwargs)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
>> > line 362, in _run
>> > self.halos_ds = self.finder_method(self.data_ds)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
>> > line 46, in __call__
>> > return self.function(ds, *self.args, **self.kwargs)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
>> > line 79, in _rockstar_method
>> > rh.run()
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_finding/rockstar/rockstar.py",
>> > line 348, in run
>> > min_halo_size = self.min_halo_size)
>> > File
>> "yt/analysis_modules/halo_finding/rockstar/rockstar_interface.pyx",
>> > line 254, in
>> >
>> yt.analysis_modules.halo_finding.rockstar.rockstar_interface.RockstarInterface.setup_rockstar
>> > (yt/analysis_modules/halo_finding/rockstar/rockstar_interface.c:3785)
>> > P002 yt : [ERROR ] 2016-06-07 14:46:19,758 TypeError: expected
>> bytes, str
>> > found
>> > File "/home/jullian/ABinitio/Out/DD0050/halo_rockstar.py", line 62, in
>> > <module>
>> > hc.create()
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
>> > line 293, in create
>> > self._run(save_halos, save_catalog, njobs=njobs, dynamic=dynamic)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/utilities/parallel_tools/parallel_analysis_interface.py",
>> > line 305, in barrierize
>> > retval = func(*args, **kwargs)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
>> > line 362, in _run
>> > self.halos_ds = self.finder_method(self.data_ds)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
>> > line 46, in __call__
>> > return self.function(ds, *self.args, **self.kwargs)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
>> > line 79, in _rockstar_method
>> > rh.run()
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_finding/rockstar/rockstar.py",
>> > line 348, in run
>> > min_halo_size = self.min_halo_size)
>> > File
>> "yt/analysis_modules/halo_finding/rockstar/rockstar_interface.pyx",
>> > line 254, in
>> >
>> yt.analysis_modules.halo_finding.rockstar.rockstar_interface.RockstarInterface.setup_rockstar
>> > (yt/analysis_modules/halo_finding/rockstar/rockstar_interface.c:3785)
>> > P003 yt : [ERROR ] 2016-06-07 14:46:19,758 TypeError: expected
>> bytes, str
>> > found
>> > P002 yt : [ERROR ] 2016-06-07 14:46:19,759 Error occured on rank 0.
>> > application called MPI_Abort(MPI_COMM_WORLD, 1) - process 2
>> > P003 yt : [ERROR ] 2016-06-07 14:46:19,759 Error occured on rank 1.
>> > application called MPI_Abort(MPI_COMM_WORLD, 1) - process 3
>> > File "/home/jullian/ABinitio/Out/DD0050/halo_rockstar.py", line 62, in
>> > <module>
>> > hc.create()
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
>> > line 293, in create
>> > self._run(save_halos, save_catalog, njobs=njobs, dynamic=dynamic)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/utilities/parallel_tools/parallel_analysis_interface.py",
>> > line 305, in barrierize
>> > retval = func(*args, **kwargs)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
>> > line 362, in _run
>> > self.halos_ds = self.finder_method(self.data_ds)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
>> > line 46, in __call__
>> > return self.function(ds, *self.args, **self.kwargs)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
>> > line 79, in _rockstar_method
>> > rh.run()
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_finding/rockstar/rockstar.py",
>> > line 348, in run
>> > min_halo_size = self.min_halo_size)
>> > File
>> "yt/analysis_modules/halo_finding/rockstar/rockstar_interface.pyx",
>> > line 254, in
>> >
>> yt.analysis_modules.halo_finding.rockstar.rockstar_interface.RockstarInterface.setup_rockstar
>> > (yt/analysis_modules/halo_finding/rockstar/rockstar_interface.c:3785)
>> > P000 yt : [ERROR ] 2016-06-07 14:46:19,761 TypeError: expected
>> bytes, str
>> > found
>> > application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
>> > File "/home/jullian/ABinitio/Out/DD0050/halo_rockstar.py", line 62, in
>> > <module>
>> > hc.create()
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
>> > line 293, in create
>> > self._run(save_halos, save_catalog, njobs=njobs, dynamic=dynamic)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/utilities/parallel_tools/parallel_analysis_interface.py",
>> > line 305, in barrierize
>> > retval = func(*args, **kwargs)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
>> > line 362, in _run
>> > self.halos_ds = self.finder_method(self.data_ds)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
>> > line 46, in __call__
>> > return self.function(ds, *self.args, **self.kwargs)
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
>> > line 79, in _rockstar_method
>> > rh.run()
>> > File
>> >
>> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_finding/rockstar/rockstar.py",
>> > line 348, in run
>> > min_halo_size = self.min_halo_size)
>> > File
>> "yt/analysis_modules/halo_finding/rockstar/rockstar_interface.pyx",
>> > line 254, in
>> >
>> yt.analysis_modules.halo_finding.rockstar.rockstar_interface.RockstarInterface.setup_rockstar
>> > (yt/analysis_modules/halo_finding/rockstar/rockstar_interface.c:3785)
>> > P001 yt : [ERROR ] 2016-06-07 14:46:19,761 TypeError: expected
>> bytes, str
>> > found
>> > application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1
>> >
>> >
>> ===================================================================================
>> > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
>> > = EXIT CODE: 1
>> > = CLEANING UP REMAINING PROCESSES
>> > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
>> >
>> ===================================================================================
>> >
>> >
>> ===================================================================================
>> > = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
>> > = EXIT CODE: 1
>> > = CLEANING UP REMAINING PROCESSES
>> > = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
>> >
>> ===================================================================================
>> >
>> >
>> >
>> > If I try to run with the command suggested in the yt page (mpirun -n 4
>> > $PBS_NODEFILE --mca btl ^openib python3
>> > "/home/jullian/ABinitio/Out/DD0050/halo_rockstar.py" --parallel) I get:
>> >
>> > [mpiexec at julian-ubuntu] match_arg (./utils/args/args.c:160):
>> unrecognized
>> > argument mca
>> > [mpiexec at julian-ubuntu] HYDU_parse_array (./utils/args/args.c:175):
>> argument
>> > matching returned error
>> > [mpiexec at julian-ubuntu] parse_args (./ui/mpich/utils.c:1609): error
>> parsing
>> > input array
>> > [mpiexec at julian-ubuntu] HYD_uii_mpx_get_parameters
>> > (./ui/mpich/utils.c:1660): unable to parse user arguments
>> > [mpiexec at julian-ubuntu] main (./ui/mpich/mpiexec.c:153): error parsing
>> > parameters
>> >
>> > How could I fix this problem? Do I have to launch the script in some
>> remote
>> > machine? Do I have to use the IPython?
>> >
>> >
>> > Thanks by the attention,
>> >
>> > Best Regards,
>> >
>> > Jullian
>> >
>> >
>> > Script link:
>> >
>> http://yt-project.org/doc/cookbook/cosmological_analysis.html#running-rockstar-to-find-halos-on-multi-resolution-particle-datasets
>> >
>> > _______________________________________________
>> > yt-users mailing list
>> > yt-users at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>> >
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20160613/82a0bef6/attachment.html>
More information about the yt-users
mailing list