[yt-users] Problems with rockstar halo finding
Matthew Turk
matthewturk at gmail.com
Thu Jun 9 08:02:34 PDT 2016
Hi Jullian,
My guess from reading is that there's a python3 incompatibility
specifically in the Rockstar code interface that yt provides. I
believe that it should be fixable, but right now I don't have the
setup to check it. If you file a bug we will mark it as a blocker and
then return to it before the impending 3.3 release of yt; I'm sorry to
not have better news right now. It should work on Python2, but for
Python3 there is some location where the unicode/non-unicode decoding
needs to happen to move between python strings and char*.
-Matt
On Tue, Jun 7, 2016 at 11:35 AM, Jullian Santos
<jullianhbsantos at gmail.com> wrote:
> Hello,
> My name is Jullian Santos, I am a Ph.D. student of the University of São
> Paulo, Brazil, and I am having some troubles with the Rockstar halo finding
> function.
>
>
> I tried to run a test analyses given by yt-project page (the link is below,
> I only modified the file to be opened) but the job always finish in error. I
> submitted the job in my own machine with the command:
>
>
> #PBS -S /bin/bash
> #PBS -l nodes=4
> #PBS -e /sto/home/jullian/M2.e.txt
> #PBS -o /sto/home/jullian/M2.o.txt
> #PBS -V
>
>
> mpirun -n 4 $PBS_NODEFILE python3
> "/home/jullian/ABinitio/Out/DD0050/halo_rockstar.py"
>
> and the program returned:
>
> writers
> P001 yt : [WARNING ] 2016-06-07 14:46:19,708 Total Particle Count:
> 1.125e+06
> File "/home/jullian/ABinitio/Out/DD0050/halo_rockstar.py", line 62, in
> <module>
> hc.create()
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
> line 293, in create
> self._run(save_halos, save_catalog, njobs=njobs, dynamic=dynamic)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/utilities/parallel_tools/parallel_analysis_interface.py",
> line 305, in barrierize
> retval = func(*args, **kwargs)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
> line 362, in _run
> self.halos_ds = self.finder_method(self.data_ds)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
> line 46, in __call__
> return self.function(ds, *self.args, **self.kwargs)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
> line 79, in _rockstar_method
> rh.run()
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_finding/rockstar/rockstar.py",
> line 348, in run
> min_halo_size = self.min_halo_size)
> File "yt/analysis_modules/halo_finding/rockstar/rockstar_interface.pyx",
> line 254, in
> yt.analysis_modules.halo_finding.rockstar.rockstar_interface.RockstarInterface.setup_rockstar
> (yt/analysis_modules/halo_finding/rockstar/rockstar_interface.c:3785)
> P002 yt : [ERROR ] 2016-06-07 14:46:19,758 TypeError: expected bytes, str
> found
> File "/home/jullian/ABinitio/Out/DD0050/halo_rockstar.py", line 62, in
> <module>
> hc.create()
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
> line 293, in create
> self._run(save_halos, save_catalog, njobs=njobs, dynamic=dynamic)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/utilities/parallel_tools/parallel_analysis_interface.py",
> line 305, in barrierize
> retval = func(*args, **kwargs)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
> line 362, in _run
> self.halos_ds = self.finder_method(self.data_ds)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
> line 46, in __call__
> return self.function(ds, *self.args, **self.kwargs)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
> line 79, in _rockstar_method
> rh.run()
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_finding/rockstar/rockstar.py",
> line 348, in run
> min_halo_size = self.min_halo_size)
> File "yt/analysis_modules/halo_finding/rockstar/rockstar_interface.pyx",
> line 254, in
> yt.analysis_modules.halo_finding.rockstar.rockstar_interface.RockstarInterface.setup_rockstar
> (yt/analysis_modules/halo_finding/rockstar/rockstar_interface.c:3785)
> P003 yt : [ERROR ] 2016-06-07 14:46:19,758 TypeError: expected bytes, str
> found
> P002 yt : [ERROR ] 2016-06-07 14:46:19,759 Error occured on rank 0.
> application called MPI_Abort(MPI_COMM_WORLD, 1) - process 2
> P003 yt : [ERROR ] 2016-06-07 14:46:19,759 Error occured on rank 1.
> application called MPI_Abort(MPI_COMM_WORLD, 1) - process 3
> File "/home/jullian/ABinitio/Out/DD0050/halo_rockstar.py", line 62, in
> <module>
> hc.create()
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
> line 293, in create
> self._run(save_halos, save_catalog, njobs=njobs, dynamic=dynamic)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/utilities/parallel_tools/parallel_analysis_interface.py",
> line 305, in barrierize
> retval = func(*args, **kwargs)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
> line 362, in _run
> self.halos_ds = self.finder_method(self.data_ds)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
> line 46, in __call__
> return self.function(ds, *self.args, **self.kwargs)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
> line 79, in _rockstar_method
> rh.run()
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_finding/rockstar/rockstar.py",
> line 348, in run
> min_halo_size = self.min_halo_size)
> File "yt/analysis_modules/halo_finding/rockstar/rockstar_interface.pyx",
> line 254, in
> yt.analysis_modules.halo_finding.rockstar.rockstar_interface.RockstarInterface.setup_rockstar
> (yt/analysis_modules/halo_finding/rockstar/rockstar_interface.c:3785)
> P000 yt : [ERROR ] 2016-06-07 14:46:19,761 TypeError: expected bytes, str
> found
> application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
> File "/home/jullian/ABinitio/Out/DD0050/halo_rockstar.py", line 62, in
> <module>
> hc.create()
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
> line 293, in create
> self._run(save_halos, save_catalog, njobs=njobs, dynamic=dynamic)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/utilities/parallel_tools/parallel_analysis_interface.py",
> line 305, in barrierize
> retval = func(*args, **kwargs)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_catalog.py",
> line 362, in _run
> self.halos_ds = self.finder_method(self.data_ds)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
> line 46, in __call__
> return self.function(ds, *self.args, **self.kwargs)
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_analysis/halo_finding_methods.py",
> line 79, in _rockstar_method
> rh.run()
> File
> "/home/jullian/yt-x86_64/src/yt-hg/yt/analysis_modules/halo_finding/rockstar/rockstar.py",
> line 348, in run
> min_halo_size = self.min_halo_size)
> File "yt/analysis_modules/halo_finding/rockstar/rockstar_interface.pyx",
> line 254, in
> yt.analysis_modules.halo_finding.rockstar.rockstar_interface.RockstarInterface.setup_rockstar
> (yt/analysis_modules/halo_finding/rockstar/rockstar_interface.c:3785)
> P001 yt : [ERROR ] 2016-06-07 14:46:19,761 TypeError: expected bytes, str
> found
> application called MPI_Abort(MPI_COMM_WORLD, 1) - process 1
>
> ===================================================================================
> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
> = EXIT CODE: 1
> = CLEANING UP REMAINING PROCESSES
> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
> ===================================================================================
>
> ===================================================================================
> = BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
> = EXIT CODE: 1
> = CLEANING UP REMAINING PROCESSES
> = YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
> ===================================================================================
>
>
>
> If I try to run with the command suggested in the yt page (mpirun -n 4
> $PBS_NODEFILE --mca btl ^openib python3
> "/home/jullian/ABinitio/Out/DD0050/halo_rockstar.py" --parallel) I get:
>
> [mpiexec at julian-ubuntu] match_arg (./utils/args/args.c:160): unrecognized
> argument mca
> [mpiexec at julian-ubuntu] HYDU_parse_array (./utils/args/args.c:175): argument
> matching returned error
> [mpiexec at julian-ubuntu] parse_args (./ui/mpich/utils.c:1609): error parsing
> input array
> [mpiexec at julian-ubuntu] HYD_uii_mpx_get_parameters
> (./ui/mpich/utils.c:1660): unable to parse user arguments
> [mpiexec at julian-ubuntu] main (./ui/mpich/mpiexec.c:153): error parsing
> parameters
>
> How could I fix this problem? Do I have to launch the script in some remote
> machine? Do I have to use the IPython?
>
>
> Thanks by the attention,
>
> Best Regards,
>
> Jullian
>
>
> Script link:
> http://yt-project.org/doc/cookbook/cosmological_analysis.html#running-rockstar-to-find-halos-on-multi-resolution-particle-datasets
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
More information about the yt-users
mailing list