[yt-users] error when running on cluster nodes

[Matthew Turk]

Hi Tsz Ka,

For me, otool is just in /usr/bin -- so it sounds like it may not be
there.  There is a chance that 'ldd' will also work for this purpose.

It sounds to me like this could be an issue of the DYLD_LIBRARY_PATH
being confused, as well as possibly being unable to load various
libraries.  I'm not sure that I'm able to debug this from a distance
-- the combination of the ppc64 cluster (which I am unfamiliar with)
and the particulars of the local disk are making things a bit
difficult.  Any chance you could run the analysis on the head node?
How big is your dataset?  It may not benefit too much from
parallelism.

-Matt

On Fri, Feb 18, 2011 at 3:39 PM, Tsz Ka Li <tszkali2 at illinois.edu> wrote:
> Hi Matt,
> I am afraid otool is not installed on the Turing cluster. (I got command not
> found.) Do you know of any substitute for that tool?
> Actually I did not follow the whole Cray installation instructions, but just
> those in the Section "Running on a compute node". So basically I just copied
> the yt directory to the scratch disk and used the python there. Maybe this
> does not make sense to you. Anyway sorry for the confusion.
> Thanks,
> Tsz Ka
>
>
> On 2/18/2011 9:26 AM, Matthew Turk wrote:
>>
>> Hi Tsz Ka,
>>
>> I have been thinking about it, and I am afraid I don't have any bright
>> ideas.  The only thing I can think of is that we may be able to get
>> more information about the dynamic loading problem by having you run:
>>
>> otool -L
>>  /turing/home/tszkali2/yt-ppc64/lib/python2.6/site-packages/matplotlib/_path.so
>>
>> I would again caution though that if you attempted to create a
>> statically linked library, this likely resulted in major breakages.
>> The static linking process described on the wiki is only really
>> designed for the Cray Compute Node Linux distribution, and it has a
>> fundamentally different mode of linking than Darwin/OSX.
>>
>> Could you send the output of that to us, run on both the compute nodes
>> and on the head node?  I presume loading yt still runs on the head
>> node?
>>
>> Thanks,
>>
>> Matt
>>
>> On Thu, Feb 17, 2011 at 12:38 PM, Tsz Ka Li<tszkali2 at illinois.edu>  wrote:
>>>
>>> Hi Stephen,
>>> I did try to include the -V flag in the interactive mode (i.e. qsub -I
>>> -V)
>>> and I saw no difference in the error outcome.
>>> Thanks,
>>> Tsz Ka
>>>
>>> On 2/17/2011 11:19 AM, Stephen Skory wrote:
>>>>
>>>> Tsz Ka,
>>>>
>>>>> I used the same python (~/yt-ppc64/bin/python) as I did in the command
>>>>> line. I think I tried python2.6 but I got the same error. (Actually how
>>>>> are they different?) The machine I am using is the Turing cluster in UI
>>>>> (http://www.cse.illinois.edu/turing/), which is an Apple Xserve cluster
>>>>> using G5 processors. You can find on the website some information about
>>>>> it. Sorry I am not that familiar with cluster issues. I suppose the
>>>>> nodes have full installation, though I don't know where to check. I am
>>>>> using the default compilers on Turing, which are gcc for C/C++ and xlf
>>>>> for Fortran.
>>>>
>>>> I'm going to chime in here. I noticed that on their FAQ page:
>>>>
>>>> http://www.cse.illinois.edu/turing/faq.html
>>>>
>>>> There's something about path problems on compute nodes. Your error could
>>>> be due to a library problem (i.e. DYLD_LIBRARY_PATH). Have you tried
>>>> submitting your job with a "#PBS -V" that keeps your environment for the
>>>> job?
>>>>
>>>> Matt may have some other bright ideas...
>>>>
>>>>
>>>>  Stephen Skory
>>>> stephenskory at yahoo.com
>>>> http://stephenskory.com/
>>>> 510.621.3687 (google voice)
>>>> _______________________________________________
>>>> yt-users mailing list
>>>> yt-users at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>>
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>