Thanks, Matt. That appears to do the trick. <br><br>~Andrew <br><br><div class="gmail_quote">On Wed, Feb 2, 2011 at 10:22 AM, Matthew Turk <span dir="ltr"><<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">Hi Andrew,<br>
<br>
Okay. In the absence of this, I've attempted a workaround. If you<br>
are on the development branch, you should be able to run:<br>
<br>
yt instinfo -u<br>
<br>
and get the latest version, which includes an explicit check on the<br>
bin_id access. If you're on stable, contact me off-list and I'll send<br>
you a modified version to make this check.<br>
<br>
Best,<br>
<br>
Matt<br>
<div><div></div><div class="h5"><br>
On Wed, Feb 2, 2011 at 1:03 PM, Andrew Myers <<a href="mailto:atmyers@berkeley.edu">atmyers@berkeley.edu</a>> wrote:<br>
> Hi guys,<br>
><br>
> Actually, the code:<br>
><br>
> for g in pf.h.grids:<br>
> if na.any(na.isnan(na.log10(g["Density"]))): raise RuntimeError<br>
><br>
> does seem to proceed to completion without raising an exception.<br>
><br>
> Andrew M<br>
><br>
><br>
> On Wed, Feb 2, 2011 at 9:55 AM, Matthew Turk <<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>> wrote:<br>
>><br>
>> Hi Andrew,<br>
>><br>
>> This is I think a spurious issue -- it comes from the calculation of<br>
>> the radius when detecting which derived fields can be used by yt. I<br>
>> would actually guess that you see this whenever the hierarchy is<br>
>> instantiated:<br>
>><br>
>> pf.h<br>
>><br>
>> and not only when the PlotCollection goes out. I tried kind of hard<br>
>> to eliminate this issue but I couldn't figure out a way; I think it's<br>
>> cosmetic and unrelated, though. Can you have a go at the checking for<br>
>> NaNs code?<br>
>><br>
>> -Matt<br>
>><br>
>> On Wed, Feb 2, 2011 at 12:51 PM, Andrew Myers <<a href="mailto:atmyers@berkeley.edu">atmyers@berkeley.edu</a>><br>
>> wrote:<br>
>> > Actually, and I probably should have mentioned this earlier, but I<br>
>> > remember<br>
>> > now that once I upgraded to yt-2.0 I started seeing warning messages<br>
>> > like:<br>
>> ><br>
>> > Warning: invalid value encountered in sqrt<br>
>> ><br>
>> > when I create a plot collection from this dataset, which supports the<br>
>> > negative value theory. I assume that these values were always there,<br>
>> > but<br>
>> > yt-2.0 started warning me about them. Probably the center of the domain<br>
>> > does<br>
>> > not include the bad value(s), so it only chokes the volume renderer if I<br>
>> > make the domain large enough. Thanks for the help, both of you!<br>
>> ><br>
>> > ~Andrew<br>
>> ><br>
>> > On Wed, Feb 2, 2011 at 9:42 AM, Matthew Turk <<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>><br>
>> > wrote:<br>
>> >><br>
>> >> Jeff, you are absolutely right. Maybe a better solution would be:<br>
>> >><br>
>> >> for g in pf.h.grids:<br>
>> >> if na.any(na.isnan.na.log10(grid["Density"]))): raise RuntimeError<br>
>> >><br>
>> >> That should be a more true-to-life method.<br>
>> >><br>
>> >> -Matt<br>
>> >><br>
>> >> On Wed, Feb 2, 2011 at 12:40 PM, j s oishi <<a href="mailto:jsoishi@gmail.com">jsoishi@gmail.com</a>> wrote:<br>
>> >> > Hi Matt,<br>
>> >> ><br>
>> >> > Could it not also be from a negative value that has had its log()<br>
>> >> > taken?<br>
>> >> ><br>
>> >> > j<br>
>> >> ><br>
>> >> ><br>
>> >> ><br>
>> >> ><br>
>> >> ><br>
>> >> > On Wed, Feb 2, 2011 at 9:38 AM, Matthew Turk <<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>><br>
>> >> > wrote:<br>
>> >> >> Hi Andrew,<br>
>> >> >><br>
>> >> >> Okay, I have seen this before. I think it is related to a bad data<br>
>> >> >> value that it's trying to traverse. In the past this has been<br>
>> >> >> caused<br>
>> >> >> by NaNs, usually from a zero value that has been logged, as I<br>
>> >> >> believe<br>
>> >> >> other values should be handled correctly. Can you try this code for<br>
>> >> >> me?<br>
>> >> >><br>
>> >> >> for g in pf.h.grids:<br>
>> >> >> if na.any(grid["Density"] == 0): raise RuntimeError<br>
>> >> >><br>
>> >> >> and see if it proceeds to completion? I will see if I can think of<br>
>> >> >> a<br>
>> >> >> good way to handle NaNs; obviously a segfault is a pretty poor<br>
>> >> >> strategy.<br>
>> >> >><br>
>> >> >> -Matt<br>
>> >> >><br>
>> >> >> On Wed, Feb 2, 2011 at 12:32 PM, Andrew Myers <<a href="mailto:atmyers@berkeley.edu">atmyers@berkeley.edu</a>><br>
>> >> >> wrote:<br>
>> >> >>> Hi Matt,<br>
>> >> >>><br>
>> >> >>> Thanks for the help. This is the outcome of the "bt" command in<br>
>> >> >>> gdb:<br>
>> >> >>><br>
>> >> >>> (gdb) bt<br>
>> >> >>> #0 __pyx_f_2yt_9utilities_9amr_utils_FIT_get_value<br>
>> >> >>> (__pyx_v_self=0x87ab9c0,<br>
>> >> >>> __pyx_v_dt=0.00024472523295100413, __pyx_v_dvs=0x50e9d670,<br>
>> >> >>> __pyx_v_rgba=0x7fffd8a94f60,<br>
>> >> >>> __pyx_v_grad=<value optimized out>) at<br>
>> >> >>> yt/utilities/amr_utils.c:13705<br>
>> >> >>> #1<br>
>> >> >>><br>
>> >> >>> __pyx_f_2yt_9utilities_9amr_utils_21TransferFunctionProxy_eval_transfer<br>
>> >> >>> (__pyx_v_self=0x87ab9c0, __pyx_v_dt=0.00024472523295100413,<br>
>> >> >>> __pyx_v_dvs=0x50e9d670,<br>
>> >> >>> __pyx_v_rgba=0x7fffd8a94f60, __pyx_v_grad=<value optimized<br>
>> >> >>> out>)<br>
>> >> >>> at<br>
>> >> >>> yt/utilities/amr_utils.c:14285<br>
>> >> >>> #2 0x00002b5e0a62c464 in<br>
>> >> >>> __pyx_f_2yt_9utilities_9amr_utils_15PartitionedGrid_sample_values<br>
>> >> >>> (__pyx_v_self=0x50e9d610, __pyx_v_v_pos=<value optimized out>,<br>
>> >> >>> __pyx_v_v_dir=<value optimized out>,<br>
>> >> >>> __pyx_v_enter_t=23.346866210722702,<br>
>> >> >>> __pyx_v_exit_t=<value optimized out>, __pyx_v_ci=<value optimized<br>
>> >> >>> out>,<br>
>> >> >>> __pyx_v_rgba=0x7fffd8a94f60, __pyx_v_tf=0x87ab9c0) at<br>
>> >> >>> yt/utilities/amr_utils.c:17719<br>
>> >> >>> #3 0x00002b5e0a62ce16 in<br>
>> >> >>> __pyx_f_2yt_9utilities_9amr_utils_15PartitionedGrid_integrate_ray<br>
>> >> >>> (__pyx_v_self=0x50e9d610, __pyx_v_v_pos=0x7fffd8a94fd0,<br>
>> >> >>> __pyx_v_v_dir=0x45457d0, __pyx_v_rgba=0x7fffd8a94f60,<br>
>> >> >>> __pyx_v_tf=0x87ab9c0) at yt/utilities/amr_utils.c:17386<br>
>> >> >>> #4 0x00002b5e0a624876 in<br>
>> >> >>> __pyx_pf_2yt_9utilities_9amr_utils_15PartitionedGrid_2cast_plane<br>
>> >> >>> (__pyx_v_self=0x50e9d610, __pyx_args=<value optimized out>,<br>
>> >> >>> __pyx_kwds=<value optimized out>) at<br>
>> >> >>> yt/utilities/amr_utils.c:16199<br>
>> >> >>> #5 0x0000000000495124 in call_function (f=0x5a7ce490,<br>
>> >> >>> throwflag=<value<br>
>> >> >>> optimized out>) at Python/ceval.c:3706<br>
>> >> >>> #6 PyEval_EvalFrameEx (f=0x5a7ce490, throwflag=<value optimized<br>
>> >> >>> out>)<br>
>> >> >>> at<br>
>> >> >>> Python/ceval.c:2389<br>
>> >> >>> #7 0x00000000004943ff in call_function (f=0x87aa260,<br>
>> >> >>> throwflag=<value<br>
>> >> >>> optimized out>) at Python/ceval.c:3792<br>
>> >> >>> #8 PyEval_EvalFrameEx (f=0x87aa260, throwflag=<value optimized<br>
>> >> >>> out>)<br>
>> >> >>> at<br>
>> >> >>> Python/ceval.c:2389<br>
>> >> >>> #9 0x0000000000495d6d in PyEval_EvalCodeEx (co=0x24286c0,<br>
>> >> >>> globals=<value<br>
>> >> >>> optimized out>, locals=<value optimized out>, args=0xb62c38,<br>
>> >> >>> argcount=2,<br>
>> >> >>> kws=0xb62c48,<br>
>> >> >>> kwcount=0, defs=0x242a2a8, defcount=1, closure=0x0) at<br>
>> >> >>> Python/ceval.c:2968<br>
>> >> >>> #10 0x0000000000493c79 in call_function (f=0xb62ac0,<br>
>> >> >>> throwflag=<value<br>
>> >> >>> optimized out>) at Python/ceval.c:3802<br>
>> >> >>> #11 PyEval_EvalFrameEx (f=0xb62ac0, throwflag=<value optimized<br>
>> >> >>> out>)<br>
>> >> >>> at<br>
>> >> >>> Python/ceval.c:2389<br>
>> >> >>> #12 0x0000000000495d6d in PyEval_EvalCodeEx (co=0x2b5e01aed288,<br>
>> >> >>> globals=<value optimized out>, locals=<value optimized out>,<br>
>> >> >>> args=0x0,<br>
>> >> >>> argcount=0, kws=0x0, kwcount=0,<br>
>> >> >>> defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2968<br>
>> >> >>> #13 0x0000000000495db2 in PyEval_EvalCode (co=0x87ab9c0,<br>
>> >> >>> globals=0x50e9d670,<br>
>> >> >>> locals=0x72f1270) at Python/ceval.c:522<br>
>> >> >>> #14 0x00000000004b7ee1 in run_mod (fp=0xb54ed0,<br>
>> >> >>> filename=0x7fffd8a965a4<br>
>> >> >>> "vr.py", start=<value optimized out>, globals=0xb03190,<br>
>> >> >>> locals=0xb03190,<br>
>> >> >>> closeit=1,<br>
>> >> >>> flags=0x7fffd8a958d0) at Python/pythonrun.c:1335<br>
>> >> >>> #15 PyRun_FileExFlags (fp=0xb54ed0, filename=0x7fffd8a965a4<br>
>> >> >>> "vr.py",<br>
>> >> >>> start=<value optimized out>, globals=0xb03190, locals=0xb03190,<br>
>> >> >>> closeit=1,<br>
>> >> >>> flags=0x7fffd8a958d0)<br>
>> >> >>> at Python/pythonrun.c:1321<br>
>> >> >>> #16 0x00000000004b8198 in PyRun_SimpleFileExFlags (fp=<value<br>
>> >> >>> optimized<br>
>> >> >>> out>,<br>
>> >> >>> filename=0x7fffd8a965a4 "vr.py", closeit=1, flags=0x7fffd8a958d0)<br>
>> >> >>> at<br>
>> >> >>> Python/pythonrun.c:931<br>
>> >> >>> #17 0x0000000000413e4f in Py_Main (argc=<value optimized out>,<br>
>> >> >>> argv=0x7fffd8a959f8) at Modules/main.c:599<br>
>> >> >>> #18 0x00002b5e0259a994 in __libc_start_main () from<br>
>> >> >>> /lib64/libc.so.6<br>
>> >> >>> #19 0x00000000004130b9 in _start ()<br>
>> >> >>><br>
>> >> >>> Thanks,<br>
>> >> >>> Andrew<br>
>> >> >>><br>
>> >> >>><br>
>> >> >>> On Wed, Feb 2, 2011 at 5:25 AM, Matthew Turk<br>
>> >> >>> <<a href="mailto:matthewturk@gmail.com">matthewturk@gmail.com</a>><br>
>> >> >>> wrote:<br>
>> >> >>>><br>
>> >> >>>> Hi Andrew,<br>
>> >> >>>><br>
>> >> >>>> That's an odd bug! Do you think you could get a backtrace from<br>
>> >> >>>> the<br>
>> >> >>>> segfault? You might do this by setting your core dump ulimit to<br>
>> >> >>>> unlimited:<br>
>> >> >>>><br>
>> >> >>>> [in base]<br>
>> >> >>>><br>
>> >> >>>> ulimit -c unlimited<br>
>> >> >>>><br>
>> >> >>>> [in csh]<br>
>> >> >>>><br>
>> >> >>>> limit coredumpsize unlimited<br>
>> >> >>>><br>
>> >> >>>> and then running again. When the core dump gets spit out,<br>
>> >> >>>><br>
>> >> >>>> gdb python2.6 -c that_core_file<br>
>> >> >>>> bt<br>
>> >> >>>><br>
>> >> >>>> should tell us where in the code it died. Sam Skillman should<br>
>> >> >>>> have a<br>
>> >> >>>> better idea about any possible memory issues, but the segfault to<br>
>> >> >>>> me<br>
>> >> >>>> feels like maybe there's a roundoff that's putting it outside a<br>
>> >> >>>> grid<br>
>> >> >>>> data array space or something.<br>
>> >> >>>><br>
>> >> >>>> Sorry for the trouble,<br>
>> >> >>>><br>
>> >> >>>> Matt<br>
>> >> >>>><br>
>> >> >>>> On Tue, Feb 1, 2011 at 11:13 PM, Andrew Myers<br>
>> >> >>>> <<a href="mailto:atmyers@berkeley.edu">atmyers@berkeley.edu</a>><br>
>> >> >>>> wrote:<br>
>> >> >>>> > Hello yt users,<br>
>> >> >>>> ><br>
>> >> >>>> > I'm trying to volume render an Orion simulation with about 6,000<br>
>> >> >>>> > grids<br>
>> >> >>>> > and<br>
>> >> >>>> > 100 million cells, and I think I'm running out of memory. I<br>
>> >> >>>> > don't<br>
>> >> >>>> > know<br>
>> >> >>>> > if<br>
>> >> >>>> > this is large compared to other simulations people have volume<br>
>> >> >>>> > rendered<br>
>> >> >>>> > before, but if I set the width of my field of view to be 0.02 pc<br>
>> >> >>>> > (20<br>
>> >> >>>> > times<br>
>> >> >>>> > smaller than the entire domain), the following code works fine.<br>
>> >> >>>> > If<br>
>> >> >>>> > I set<br>
>> >> >>>> > it<br>
>> >> >>>> > to 0.04 pc or anything larger, the code segfaults, which I<br>
>> >> >>>> > assume<br>
>> >> >>>> > means<br>
>> >> >>>> > I'm<br>
>> >> >>>> > running out of memory. This happens no matter how many cores I<br>
>> >> >>>> > run<br>
>> >> >>>> > on -<br>
>> >> >>>> > running in parallel seems to be speed up the calculation, but<br>
>> >> >>>> > not<br>
>> >> >>>> > increase<br>
>> >> >>>> > the size of the domain I can render. Am I doing something wrong?<br>
>> >> >>>> > Or<br>
>> >> >>>> > do I<br>
>> >> >>>> > just need to find a machine with more memory to do this on? The<br>
>> >> >>>> > one<br>
>> >> >>>> > I'm<br>
>> >> >>>> > using now has 3 gigs per core, which strikes me as pretty solid.<br>
>> >> >>>> > I'm<br>
>> >> >>>> > using<br>
>> >> >>>> > the trunk version of yt-2.0. Here's the script for reference:<br>
>> >> >>>> ><br>
>> >> >>>> > from yt.mods import *<br>
>> >> >>>> ><br>
>> >> >>>> > pf = load("plt01120")<br>
>> >> >>>> ><br>
>> >> >>>> > dd = pf.h.all_data()<br>
>> >> >>>> > mi, ma = na.log10(dd.quantities["Extrema"]("Density")[0])<br>
>> >> >>>> > mi -= 0.1 ; ma += 0.1 # To allow a bit of room at the<br>
>> >> >>>> > edges<br>
>> >> >>>> ><br>
>> >> >>>> > tf = ColorTransferFunction((mi, ma))<br>
>> >> >>>> > tf.add_layers(8, w=0.01)<br>
>> >> >>>> > c = na.array([0.0,0.0,0.0])<br>
>> >> >>>> > L = na.array([1.0, 1.0, 1.0])<br>
>> >> >>>> > W = 6.17e+16 # 0.02<br>
>> >> >>>> > pc<br>
>> >> >>>> ><br>
>> >> >>>> > N = 512<br>
>> >> >>>> ><br>
>> >> >>>> > cam = Camera(c, L, W, (N,N), tf, pf=pf)<br>
>> >> >>>> > fn = "%s_image.png" % pf<br>
>> >> >>>> ><br>
>> >> >>>> > cam.snapshot(fn)<br>
>> >> >>>> ><br>
>> >> >>>> > Thanks,<br>
>> >> >>>> > Andrew Myers<br>
>> >> >>>> ><br>
>> >> >>>> ><br>
>> >> >>>> ><br>
>> >> >>>> > _______________________________________________<br>
>> >> >>>> > yt-users mailing list<br>
>> >> >>>> > <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
>> >> >>>> > <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
>> >> >>>> ><br>
>> >> >>>> ><br>
>> >> >>>> _______________________________________________<br>
>> >> >>>> yt-users mailing list<br>
>> >> >>>> <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
>> >> >>>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
>> >> >>><br>
>> >> >>><br>
>> >> >>> _______________________________________________<br>
>> >> >>> yt-users mailing list<br>
>> >> >>> <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
>> >> >>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
>> >> >>><br>
>> >> >>><br>
>> >> >> _______________________________________________<br>
>> >> >> yt-users mailing list<br>
>> >> >> <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
>> >> >> <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
>> >> >><br>
>> >> > _______________________________________________<br>
>> >> > yt-users mailing list<br>
>> >> > <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
>> >> > <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
>> >> ><br>
>> >> _______________________________________________<br>
>> >> yt-users mailing list<br>
>> >> <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
>> >> <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
>> ><br>
>> ><br>
>> > _______________________________________________<br>
>> > yt-users mailing list<br>
>> > <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
>> > <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
>> ><br>
>> ><br>
>> _______________________________________________<br>
>> yt-users mailing list<br>
>> <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
>> <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
><br>
><br>
> _______________________________________________<br>
> yt-users mailing list<br>
> <a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
> <a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
><br>
><br>
_______________________________________________<br>
yt-users mailing list<br>
<a href="mailto:yt-users@lists.spacepope.org">yt-users@lists.spacepope.org</a><br>
<a href="http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org" target="_blank">http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org</a><br>
</div></div></blockquote></div><br>