[yt-users] in-situ visualization possible with yt?

Tue Nov 8 11:31:10 PST 2011

Thanks, I managed to make what I wanted with matplotlib.

Matthieu

2011/11/8 Britton Smith <brittonsmith at gmail.com>

> Hi Matthieu,
>
> If you already have arrays that you are ready to plot, you are probably
> better going straight to matplotlib from here.  You can find their
> documentation here: http://matplotlib.sourceforge.net/
> This is the package used for most of the plotting and visualization that
> yt does.
>
> For example, you would probably want to use pcolormesh to plot up a 2D
> numpy array:
>
> http://matplotlib.sourceforge.net/api/pyplot_api.html#matplotlib.pyplot.pcolormesh
>
> Britton
>
>
> On Tue, Nov 8, 2011 at 8:50 AM, Matthieu Dorier <matthieu.dorier at gmail.com
> > wrote:
>
>> Hello,
>>
>> I come back to this discussion, as I have instrumented my simulation to
>> wrap the C++ arrays into NumPy arrays and call a python script in each
>> process.
>> This currently works and now I would like to actually do something with
>> these NumPy arrays. I can't find any tutorial on the YT homepage on how to
>> create an image from a NumPy array.
>> (I'm not only new to YT, I'm also new to Python and learned the basic
>> syntax while coding the wrapper).
>>
>> Let's imagine I have a 2D NumPy array of floating point values, and I
>> want to plot it as an image, how would the python code look like?
>>
>> Thanks
>>
>> Matthieu
>>
>> 2011/10/5 Matthew Turk <matthewturk at gmail.com>
>>
>>> Hi Matthieu,
>>>
>>> On Wed, Oct 5, 2011 at 11:45 AM, Matthieu Dorier
>>> <matthieu.dorier at gmail.com> wrote:
>>> > Hello Matthew,
>>> >
>>> > Thank you for your answer.
>>> >
>>> > Can YT work in parallel?
>>>
>>> Yup.
>>>
>>> > I intend to embed a python interpreter within my simulation in order to
>>> > allow
>>> > some simple data analysis to be performed without having to re-compile
>>> the
>>> > application every time this analysis changes, so maybe it would be
>>> possible
>>> > to call YT from an embedded python interpreter? I guess the problem is
>>> > transferring raw data from the simulation to the interpreter without
>>> too
>>> > much
>>> > overhead...
>>>
>>> That's actually how Enzo works as well.  We then construct numpy array
>>> wrappers around the already-allocated data, and then feed those to yt
>>> in a dict.
>>>
>>> >
>>> > My output is actually very simple, consisting in 2D arrays, each MPI
>>> rank
>>> > being responsible for several chunks (equally sized, but not
>>> necessarily
>>> > contiguous). I would like to avoid writing these 2D arrays into HDF5
>>> file
>>> > and perform visualization on these files, but automatically make the
>>> > processes
>>> > synchronize to compute a single image from all these parts.
>>>
>>> That's what we do with Enzo, except with 3D.  There may be some
>>> oddities to work out with 2D data.
>>>
>>> >
>>> > Do you think it is worth investigating the use of YT for that or
>>> should I
>>> > try other
>>> > softwares?
>>> > Thanks
>>> >
>>> > Matthieu
>>>
>>> I think you should give a look at yt -- but also guide your views by
>>> reading how we do in situ with Enzo.  If you look in the Enzo source
>>> code at InitializePythonInterface.C and ExposeDataHierarchy.C you can
>>> see our current method, although I believe in the future with better
>>> memory-pool allocations in Enzo we'll make this a bit more
>>> streamlined.
>>>
>>> Please don't hesitate to write back with more questions!
>>>
>>> Best,
>>>
>>> Matt
>>>
>>> >
>>> > 2011/10/5 Matthew Turk <matthewturk at gmail.com>
>>> >>
>>> >> Hi Matthieu,
>>> >>
>>> >> The short answer is that for some codes, yes, yt works as an embedding
>>> >> system.  The only code that this has been used extensively in is Enzo,
>>> >> and it will likely take a bit of work to get it to work.  We're eager
>>> >> to test it out on other codes, however; the first and easiest
>>> >> application would be a non-Enzo, patch-based AMR code.
>>> >>
>>> >> The current state is:
>>> >>
>>> >>  * In-memory between Enzo and yt works (Britton Smith can speak more
>>> to
>>> >> this)
>>> >>  * Creation of arbitrary sources (using the Stream frontend) currently
>>> >> works, which can be used from within any Python code
>>> >>  * No protocol exists (yet) for MPI inter-communicator data loading
>>> >>
>>> >> Our strategies for the future:
>>> >>
>>> >>  * We're working to create simple C libraries that can create the
>>> >> necessary in-memory data objects to link against yt directly
>>> >>  * The Stream frontend is usable and should be used by non-Enzo codes
>>> >> as a prototyping mechanism
>>> >>  * ParaView can call yt for serial calculations, and we are working to
>>> >> utilize its CoProcessing facilities to analyze and visualize in
>>> >> parallel from running simulations
>>> >>
>>> >> So I think you should be able to add a code to in situ viz/analysis,
>>> >> but it will likely take a bit of work at the current time to make sure
>>> >> everything correctly passes.  I'd be able to help you and send you
>>> >> demo code if you would like.  Alternately, we intend to instrument yt
>>> >> to use ParaView as a data loader, which will enable the CoProcessing
>>> >> techniques from ParaView to be used as well.
>>> >>
>>> >> Let me know what you think,
>>> >>
>>> >> Matt
>>> >>
>>> >> On Tue, Oct 4, 2011 at 1:28 PM, Matthieu Dorier
>>> >> <matthieu.dorier at gmail.com> wrote:
>>> >> > Hello,
>>> >> >
>>> >> > I'm trying to make in-situ visualization available for a simulation,
>>> >> > and I'm considering different softwares for that (VisIt, YT,...).
>>> >> > I've never used YT, so my question is: is it possible to add
>>> >> > some code in a simulation (written in C++) to have some kind of
>>> >> > distributed rendering engine using YT, just like VisIt does with
>>> >> > code-instrumentation?
>>> >> > Thanks,
>>> >> >
>>> >> > Matthieu
>>> >> >
>>> >> > --
>>> >> > Matthieu Dorier
>>> >> > ENS Cachan, antenne de Bretagne
>>> >> > Département informatique et télécommunication
>>> >> > http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/
>>> >> >
>>> >> > _______________________________________________
>>> >> > yt-users mailing list
>>> >> > yt-users at lists.spacepope.org
>>> >> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>> >> >
>>> >> >
>>> >> _______________________________________________
>>> >> yt-users mailing list
>>> >> yt-users at lists.spacepope.org
>>> >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>> >
>>> >
>>> >
>>> > --
>>> > Matthieu Dorier
>>> > ENS Cachan, antenne de Bretagne
>>> > Département informatique et télécommunication
>>> > http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/
>>> >
>>> > _______________________________________________
>>> > yt-users mailing list
>>> > yt-users at lists.spacepope.org
>>> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>> >
>>> >
>>> _______________________________________________
>>> yt-users mailing list
>>> yt-users at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>>
>>
>>
>>
>> --
>> Matthieu Dorier
>> ENS Cachan, antenne de Bretagne
>> Département informatique et télécommunication
>> http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/
>>
>> _______________________________________________
>> yt-users mailing list
>> yt-users at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>>
>>
>
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>
>

-- 
Matthieu Dorier
ENS Cachan, antenne de Bretagne
Département informatique et télécommunication
http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20111108/05168d2f/attachment.htm>