[yt-users] in-situ visualization possible with yt?

Tue Nov 8 05:50:42 PST 2011

Hello,

I come back to this discussion, as I have instrumented my simulation to
wrap the C++ arrays into NumPy arrays and call a python script in each
process.
This currently works and now I would like to actually do something with
these NumPy arrays. I can't find any tutorial on the YT homepage on how to
create an image from a NumPy array.
(I'm not only new to YT, I'm also new to Python and learned the basic
syntax while coding the wrapper).

Let's imagine I have a 2D NumPy array of floating point values, and I want
to plot it as an image, how would the python code look like?

Thanks

Matthieu

2011/10/5 Matthew Turk <matthewturk at gmail.com>

> Hi Matthieu,
>
> On Wed, Oct 5, 2011 at 11:45 AM, Matthieu Dorier
> <matthieu.dorier at gmail.com> wrote:
> > Hello Matthew,
> >
> > Thank you for your answer.
> >
> > Can YT work in parallel?
>
> Yup.
>
> > I intend to embed a python interpreter within my simulation in order to
> > allow
> > some simple data analysis to be performed without having to re-compile
> the
> > application every time this analysis changes, so maybe it would be
> possible
> > to call YT from an embedded python interpreter? I guess the problem is
> > transferring raw data from the simulation to the interpreter without too
> > much
> > overhead...
>
> That's actually how Enzo works as well.  We then construct numpy array
> wrappers around the already-allocated data, and then feed those to yt
> in a dict.
>
> >
> > My output is actually very simple, consisting in 2D arrays, each MPI rank
> > being responsible for several chunks (equally sized, but not necessarily
> > contiguous). I would like to avoid writing these 2D arrays into HDF5 file
> > and perform visualization on these files, but automatically make the
> > processes
> > synchronize to compute a single image from all these parts.
>
> That's what we do with Enzo, except with 3D.  There may be some
> oddities to work out with 2D data.
>
> >
> > Do you think it is worth investigating the use of YT for that or should I
> > try other
> > softwares?
> > Thanks
> >
> > Matthieu
>
> I think you should give a look at yt -- but also guide your views by
> reading how we do in situ with Enzo.  If you look in the Enzo source
> code at InitializePythonInterface.C and ExposeDataHierarchy.C you can
> see our current method, although I believe in the future with better
> memory-pool allocations in Enzo we'll make this a bit more
> streamlined.
>
> Please don't hesitate to write back with more questions!
>
> Best,
>
> Matt
>
> >
> > 2011/10/5 Matthew Turk <matthewturk at gmail.com>
> >>
> >> Hi Matthieu,
> >>
> >> The short answer is that for some codes, yes, yt works as an embedding
> >> system.  The only code that this has been used extensively in is Enzo,
> >> and it will likely take a bit of work to get it to work.  We're eager
> >> to test it out on other codes, however; the first and easiest
> >> application would be a non-Enzo, patch-based AMR code.
> >>
> >> The current state is:
> >>
> >>  * In-memory between Enzo and yt works (Britton Smith can speak more to
> >> this)
> >>  * Creation of arbitrary sources (using the Stream frontend) currently
> >> works, which can be used from within any Python code
> >>  * No protocol exists (yet) for MPI inter-communicator data loading
> >>
> >> Our strategies for the future:
> >>
> >>  * We're working to create simple C libraries that can create the
> >> necessary in-memory data objects to link against yt directly
> >>  * The Stream frontend is usable and should be used by non-Enzo codes
> >> as a prototyping mechanism
> >>  * ParaView can call yt for serial calculations, and we are working to
> >> utilize its CoProcessing facilities to analyze and visualize in
> >> parallel from running simulations
> >>
> >> So I think you should be able to add a code to in situ viz/analysis,
> >> but it will likely take a bit of work at the current time to make sure
> >> everything correctly passes.  I'd be able to help you and send you
> >> demo code if you would like.  Alternately, we intend to instrument yt
> >> to use ParaView as a data loader, which will enable the CoProcessing
> >> techniques from ParaView to be used as well.
> >>
> >> Let me know what you think,
> >>
> >> Matt
> >>
> >> On Tue, Oct 4, 2011 at 1:28 PM, Matthieu Dorier
> >> <matthieu.dorier at gmail.com> wrote:
> >> > Hello,
> >> >
> >> > I'm trying to make in-situ visualization available for a simulation,
> >> > and I'm considering different softwares for that (VisIt, YT,...).
> >> > I've never used YT, so my question is: is it possible to add
> >> > some code in a simulation (written in C++) to have some kind of
> >> > distributed rendering engine using YT, just like VisIt does with
> >> > code-instrumentation?
> >> > Thanks,
> >> >
> >> > Matthieu
> >> >
> >> > --
> >> > Matthieu Dorier
> >> > ENS Cachan, antenne de Bretagne
> >> > Département informatique et télécommunication
> >> > http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/
> >> >
> >> > _______________________________________________
> >> > yt-users mailing list
> >> > yt-users at lists.spacepope.org
> >> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >> >
> >> >
> >> _______________________________________________
> >> yt-users mailing list
> >> yt-users at lists.spacepope.org
> >> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >
> >
> >
> > --
> > Matthieu Dorier
> > ENS Cachan, antenne de Bretagne
> > Département informatique et télécommunication
> > http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/
> >
> > _______________________________________________
> > yt-users mailing list
> > yt-users at lists.spacepope.org
> > http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
> >
> >
> _______________________________________________
> yt-users mailing list
> yt-users at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-users-spacepope.org
>

-- 
Matthieu Dorier
ENS Cachan, antenne de Bretagne
Département informatique et télécommunication
http://perso.eleves.bretagne.ens-cachan.fr/~mdori307/wiki/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-users-spacepope.org/attachments/20111108/a23f361c/attachment.htm>