[Yt-dev] A fun interface for data

Matthew Turk matthewturk at gmail.com
Thu May 19 13:01:33 PDT 2011


Sam,

If you run "yt instinfo -u" to get the latest tip (changeset hash
4d3d3d3) and then import from yt.visualization.cookies, you should be
able to use this script to get the chocolate chip cookies:

http://paste.enzotools.org/show/1646/

Remember that you can download from the pastebin with "yt_lodgeit.py
--download=1646".

-Matt

On Thu, May 19, 2011 at 3:58 PM, Sam Skillman <samskillman at gmail.com> wrote:
> Given the quick turnaround, I also think it'd be cool to implement yt
> cookies, which causes chocolate chip cookies to appear on my desk.
>  Seriously though, this is awesome.
> Sam
>
> On Thu, May 19, 2011 at 1:52 PM, Matthew Turk <matthewturk at gmail.com> wrote:
>>
>> Hi Sam,
>>
>> I am totally on board.  I wrote up a quick script that does this -- it
>> requires both the parameter file and a .yt file, but nothing else.
>> Here's what the source looks like:
>>
>> http://paste.enzotools.org/show/1645/
>>
>> I tested this with the 1024 L7, for which I only have the .yt and
>> parameter file, and it works.  :)
>>
>> -Matt
>>
>> On Thu, May 19, 2011 at 1:45 PM, Sam Skillman <samskillman at gmail.com>
>> wrote:
>> > I was just thinking about this a bit, and I do think that while we will
>> > want
>> > to roll this into reason, I could also see a use case for a very minimal
>> > stand-alone viewer that doesn't know about anything outside of a .yt
>> > file.
>> >  I think it could be neat to set up something where you could do yt
>> > explore
>> > DD1234.yt, which brings up all of the data objects that have been
>> > stored,
>> > allowing you to navigate through your saved projections.  Anyways, just
>> > throwing this out there...I haven't thought it through very much.
>> > A totally agree with Cameron, this is going to be super awesome.
>> > Sam
>> >
>> > On Wed, May 18, 2011 at 2:13 PM, Cameron Hummels
>> > <chummels at astro.columbia.edu> wrote:
>> >>
>> >> This is a super cool idea, especially given what I saw when you demo'd
>> >> it to me, Matt.  So what you're proposing is a sort of "standalone"
>> >> version of the applet that allows pan-n-scan in our plot windows, yes?
>> >> One so that a person could just go to a remote website that had the
>> >> data
>> >> on it and be able to move around in that data interactively for
>> >> presentation to an audience, or basically like a googlemaps interace
>> >> for
>> >> a few datasets?
>> >>
>> >> I think this would be very useful, however, we need to take things one
>> >> step at a time, I think.  Why don't we implement this in the existing
>> >> reason for now, and see how that works.  After all, reason will be able
>> >> to deal with more than just projections of the dataset and more than
>> >> just looking at it from one perspective, right?  Correct me if I'm
>> >> wrong, but what you're describing is a scaled down version of this
>> >> which
>> >> would only be able to look at the tiles for one perspective's
>> >> projections?  Because if we want to do more than one perspective, or
>> >> more than one type of data (slices, projections, off-axis projections,
>> >> etc.), then we need to store more tiles, right?  Then we need more room
>> >> for the data, and we need a means in the window of switching between
>> >> the
>> >> various views and modes, and then it becomes closer and closer to the
>> >> original reason.  So what I think we should do is first focus on
>> >> implementing it with the full suite of tools that we're doing in
>> >> reason,
>> >> and THEN if things are slow for the full reason implementation, we
>> >> could
>> >> make a reason-light version that scales down the feature set to only
>> >> pan-n-scan for one mode and one perspective.
>> >>
>> >> What do you think?
>> >>
>> >> Oh, btw, I totally want to be involved in this, because it is super
>> >> awesome.
>> >>
>> >> Cameron
>> >>
>> >> On 05/18/2011 01:52 PM, Matthew Turk wrote:
>> >> > Thanks, Sam!
>> >> >
>> >> > As a quick note, I changed all of the PNG writing to not use tempfile
>> >> > but instead write directly to in-memory, which was a pretty glaring
>> >> > problem waiting to rear its head.
>> >> >
>> >> > I was idly thinking about this yesterday, and after some
>> >> > conversations
>> >> > I've had with Tom and others I think that one could imagine a very
>> >> > cool project coming out of this.  The idea would be that the minimum
>> >> > amount of information to do an entire pan-n-scan is really just the
>> >> > 5xN array for a given projection or slice: (px, py, pdx, pdy, z).
>> >> > This is how it currently works -- it simply repixelizes that array
>> >> > for
>> >> > each tile segment, and returns a PNG of that newly created tile.  The
>> >> > storage on the backend is really quite small; these 5xN arrays are
>> >> > much more efficient than creating all the possible tiles in advance.
>> >> > They're not free in terms of web-server memory, like the tiles would
>> >> > be, but they're quite inexpensive.  I believe that even for the
>> >> > 1024^3
>> >> > L7 dataset it was on the order of a hundred megs or so.
>> >> >
>> >> > So imagine having a server that lived somewhere that stored a bunch
>> >> > of
>> >> > these 5xN arrays in memory, along with appropriate metadata.  Go to
>> >> > the frontend of the server, select one to view, and then having that
>> >> > dataset pop up.  Although they aren't in the mapserver command, one
>> >> > can supply annotations and even drawings with the leaflet library.
>> >> >
>> >> > (I've basically just described a number of existing services.
>> >> >  Except,
>> >> > ours would be based on simulation data exclusively and would probably
>> >> > be able to scale to having many, many datasets!)
>> >> >
>> >> > You'd have to implement a frontend, a backend that stored these
>> >> > datasets, and likely have some kind of garbage collection -- if the
>> >> > last request for a given dataset was more than N minutes ago, remove
>> >> > it from memory.  I think this could be a viable, fun project, and
>> >> > would be really excellent to use as supplemental data for papers and
>> >> > presentations -- particularly for star formation and cosmology, where
>> >> > there are a number of points of interest within relatively high
>> >> > dynamical range.
>> >> >
>> >> > Anyway, if someone is interested in working on this, it could be VERY
>> >> > fun and possibly quite straightforward to implement.  I will confess
>> >> > I
>> >> > don't know enough about web apps, server deployment and database
>> >> > backends to move forward on it, but I'd be more than happy to help
>> >> > out
>> >> > with the data transport and visualization aspects, which I think the
>> >> > mapserver already has a good start on.  Coming up with money to run
>> >> > this on, say, EC2 is probably also possible; it may even be
>> >> > reasonable
>> >> > to think we could make this a project that people in the community
>> >> > could upload to and use.
>> >> >
>> >> > -Matt
>> >> >
>> >> > On Wed, May 18, 2011 at 1:37 PM, Sam Skillman <samskillman at gmail.com>
>> >> > wrote:
>> >> >> Matt,
>> >> >> This is awesome, and I'm definitely a +1 on rolling this into reason
>> >> >> in
>> >> >> the
>> >> >> future.  I'm very tempted to throw this up on a big set of displays,
>> >> >> and up
>> >> >> the window size :).
>> >> >> Well done!
>> >> >> Sam
>> >> >> On Wed, May 18, 2011 at 1:42 AM, Matthew Turk
>> >> >> <matthewturk at gmail.com>
>> >> >> wrote:
>> >> >>> Thanks for the positive feedback, Stephen -- I'm pretty excited
>> >> >>> about
>> >> >>> this, too.
>> >> >>>
>> >> >>> As a stopgap before it gets rolled into reason, which may end up
>> >> >>> being
>> >> >>> sort of tricky, I have added a "mapserver" command to the yt
>> >> >>> command
>> >> >>> line utility.  If you update your installation, you can do:
>> >> >>>
>> >> >>> yt mapserver DD0030/DD0030
>> >> >>>
>> >> >>> and it'll spawn on http://127.0.0.1:8080/  , which you can then
>> >> >>> open
>> >> >>> in your browser.  There are a couple options for this command, too
>> >> >>> --
>> >> >>> axis, field, projection, and weight, which you can see with --help.
>> >> >>>
>> >> >>> -Matt
>> >> >>>
>> >> >>> On Tue, May 17, 2011 at 4:15 PM, Stephen Skory <s at skory.us> wrote:
>> >> >>>> Hi Matt,
>> >> >>>>
>> >> >>>>> I'd love it if people could download and try it out.  If you have
>> >> >>>>> yt
>> >> >>>>> installed, then you don't need anything other than this repo:
>> >> >>>> I just gave it a go on a couple datasets and it worked pretty
>> >> >>>> well!
>> >> >>>> The periodicity is handled interestingly - just like Google Maps
>> >> >>>> where
>> >> >>>> you can pan left and right and get the same thing, but up and down
>> >> >>>> cuts off the data. Really neat!
>> >> >>>>
>> >> >>>> --
>> >> >>>> Stephen Skory
>> >> >>>> s at skory.us
>> >> >>>> http://stephenskory.com/
>> >> >>>> 510.621.3687 (google voice)
>> >> >>>> _______________________________________________
>> >> >>>> Yt-dev mailing list
>> >> >>>> Yt-dev at lists.spacepope.org
>> >> >>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >> >>>>
>> >> >>> _______________________________________________
>> >> >>> Yt-dev mailing list
>> >> >>> Yt-dev at lists.spacepope.org
>> >> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >> >>
>> >> >> _______________________________________________
>> >> >> Yt-dev mailing list
>> >> >> Yt-dev at lists.spacepope.org
>> >> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >> >>
>> >> >>
>> >> > _______________________________________________
>> >> > Yt-dev mailing list
>> >> > Yt-dev at lists.spacepope.org
>> >> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >> >
>> >>
>> >> --
>> >> Cameron Hummels
>> >> PhD Candidate, Astronomy Department of Columbia University
>> >> Public Outreach Director, Astronomy Department of Columbia University
>> >> NASA IYA New York State Student Ambassador
>> >> http://outreach.astro.columbia.edu
>> >> PGP: 0x06F886E3
>> >>
>> >> _______________________________________________
>> >> Yt-dev mailing list
>> >> Yt-dev at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> >
>> > _______________________________________________
>> > Yt-dev mailing list
>> > Yt-dev at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> >
>> _______________________________________________
>> Yt-dev mailing list
>> Yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
> _______________________________________________
> Yt-dev mailing list
> Yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>



More information about the yt-dev mailing list