[Yt-dev] A fun interface for data
Cameron Hummels
chummels at astro.columbia.edu
Thu May 19 13:03:31 PDT 2011
I would like yt phds as well, and yt space shuttles.
On 5/19/11 3:58 PM, Sam Skillman wrote:
> Given the quick turnaround, I also think it'd be cool to implement yt
> cookies, which causes chocolate chip cookies to appear on my desk.
> Seriously though, this is awesome.
>
> Sam
>
> On Thu, May 19, 2011 at 1:52 PM, Matthew Turk <matthewturk at gmail.com
> <mailto:matthewturk at gmail.com>> wrote:
>
> Hi Sam,
>
> I am totally on board. I wrote up a quick script that does this -- it
> requires both the parameter file and a .yt file, but nothing else.
> Here's what the source looks like:
>
> http://paste.enzotools.org/show/1645/
>
> I tested this with the 1024 L7, for which I only have the .yt and
> parameter file, and it works. :)
>
> -Matt
>
> On Thu, May 19, 2011 at 1:45 PM, Sam Skillman
> <samskillman at gmail.com <mailto:samskillman at gmail.com>> wrote:
> > I was just thinking about this a bit, and I do think that while
> we will want
> > to roll this into reason, I could also see a use case for a very
> minimal
> > stand-alone viewer that doesn't know about anything outside of a
> .yt file.
> > I think it could be neat to set up something where you could do
> yt explore
> > DD1234.yt, which brings up all of the data objects that have
> been stored,
> > allowing you to navigate through your saved projections.
> Anyways, just
> > throwing this out there...I haven't thought it through very much.
> > A totally agree with Cameron, this is going to be super awesome.
> > Sam
> >
> > On Wed, May 18, 2011 at 2:13 PM, Cameron Hummels
> > <chummels at astro.columbia.edu
> <mailto:chummels at astro.columbia.edu>> wrote:
> >>
> >> This is a super cool idea, especially given what I saw when you
> demo'd
> >> it to me, Matt. So what you're proposing is a sort of "standalone"
> >> version of the applet that allows pan-n-scan in our plot
> windows, yes?
> >> One so that a person could just go to a remote website that had
> the data
> >> on it and be able to move around in that data interactively for
> >> presentation to an audience, or basically like a googlemaps
> interace for
> >> a few datasets?
> >>
> >> I think this would be very useful, however, we need to take
> things one
> >> step at a time, I think. Why don't we implement this in the
> existing
> >> reason for now, and see how that works. After all, reason will
> be able
> >> to deal with more than just projections of the dataset and more
> than
> >> just looking at it from one perspective, right? Correct me if I'm
> >> wrong, but what you're describing is a scaled down version of
> this which
> >> would only be able to look at the tiles for one perspective's
> >> projections? Because if we want to do more than one
> perspective, or
> >> more than one type of data (slices, projections, off-axis
> projections,
> >> etc.), then we need to store more tiles, right? Then we need
> more room
> >> for the data, and we need a means in the window of switching
> between the
> >> various views and modes, and then it becomes closer and closer
> to the
> >> original reason. So what I think we should do is first focus on
> >> implementing it with the full suite of tools that we're doing
> in reason,
> >> and THEN if things are slow for the full reason implementation,
> we could
> >> make a reason-light version that scales down the feature set to
> only
> >> pan-n-scan for one mode and one perspective.
> >>
> >> What do you think?
> >>
> >> Oh, btw, I totally want to be involved in this, because it is super
> >> awesome.
> >>
> >> Cameron
> >>
> >> On 05/18/2011 01:52 PM, Matthew Turk wrote:
> >> > Thanks, Sam!
> >> >
> >> > As a quick note, I changed all of the PNG writing to not use
> tempfile
> >> > but instead write directly to in-memory, which was a pretty
> glaring
> >> > problem waiting to rear its head.
> >> >
> >> > I was idly thinking about this yesterday, and after some
> conversations
> >> > I've had with Tom and others I think that one could imagine a
> very
> >> > cool project coming out of this. The idea would be that the
> minimum
> >> > amount of information to do an entire pan-n-scan is really
> just the
> >> > 5xN array for a given projection or slice: (px, py, pdx, pdy, z).
> >> > This is how it currently works -- it simply repixelizes that
> array for
> >> > each tile segment, and returns a PNG of that newly created
> tile. The
> >> > storage on the backend is really quite small; these 5xN
> arrays are
> >> > much more efficient than creating all the possible tiles in
> advance.
> >> > They're not free in terms of web-server memory, like the
> tiles would
> >> > be, but they're quite inexpensive. I believe that even for
> the 1024^3
> >> > L7 dataset it was on the order of a hundred megs or so.
> >> >
> >> > So imagine having a server that lived somewhere that stored a
> bunch of
> >> > these 5xN arrays in memory, along with appropriate metadata.
> Go to
> >> > the frontend of the server, select one to view, and then
> having that
> >> > dataset pop up. Although they aren't in the mapserver
> command, one
> >> > can supply annotations and even drawings with the leaflet
> library.
> >> >
> >> > (I've basically just described a number of existing services.
> Except,
> >> > ours would be based on simulation data exclusively and would
> probably
> >> > be able to scale to having many, many datasets!)
> >> >
> >> > You'd have to implement a frontend, a backend that stored these
> >> > datasets, and likely have some kind of garbage collection --
> if the
> >> > last request for a given dataset was more than N minutes ago,
> remove
> >> > it from memory. I think this could be a viable, fun project, and
> >> > would be really excellent to use as supplemental data for
> papers and
> >> > presentations -- particularly for star formation and
> cosmology, where
> >> > there are a number of points of interest within relatively high
> >> > dynamical range.
> >> >
> >> > Anyway, if someone is interested in working on this, it could
> be VERY
> >> > fun and possibly quite straightforward to implement. I will
> confess I
> >> > don't know enough about web apps, server deployment and database
> >> > backends to move forward on it, but I'd be more than happy to
> help out
> >> > with the data transport and visualization aspects, which I
> think the
> >> > mapserver already has a good start on. Coming up with money
> to run
> >> > this on, say, EC2 is probably also possible; it may even be
> reasonable
> >> > to think we could make this a project that people in the
> community
> >> > could upload to and use.
> >> >
> >> > -Matt
> >> >
> >> > On Wed, May 18, 2011 at 1:37 PM, Sam Skillman
> <samskillman at gmail.com <mailto:samskillman at gmail.com>>
> >> > wrote:
> >> >> Matt,
> >> >> This is awesome, and I'm definitely a +1 on rolling this
> into reason in
> >> >> the
> >> >> future. I'm very tempted to throw this up on a big set of
> displays,
> >> >> and up
> >> >> the window size :).
> >> >> Well done!
> >> >> Sam
> >> >> On Wed, May 18, 2011 at 1:42 AM, Matthew Turk
> <matthewturk at gmail.com <mailto:matthewturk at gmail.com>>
> >> >> wrote:
> >> >>> Thanks for the positive feedback, Stephen -- I'm pretty
> excited about
> >> >>> this, too.
> >> >>>
> >> >>> As a stopgap before it gets rolled into reason, which may
> end up being
> >> >>> sort of tricky, I have added a "mapserver" command to the
> yt command
> >> >>> line utility. If you update your installation, you can do:
> >> >>>
> >> >>> yt mapserver DD0030/DD0030
> >> >>>
> >> >>> and it'll spawn on http://127.0.0.1:8080/ , which you can
> then open
> >> >>> in your browser. There are a couple options for this
> command, too --
> >> >>> axis, field, projection, and weight, which you can see with
> --help.
> >> >>>
> >> >>> -Matt
> >> >>>
> >> >>> On Tue, May 17, 2011 at 4:15 PM, Stephen Skory <s at skory.us
> <mailto:s at skory.us>> wrote:
> >> >>>> Hi Matt,
> >> >>>>
> >> >>>>> I'd love it if people could download and try it out. If
> you have yt
> >> >>>>> installed, then you don't need anything other than this repo:
> >> >>>> I just gave it a go on a couple datasets and it worked
> pretty well!
> >> >>>> The periodicity is handled interestingly - just like
> Google Maps
> >> >>>> where
> >> >>>> you can pan left and right and get the same thing, but up
> and down
> >> >>>> cuts off the data. Really neat!
> >> >>>>
> >> >>>> --
> >> >>>> Stephen Skory
> >> >>>> s at skory.us <mailto:s at skory.us>
> >> >>>> http://stephenskory.com/
> >> >>>> 510.621.3687 <tel:510.621.3687> (google voice)
> >> >>>> _______________________________________________
> >> >>>> Yt-dev mailing list
> >> >>>> Yt-dev at lists.spacepope.org <mailto:Yt-dev at lists.spacepope.org>
> >> >>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>>>
> >> >>> _______________________________________________
> >> >>> Yt-dev mailing list
> >> >>> Yt-dev at lists.spacepope.org <mailto:Yt-dev at lists.spacepope.org>
> >> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>
> >> >> _______________________________________________
> >> >> Yt-dev mailing list
> >> >> Yt-dev at lists.spacepope.org <mailto:Yt-dev at lists.spacepope.org>
> >> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >>
> >> >>
> >> > _______________________________________________
> >> > Yt-dev mailing list
> >> > Yt-dev at lists.spacepope.org <mailto:Yt-dev at lists.spacepope.org>
> >> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >> >
> >>
> >> --
> >> Cameron Hummels
> >> PhD Candidate, Astronomy Department of Columbia University
> >> Public Outreach Director, Astronomy Department of Columbia
> University
> >> NASA IYA New York State Student Ambassador
> >> http://outreach.astro.columbia.edu
> >> PGP: 0x06F886E3
> >>
> >> _______________________________________________
> >> Yt-dev mailing list
> >> Yt-dev at lists.spacepope.org <mailto:Yt-dev at lists.spacepope.org>
> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> >
> > _______________________________________________
> > Yt-dev mailing list
> > Yt-dev at lists.spacepope.org <mailto:Yt-dev at lists.spacepope.org>
> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> >
> >
> _______________________________________________
> Yt-dev mailing list
> Yt-dev at lists.spacepope.org <mailto:Yt-dev at lists.spacepope.org>
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
>
> _______________________________________________
> Yt-dev mailing list
> Yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.spacepope.org/pipermail/yt-dev-spacepope.org/attachments/20110519/041e7c05/attachment.html>
More information about the yt-dev
mailing list