[Yt-dev] A fun interface for data

Wed May 18 11:13:01 PDT 2011

This is a super cool idea, especially given what I saw when you demo'd
it to me, Matt.  So what you're proposing is a sort of "standalone"
version of the applet that allows pan-n-scan in our plot windows, yes? 
One so that a person could just go to a remote website that had the data
on it and be able to move around in that data interactively for
presentation to an audience, or basically like a googlemaps interace for
a few datasets? 

I think this would be very useful, however, we need to take things one
step at a time, I think.  Why don't we implement this in the existing
reason for now, and see how that works.  After all, reason will be able
to deal with more than just projections of the dataset and more than
just looking at it from one perspective, right?  Correct me if I'm
wrong, but what you're describing is a scaled down version of this which
would only be able to look at the tiles for one perspective's
projections?  Because if we want to do more than one perspective, or
more than one type of data (slices, projections, off-axis projections,
etc.), then we need to store more tiles, right?  Then we need more room
for the data, and we need a means in the window of switching between the
various views and modes, and then it becomes closer and closer to the
original reason.  So what I think we should do is first focus on
implementing it with the full suite of tools that we're doing in reason,
and THEN if things are slow for the full reason implementation, we could
make a reason-light version that scales down the feature set to only
pan-n-scan for one mode and one perspective.

What do you think?

Oh, btw, I totally want to be involved in this, because it is super awesome.

Cameron

On 05/18/2011 01:52 PM, Matthew Turk wrote:
> Thanks, Sam!
>
> As a quick note, I changed all of the PNG writing to not use tempfile
> but instead write directly to in-memory, which was a pretty glaring
> problem waiting to rear its head.
>
> I was idly thinking about this yesterday, and after some conversations
> I've had with Tom and others I think that one could imagine a very
> cool project coming out of this.  The idea would be that the minimum
> amount of information to do an entire pan-n-scan is really just the
> 5xN array for a given projection or slice: (px, py, pdx, pdy, z).
> This is how it currently works -- it simply repixelizes that array for
> each tile segment, and returns a PNG of that newly created tile.  The
> storage on the backend is really quite small; these 5xN arrays are
> much more efficient than creating all the possible tiles in advance.
> They're not free in terms of web-server memory, like the tiles would
> be, but they're quite inexpensive.  I believe that even for the 1024^3
> L7 dataset it was on the order of a hundred megs or so.
>
> So imagine having a server that lived somewhere that stored a bunch of
> these 5xN arrays in memory, along with appropriate metadata.  Go to
> the frontend of the server, select one to view, and then having that
> dataset pop up.  Although they aren't in the mapserver command, one
> can supply annotations and even drawings with the leaflet library.
>
> (I've basically just described a number of existing services.  Except,
> ours would be based on simulation data exclusively and would probably
> be able to scale to having many, many datasets!)
>
> You'd have to implement a frontend, a backend that stored these
> datasets, and likely have some kind of garbage collection -- if the
> last request for a given dataset was more than N minutes ago, remove
> it from memory.  I think this could be a viable, fun project, and
> would be really excellent to use as supplemental data for papers and
> presentations -- particularly for star formation and cosmology, where
> there are a number of points of interest within relatively high
> dynamical range.
>
> Anyway, if someone is interested in working on this, it could be VERY
> fun and possibly quite straightforward to implement.  I will confess I
> don't know enough about web apps, server deployment and database
> backends to move forward on it, but I'd be more than happy to help out
> with the data transport and visualization aspects, which I think the
> mapserver already has a good start on.  Coming up with money to run
> this on, say, EC2 is probably also possible; it may even be reasonable
> to think we could make this a project that people in the community
> could upload to and use.
>
> -Matt
>
> On Wed, May 18, 2011 at 1:37 PM, Sam Skillman <samskillman at gmail.com> wrote:
>> Matt,
>> This is awesome, and I'm definitely a +1 on rolling this into reason in the
>> future.  I'm very tempted to throw this up on a big set of displays, and up
>> the window size :).
>> Well done!
>> Sam
>> On Wed, May 18, 2011 at 1:42 AM, Matthew Turk <matthewturk at gmail.com> wrote:
>>> Thanks for the positive feedback, Stephen -- I'm pretty excited about
>>> this, too.
>>>
>>> As a stopgap before it gets rolled into reason, which may end up being
>>> sort of tricky, I have added a "mapserver" command to the yt command
>>> line utility.  If you update your installation, you can do:
>>>
>>> yt mapserver DD0030/DD0030
>>>
>>> and it'll spawn on http://127.0.0.1:8080/  , which you can then open
>>> in your browser.  There are a couple options for this command, too --
>>> axis, field, projection, and weight, which you can see with --help.
>>>
>>> -Matt
>>>
>>> On Tue, May 17, 2011 at 4:15 PM, Stephen Skory <s at skory.us> wrote:
>>>> Hi Matt,
>>>>
>>>>> I'd love it if people could download and try it out.  If you have yt
>>>>> installed, then you don't need anything other than this repo:
>>>> I just gave it a go on a couple datasets and it worked pretty well!
>>>> The periodicity is handled interestingly - just like Google Maps where
>>>> you can pan left and right and get the same thing, but up and down
>>>> cuts off the data. Really neat!
>>>>
>>>> --
>>>> Stephen Skory
>>>> s at skory.us
>>>> http://stephenskory.com/
>>>> 510.621.3687 (google voice)
>>>> _______________________________________________
>>>> Yt-dev mailing list
>>>> Yt-dev at lists.spacepope.org
>>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>>>
>>> _______________________________________________
>>> Yt-dev mailing list
>>> Yt-dev at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>> _______________________________________________
>> Yt-dev mailing list
>> Yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>>
> _______________________________________________
> Yt-dev mailing list
> Yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>

-- 
Cameron Hummels
PhD Candidate, Astronomy Department of Columbia University
Public Outreach Director, Astronomy Department of Columbia University
NASA IYA New York State Student Ambassador
http://outreach.astro.columbia.edu 
PGP: 0x06F886E3