[yt-dev] Data downloader

Matthew Turk matthewturk at gmail.com
Tue Jul 24 11:01:05 PDT 2012


Normally I would be too, but this is the breakdown:

enzo_tiny_cosmology: 0.68 gb
Enzo_64 : 4.0 gb
IsolatedGalaxy: 0.5 gb
Sedov_3d: 0.2 gb
GasSloshing: 0.65 gb
GasSloshingLowRes: 2.0 gb
WindTunnel : 1.0 gb
GalaxyClusterMerger: 6.0 gb

We could get rid of a few, but part of the idea is to keep some big
ones around for people to really test out and use -- both big in
number and in size of individual outputs.

Slimming this list down could work, but I'd be somewhat opposed to
removing either Enzo_64 or GasSloshingLowRes.

-Matt


On Tue, Jul 24, 2012 at 1:55 PM, Casey W. Stark <caseywstark at gmail.com> wrote:
> I would be in favor of one tarball for simplicity. Are the example files
> that large?
>
> - Casey
>
>
> On Tue, Jul 24, 2012 at 10:48 AM, Matthew Turk <matthewturk at gmail.com>
> wrote:
>>
>> Hi John,
>>
>> Hm, that's puzzling.  Stephen, any ideas?
>>
>> After Stephen's email, I went from +0 on keeping the downloader to +1,
>> because I think having it from the command line is a much simpler
>> solution than what we had tried before, which was the
>> download-by-hand.  So let's see if we can address this, and then check
>> it in to scripts/ .
>>
>> -Matt
>>
>> On Tue, Jul 24, 2012 at 1:45 PM, John ZuHone <jzuhone at gmail.com> wrote:
>> > Hi all,
>> >
>> > What I don't like about the downloader is the directory structure it
>> > creates. At least on my machine, if I download only the sloshing dataset, I
>> > get:
>> >
>> > GasSloshing/GasSloshing/sloshing_nomag2*
>> >
>> > as the location of the files. Is there any reason why it ended up this
>> > way?
>> >
>> > John
>> >
>> > On Jul 24, 2012, at 11:26 AM, Stephen Skory wrote:
>> >
>> >> Hi Matt,
>> >>
>> >>> One question that's come up is: do we want to continue using
>> >>> download.py?  The burden on uploaders is moderately higher, in that
>> >>> the files all have to be tarred up in a particular way, and they have
>> >>> to be added to download.py, but it does provide a measure of
>> >>> robustness.  If we do this, can we move download.py into the main
>> >>> distribution, under scripts/ ?  Stephen, John, others who have used
>> >>> the script, what do you think about this?
>> >>
>> >> I will not be insulted if we do away with the download script. If a
>> >> web page of download links is easier for everyone around, that's fine
>> >> by me. The best argument I can think of for keeping it, or something
>> >> similar, it is forces some kind of uniformity so that the datasets are
>> >> in an expected layout. Also, downloading tens of data dumps one at at
>> >> time from a webpage is kind of tedious. That could be solved by having
>> >> both the separate data dumps and a big 'ol tarball of the whole thing
>> >> so people could get exactly what they want, but that doubles disk
>> >> space on someone's computer.
>> >>
>> >> But this isn't HIPPA medical data, so we can be loosie goosie if that
>> >> makes things easier for everyone!
>> >>
>> >> --
>> >> Stephen Skory
>> >> s at skory.us
>> >> http://stephenskory.com/
>> >> 510.621.3687 (google voice)
>> >> _______________________________________________
>> >> yt-dev mailing list
>> >> yt-dev at lists.spacepope.org
>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> >
>> > _______________________________________________
>> > yt-dev mailing list
>> > yt-dev at lists.spacepope.org
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>
>
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>



More information about the yt-dev mailing list