[yt-dev] Data downloader

[Stephen Skory]

Hi Matt,

> One question that's come up is: do we want to continue using
> download.py?  The burden on uploaders is moderately higher, in that
> the files all have to be tarred up in a particular way, and they have
> to be added to download.py, but it does provide a measure of
> robustness.  If we do this, can we move download.py into the main
> distribution, under scripts/ ?  Stephen, John, others who have used
> the script, what do you think about this?

I will not be insulted if we do away with the download script. If a
web page of download links is easier for everyone around, that's fine
by me. The best argument I can think of for keeping it, or something
similar, it is forces some kind of uniformity so that the datasets are
in an expected layout. Also, downloading tens of data dumps one at at
time from a webpage is kind of tedious. That could be solved by having
both the separate data dumps and a big 'ol tarball of the whole thing
so people could get exactly what they want, but that doubles disk
space on someone's computer.

But this isn't HIPPA medical data, so we can be loosie goosie if that
makes things easier for everyone!

-- 
Stephen Skory
s at skory.us
http://stephenskory.com/
510.621.3687 (google voice)