[Yt-dev] Request for Review: bootstrap script; also, new 'pasteboards'

Matthew Turk matthewturk at gmail.com
Wed Mar 23 11:28:40 PDT 2011 

Hi Chris,

Ah, blast, you're right -- it's a dependency.  "pip install lxml"
should fix it.  I will add it to the bootstrap (not install) script.

-Matt

On Wed, Mar 23, 2011 at 12:32 PM,  <chris.m.malone at gmail.com> wrote:
> Hi Matt,
>
> I took a look at the docs you coded up, and indeed there are some
> interesting features with BB. In particular, I thought the `bbforks` command
> sounded pretty cool - however, it appears I am missing the lxml.html
> package, which is needed to get this to work:
>
> ---------------------------------------------------------------------------------------
> [cmalone at xrb yt]$ hg bbforks
> using 'ChrisMalone/yt-hg' as repo name
> getting descendants list
> abort: lxml.html is (currently) needed to run bbforks
> ---------------------------------------------------------------------------------------
>
> I thought maybe it was perhaps a result of my BB repo not having any forks,
> so I tried yours:
>
> ---------------------------------------------------------------------------------------
> [cmalone at xrb yt]$ hg bbforks -n MatthewTurk/yt
> getting descendants list
> abort: lxml.html is (currently) needed to run bbforks
> ---------------------------------------------------------------------------------------
>
> Is this missing in the install, or is it something on my end?
>
> Chris
>
> On Mar 23, 2011 10:31am, Matthew Turk <matthewturk at gmail.com> wrote:
>> Hi John, and everyone else,
>>
>>
>>
>> Thanks!  Plane flights are great for documentation (writing docs
>>
>> doesn't have to hit the disk or grind the processor) so I started
>>
>> writing up what it does:
>>
>>
>>
>> http://hg.enzotools.org/yt-doc/changeset/e8f532648a50
>>
>>
>>
>> I also went ahead, with the couple votes, and pushed it to the main
>> repository.
>>
>>
>>
>> Thanks everybody,
>>
>>
>>
>> Matt
>>
>>
>>
>> On Wed, Mar 23, 2011 at 9:07 AM, John Wise jwise at astro.princeton.edu>
>> wrote:
>>
>> > Hi Matt,
>>
>> >
>>
>> > I'm a little late to the game since other less interesting things took
>> > priority.  I tested out the bootstrap script, and everything went smoothly!
>> >  I voted "0" before because I didn't really see the bigger picture of what
>> > you were talking about, but now I'm convinced.
>>
>> >
>>
>> > +1.
>>
>> >
>>
>> > John
>>
>> >
>>
>> > On 21 Mar 2011, at 16:14, Matthew Turk wrote:
>>
>> >
>>
>> >> Hi Chris,
>>
>> >>
>>
>> >> Ah, yes, you're totally right.  I'll add an inventory-regeneration
>>
>> >> command, which looks at the current manifest of the repo and then
>>
>> >> regenerates inventory.json to match it.  (This allows you to, for
>>
>> >> instance, remove pastes from the current display, add supplemental
>>
>> >> images, etc etc.)  Pushing, like you say, also fixes the display on
>>
>> >> the website.
>>
>> >>
>>
>> >> -Matt
>>
>> >>
>>
>> >> On Mon, Mar 21, 2011 at 1:11 PM,  chris.m.malone at gmail.com> wrote:
>>
>> >>> I should chime in and say that my pasteboard page was not setup after
>> >>> the
>>
>> >>> bootstrapp'ing - the page just 404'ed. To fix this I just did a test
>> >>> commit
>>
>> >>> to the pasteboard
>>
>> >>>
>>
>> >>> yt pasteboard --desc="This is just a test" test.py
>>
>> >>>
>>
>> >>> and it worked fine. So, it _is_ set up with BB, as far as I can tell,
>> >>> but
>>
>> >>> the page just shows up as 404 until something is checked in, or
>> >>> following
>>
>> >>> Briton's elegant solution of just doing a 'hg push'
>>
>> >>>
>>
>> >>> Chris
>>
>> >>> On Mar 21, 2011 4:03pm, Britton Smith brittonsmith at gmail.com> wrote:
>>
>> >>>> I had a similar experience to Chris's.  I gave the script my
>> >>>> bitbucket
>>
>> >>>> username, but on the first try it wanted to use my system username,
>> >>>> which is
>>
>> >>>> also not the same.  Just like for Chris, it got it right the second
>> >>>> time.
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> As one of original nay-sayers on this, I have to say I liked
>> >>>> everything
>>
>> >>>> about how this worked.  I especially think the pasteboard is really
>> >>>> great.
>>
>> >>>> The one thing that wasn't very clear to me was that I had to do an
>> >>>> 'hg push'
>>
>> >>>> from within my pasteboard repo to get the page set up on bitbucket.
>> >>>>  Could a
>>
>> >>>> note of this be made in the script after this step is done?
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> Since I'm wrong so often, it's quite easy for me to admit that I was
>> >>>> wrong
>>
>> >>>> about this one.
>>
>> >>>>
>>
>> >>>> Nice job, Matt.
>>
>> >>>>
>>
>> >>>> Britton
>>
>> >>>>
>>
>> >>>> On Mon, Mar 21, 2011 at 2:49 PM, Chris Malone
>> >>>> chris.m.malone at gmail.com>
>>
>> >>>> wrote:
>>
>> >>>>
>>
>> >>>> Hi Matt,
>>
>> >>>>
>>
>> >>>> I pulled from your repo and ran the
>>
>> >>>>
>>
>> >>>> yt bootstrap_dev
>>
>> >>>>
>>
>> >>>> method.  Everything was going fine - it checked out yt-supplemental,
>>
>> >>>> looked at my .hgrc and added my BB username, etc - until it wanted to
>> >>>> set up
>>
>> >>>> my pasteboard.  It seems like when it made the hgbb._bb_apicall, it
>> >>>> used my
>>
>> >>>> system user name, which is different from my BB username:
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>> >>>> -------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> >>>> I am now going to create the repository:
>>
>> >>>>      chrismalone.bitbucket.org
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> on BitBucket.org.  This will set up the domain
>>
>> >>>>      http://chrismalone.bitbucket.org
>>
>> >>>> which will point to the current contents of the repo.
>>
>> >>>>
>>
>> >>>> Press enter to go on, Ctrl-C to exit.
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> using system user 'cmalone' as usernamehttp authorization required
>>
>> >>>> realm: None
>>
>> >>>> user: cmalone
>>
>> >>>> password:
>>
>> >>>>
>>
>> >>>>
>> >>>> -------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> Obviously this failed; however, when I ran this a second time it
>> >>>> picked up
>>
>> >>>> my correct BB username:
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>> >>>> -------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> I am now going to create the repository:
>>
>> >>>>      chrismalone.bitbucket.org
>>
>> >>>> on BitBucket.org.  This will set up the domain
>>
>> >>>>      http://chrismalone.bitbucket.org
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> which will point to the current contents of the repo.
>>
>> >>>>
>>
>> >>>> Press enter to go on, Ctrl-C to exit.
>>
>> >>>> http authorization required
>>
>> >>>> realm: None
>>
>> >>>> user: ChrisMalone
>>
>> >>>> password:
>>
>> >>>>
>> >>>> -------------------------------------------------------------------------------------------------------------------------------------------------
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> and then it worked fine.  I can now use the pasteboard facilities,
>> >>>> which
>>
>> >>>> are pretty cool!
>>
>> >>>>
>>
>> >>>> I think for the bootstrap utility to be a bit more user-friendly, you
>>
>> >>>> might want to add a bit more description about why the user cares
>> >>>> about the
>>
>> >>>> various components that are about to be installed  For example, the
>> >>>> user
>>
>> >>>> probably doesn't know that the yt-supplemental repo is where all the
>> >>>> hgbb or
>>
>> >>>> pasteboard stuff is contained or why they should want a BB account if
>> >>>> they
>>
>> >>>> don't have one already.  This might be too much information to have
>> >>>> in the
>>
>> >>>> bootstrap script itself, so perhaps provide a link to a page where
>> >>>> these
>>
>> >>>> things are mentioned.
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> Anyhow, aside from the above (minor) points, I really like the
>> >>>> bootstrap
>>
>> >>>> utility.  You've gone out of your way to make sure nothing was done
>> >>>> without
>>
>> >>>> the user's permission.
>>
>> >>>>
>>
>> >>>> +1
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> Chris
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> On Mon, Mar 21, 2011 at 12:34 AM, Matthew Turk matthewturk at gmail.com>
>>
>> >>>> wrote:
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> Hi all,
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> Last week we had a conversation about a bootstrap script, where the
>>
>> >>>>
>>
>> >>>> feel of the room was either lukewarm or positive.  Mostly people
>>
>> >>>>
>>
>> >>>> seemed to think it was ill-motivated or would take too many
>> >>>> liberties.
>>
>> >>>>
>>
>> >>>>  Over the last week, in the evenings etc I've created a bootstrap
>>
>> >>>>
>>
>> >>>> script for development, which overall I think is pretty
>>
>> >>>>
>>
>> >>>> well-motivated.  The script accomplishes the following things, each
>> >>>> of
>>
>> >>>>
>>
>> >>>> which requests for confirmation before going on.  It uses a modified
>>
>> >>>>
>>
>> >>>> iniparse library so that any modifications to any ini files are made
>>
>> >>>>
>>
>> >>>> in a non-destructive manner.  It accomplishes these tasks:
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> 1) Ensure username is set up in ~/.hgrc
>>
>> >>>>
>>
>> >>>> 2) Ensure that the cedit and hgbb are enabled
>>
>> >>>>
>>
>> >>>> 3) Asks for and if needed creates a username on BitBucket
>>
>> >>>>
>>
>> >>>> 4) Sets up hgbb
>>
>> >>>>
>>
>> >>>> 5) Creates a pasteboard repository
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> The diff is here:
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> https://bitbucket.org/yt_analysis/yt/compare/MatthewTurk/yt..default
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> You can pull from my repo:
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> hg pull https://bitbucket.org/MatthewTurk/yt/
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> (But note the comment below about yt-supplemental)
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> There are a couple things in this:
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>  * I wrote up something called a "pasteboard" this last week for more
>>
>> >>>>
>>
>> >>>> permanent mini-pastes of info.  This uses mercurial to create/destroy
>>
>> >>>>
>>
>> >>>> items in it.  You can see some examples here:
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> http://matthewturk.bitbucket.org/
>>
>> >>>>
>>
>> >>>> http://jsoishi.bitbucket.org/
>>
>> >>>>
>>
>> >>>> http://samskillman.bitbucket.org/
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> This was born out of the SFLC thing that happened on the mailing
>> >>>> list,
>>
>> >>>>
>>
>> >>>> when Sam emailed his script.  I thought, wouldn't it be nice if we
>>
>> >>>>
>>
>> >>>> were older, and we had a pasteboard that was persistent, with
>>
>> >>>>
>>
>> >>>> descriptions, which we could programmatically upload/download from.
>>
>> >>>>
>>
>> >>>> This adds the new commands:
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> yt pasteboard --desc="Something" some_file.py
>>
>> >>>>
>>
>> >>>> yt pastegrab PASTE_ID USERNAME
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> This automatically tosses it up.  You get embed codes and some
>>
>> >>>>
>>
>> >>>> highlights, but it's just easier to keep these things around than the
>>
>> >>>>
>>
>> >>>> pastebin.
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>  * The idea of enabling hgbb, cedit and ensuring a bitbucket user is
>>
>> >>>>
>>
>> >>>> because I am of the opinion we should make it easier to fork the
>>
>> >>>>
>>
>> >>>> repository to make changes.  If those three are enabled, I believe a
>>
>> >>>>
>>
>> >>>> command-line forking, editing of sources and so on, are all within
>> >>>> our
>>
>> >>>>
>>
>> >>>> reach.
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>  * This bootstrap script requires that the yt-supplemental repo be
>>
>> >>>>
>>
>> >>>> checked out in ${YT_DEST}/src/ .  This repo is actually kind of cool,
>>
>> >>>>
>>
>> >>>> in that it is a subrepo setup, and it pulls in a couple external
>>
>> >>>>
>>
>> >>>> libraries that we can maintain.  Right now it grabs the docs, the
>>
>> >>>>
>>
>> >>>> cookbook, and the extensions mentioned above.  This repo is now
>> >>>> pulled
>>
>> >>>>
>>
>> >>>> in the current install_script.sh.
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> Anyway, I am submitting this to the list to get some review.  There
>>
>> >>>>
>>
>> >>>> may be errors, but I have tested it quite a bit.  It also informs the
>>
>> >>>>
>>
>> >>>> user of what it's about to do and requires the user hit enter.  At
>> >>>> one
>>
>> >>>>
>>
>> >>>> point it even asks for a yes or no.
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> The reason I am interested in this is that I think the idea of
>>
>> >>>>
>>
>> >>>> bootstrapping people into a development environment is, generally, a
>>
>> >>>>
>>
>> >>>> good idea.  Particularly because some things -- like the pasteboard,
>>
>> >>>>
>>
>> >>>> like forking, like contributing changes -- should be made really easy
>>
>> >>>>
>>
>> >>>> and have the potential for a very high payoff.  I hope I've motivated
>>
>> >>>>
>>
>> >>>> things a bit better.
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> I was hoping that before I merged it I could get someone else to read
>>
>> >>>>
>>
>> >>>> it over, maybe a couple people even to test it, and to get a good
>> >>>> feel
>>
>> >>>>
>>
>> >>>> for whether this is worthwhile or whether I should pull these changes
>>
>> >>>>
>>
>> >>>> and just ignore the idea.  What do people think?
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> Thanks,
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> Matt
>>
>> >>>>
>>
>> >>>> _______________________________________________
>>
>> >>>>
>>
>> >>>> Yt-dev mailing list
>>
>> >>>>
>>
>> >>>> Yt-dev at lists.spacepope.org
>>
>> >>>>
>>
>> >>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>> _______________________________________________
>>
>> >>>>
>>
>> >>>> Yt-dev mailing list
>>
>> >>>>
>>
>> >>>> Yt-dev at lists.spacepope.org
>>
>> >>>>
>>
>> >>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>>>
>>
>> >>> _______________________________________________
>>
>> >>> Yt-dev mailing list
>>
>> >>> Yt-dev at lists.spacepope.org
>>
>> >>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>> >>>
>>
>> >>>
>>
>> >> _______________________________________________
>>
>> >> Yt-dev mailing list
>>
>> >> Yt-dev at lists.spacepope.org
>>
>> >> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>> >
>>
>> > _______________________________________________
>>
>> > Yt-dev mailing list
>>
>> > Yt-dev at lists.spacepope.org
>>
>> > http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>> >
>>
>> _______________________________________________
>>
>> Yt-dev mailing list
>>
>> Yt-dev at lists.spacepope.org
>>
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
> _______________________________________________
> Yt-dev mailing list
> Yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>
>

More information about the yt-dev mailing list