[yt-svn] commit/yt: jzuhone: Merged in ngoldbaum/yt (pull request #1717)
commits-noreply at bitbucket.org
commits-noreply at bitbucket.org
Mon Oct 5 11:44:03 PDT 2015
1 new commit in yt:
https://bitbucket.org/yt_analysis/yt/commits/2e98bc17afff/
Changeset: 2e98bc17afff
Branch: yt
User: jzuhone
Date: 2015-10-05 18:43:51+00:00
Summary: Merged in ngoldbaum/yt (pull request #1717)
Add pr backport script
Affected #: 6 files
diff -r 2f95674671178a4868f4f6aeff56b041127d189e -r 2e98bc17afff89f7b63aa763b45a82bb8b9fac97 MANIFEST.in
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -3,6 +3,7 @@
include yt/visualization/mapserver/html/leaflet/*.css
include yt/visualization/mapserver/html/leaflet/*.js
include yt/visualization/mapserver/html/leaflet/images/*.png
+exclude scripts/pr_backport.py
recursive-include yt *.py *.pyx *.pxd *.h README* *.txt LICENSE* *.cu
recursive-include doc *.rst *.txt *.py *.ipynb *.png *.jpg *.css *.html
recursive-include doc *.h *.c *.sh *.svgz *.pdf *.svg *.pyx
diff -r 2f95674671178a4868f4f6aeff56b041127d189e -r 2e98bc17afff89f7b63aa763b45a82bb8b9fac97 doc/source/developing/index.rst
--- a/doc/source/developing/index.rst
+++ b/doc/source/developing/index.rst
@@ -21,6 +21,7 @@
building_the_docs
testing
debugdrive
+ releasing
creating_datatypes
creating_derived_fields
creating_derived_quantities
diff -r 2f95674671178a4868f4f6aeff56b041127d189e -r 2e98bc17afff89f7b63aa763b45a82bb8b9fac97 doc/source/developing/releasing.rst
--- /dev/null
+++ b/doc/source/developing/releasing.rst
@@ -0,0 +1,208 @@
+How to Do a Release
+-------------------
+
+Periodically, the yt development community issues new releases. Since yt follows
+`semantic versioning <http://semver.org/>`_, the type of release can be read off
+from the version number used. Version numbers should follow the scheme
+``MAJOR.MINOR.PATCH``. There are three kinds of possible releases:
+
+* Bugfix releases
+
+ These releases are regularly scheduled and will optimally happen approximately
+ once a month. These releases should contain only fixes for bugs discovered in
+ earlier releases and should not contain new features or API changes. Bugfix
+ releases should increment the ``PATCH`` version number. Bugfix releases should
+ *not* be generated by merging from the ``yt`` branch, instead bugfix pull
+ requests should be manually backported using the PR backport script, described
+ below. Version ``3.2.2`` is a bugfix release.
+
+* Minor releases
+
+ These releases happen when new features are deemed ready to be merged into the
+ ``stable`` branch and should not happen on a regular schedule. Minor releases
+ can also include fixes for bugs if the fix is determined to be too invasive
+ for a bugfix release. Minor releases should *not* inlucde
+ backwards-incompatible changes and should not change APIs. If an API change
+ is deemed to be necessary, the old API should continue to function but might
+ trigger deprecation warnings. Minor releases should happen by merging the
+ ``yt`` branch into the ``stable`` branch. Minor releases should increment the
+ ``MINOR`` version number and reset the ``PATCH`` version number to zero.
+ Version ``3.3.0`` is a minor release.
+
+* Major releases
+
+ These releases happen when the development community decides to make major
+ backwards-incompatible changes. In principle a major version release could
+ include arbitrary changes to the library. Major version releases should only
+ happen after extensive discussion and vetting among the developer and user
+ community. Like minor releases, a major release should happen by merging the
+ ``yt`` branch into the ``stable`` branch. Major releases should increment the
+ ``MAJOR`` version number and reset the ``MINOR`` and ``PATCH`` version numbers
+ to zero. If it ever happens, version ``4.0.0`` will be a major release.
+
+The job of doing a release differs depending on the kind of release. Below, we
+describe the necessary steps for each kind of release in detail.
+
+Doing a Bugfix Release
+~~~~~~~~~~~~~~~~~~~~~~
+
+As described above, bugfix releases are regularly scheduled updates for minor
+releases to ensure fixes for bugs make their way out to users in a timely
+manner. Since bugfix releases should not include new features, we do not issue
+bugfix releases by simply merging from the development ``yt`` branch into the
+``stable`` branch. Instead, we make use of the ``pr_backport.py`` script to
+manually cherry-pick bugfixes from the from ``yt`` branch onto the ``stable``
+branch.
+
+The backport script issues interactive prompts to backport individual pull
+requests to the ``stable`` branch in a temporary clone of the main yt mercurial
+repository on bitbucket. The script is written this way to to avoid editing
+history in a clone of the repository that a developer uses for day-to-day work
+and to avoid mixing work-in-progress changes with changes that have made their
+way to the "canonical" yt repository on bitbucket.
+
+Rather than automatically manipulating the temporary repository by scripting
+mercurial commands using ``python-hglib``, the script must be "operated" by a
+human who is ready to think carefully about what the script is telling them
+to do. Most operations will merely require copy/pasting a suggested mercurial
+command. However, some changes will require manual backporting.
+
+To run the backport script, first open two terminal windows. The first window
+will be used to run the backport script. The second terminal will be used to
+manipulate a temporary clone of the yt mercurial repository. In the first
+window, navigate to the ``scripts`` directory at the root of the yt repository
+and run the backport script,
+
+.. code-block:: bash
+
+ $ cd $YT_HG/scripts
+ $ python pr_backport.py
+
+You will then need to wait for about a minute (depending on the speed of your
+internet connection and bitbucket's servers) while the script makes a clone of
+the main yt repository and then gathers information about pull requests that
+have been merged since the last tagged release. Once this step finishes, you
+will be prompted to navigate to the temporary folder in a new separate terminal
+session. Do so, and then hit the enter key in the original terminal session.
+
+For each pull request in the set of pull requests that were merged since the
+last tagged release that were pointed at the "main" line of development
+(e.g. not the ``experimental`` bookmark), you will be prompted by the script
+with the PR number, title, description, and a suggested mercurial
+command to use to backport the pull request. If the pull request consists of a
+single changeset, you will be prompted to use ``hg graft``. If it contains more
+than one changeset, you will be prompted to use ``hg rebase``. Note that
+``rebase`` is an optional extension for mercurial that is not turned on by
+default. To enable it, add a section like the following in your ``.hgrc`` file:
+
+.. code-block:: none
+
+ [extensions]
+ rebase=
+
+Since ``rebase`` is bundled with core mercurial, you do not need to specify a
+path to the rebase extension, just say ``rebase=`` and mercurial will find the
+version of ``rebase`` bundled with mercurial. Note also that mercurial does not
+automatically update to the tip of the rebased head after executing ``hg
+rebase`` so you will need to manually issue ``hg update stable`` to move your
+working directory to the new head of the stable branch. The backport script
+should prompt you with a suggestion to update as well.
+
+If the pull request contains merge commits, you must take care to *not* backport
+commits that merge with the main line of development on the ``yt`` branch. Doing
+so may bring unrelated changes, including new features, into a bugfix
+release. If the pull request you'd like to backport contains merge commits, the
+backport script should warn you to be extra careful.
+
+Once you've finished backporting, the script will let you know that you are done
+and warn you to push your work. The temporary repository you have been working
+with will be deleted as soon as the script exits, so take care to push your work
+on the ``stable`` branch to your fork on bitbucket. Once you've pushed to your
+fork, you will be able to issue a pull request containing the backported fixes
+just like any other yt pull request.
+
+Doing a Minor or Major Release
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is much simpler than a bugfix release. All that needs to happen is the
+``yt`` branch must get merged into the ``stable`` branch, and any conflicts that
+happen must be resolved, almost certainly in favor of the yt branch. This can
+happen either using a merge tool like ``vimdiff`` and ``kdiff3`` or by telling
+mercurial to write merge markers. If you prefer merge markers, the following
+configuration options should be turned on in your ``hgrc`` to get more detail
+during the merge:
+
+.. code-block:: none
+
+ [ui]
+ merge = internal:merge3
+ mergemarkers = detailed
+
+The first option tells mercurial to write merge markers that show the state of
+the conflicted region of the code on both sides of the merge as well as the
+"base" most recent common ancestor changeset. The second option tells mercurial
+to add extra information about the code near the merge markers.
+
+
+Incrementing Version Numbers and Tagging a Release
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Before creating the tag for the release, you must increment the version numbers
+that are hard-coded in a few files in the yt source so that version metadata
+for the code is generated correctly. This includes things like ``yt.__version__``
+and the version that gets read by the Python Package Index (PyPI) infrastructure.
+
+The paths relative to the root of the repository for the three files that need
+to be edited are:
+
+* ``doc/source/conf.py``
+
+ The ``version`` and ``release`` variables need to be updated.
+
+* ``setup.py``
+
+ The ``VERSION`` variable needs to be updated
+
+* ``yt/__init__.py``
+
+ The ``__version__`` variable must be updated.
+
+Once these files have been updated, commit these updates. This is the commit we
+will tag for the release.
+
+To actually create the tag, issue the following command:
+
+.. code-block:: bash
+
+ hg tag <tag-name>
+
+Where ``<tag-name>`` follows the project's naming scheme for tags
+(e.g. ``yt-3.2.1``). Commit the tag, and you should be ready to upload the
+release to pypi.
+
+If you are doing a minor or major version number release, you will also need to
+update back to the development branch and update the development version numbers
+in the same files.
+
+
+Uploading to PyPI
+~~~~~~~~~~~~~~~~~
+
+To actually upload the release to the Python Package Index, you just need to
+issue the following command:
+
+.. code-block:: bash
+
+ python setup.py sdist upload -r https://pypi.python.org/pypi
+
+You will be prompted for your PyPI credentials and then the package should
+upload. Note that for this to complete successfully, you will need an account on
+PyPI and that account will need to be registered as an "owner" of the yt
+package. Right now there are three owners: Matt Turk, Britton Smith, and Nathan
+Goldbaum.
+
+After the release is uploaded to PyPI, you should send out an announcement
+e-mail to the yt mailing lists as well as other possibly interested mailing
+lists for all but bugfix releases. In addition, you should contact John ZuHone
+about uploading binary wheels to PyPI for Windows and OS X users and contact
+Nathan Goldbaum about getting the Anaconda packages updated.
diff -r 2f95674671178a4868f4f6aeff56b041127d189e -r 2e98bc17afff89f7b63aa763b45a82bb8b9fac97 scripts/pr_backport.py
--- /dev/null
+++ b/scripts/pr_backport.py
@@ -0,0 +1,311 @@
+import hglib
+import requests
+import shutil
+import tempfile
+
+from datetime import datetime
+from distutils.version import LooseVersion
+from time import strptime, mktime
+
+MERGED_PR_ENDPOINT = ("http://bitbucket.org/api/2.0/repositories/yt_analysis/"
+ "yt/pullrequests/?state=MERGED")
+
+YT_REPO = "https://bitbucket.org/yt_analysis/yt"
+
+
+def clone_new_repo(source=None):
+ """Clones a new copy of yt_analysis/yt and returns a path to it"""
+ path = tempfile.mkdtemp()
+ dest_repo_path = path+'/yt-backport'
+ if source is None:
+ source = YT_REPO
+ hglib.clone(source=source, dest=dest_repo_path)
+ with hglib.open(dest_repo_path) as client:
+ # Changesets that are on the yt branch but aren't topological ancestors
+ # of whichever changeset the experimental bookmark is pointing at
+ client.update('heads(branch(yt) - ::bookmark(experimental))')
+ return dest_repo_path
+
+
+def get_first_commit_after_last_major_release(repo_path):
+ """Returns the SHA1 hash of the first commit to the yt branch that wasn't
+ included in the last tagged release.
+ """
+ with hglib.open(repo_path) as client:
+ tags = client.log("reverse(tag())")
+ tags = sorted([LooseVersion(t[2]) for t in tags])
+ for t in tags[::-1]:
+ if t.version[0:2] != ['yt', '-']:
+ continue
+ if len(t.version) == 4 or t.version[4] == 0:
+ last_major_tag = t
+ break
+ last_before_release = client.log(
+ "last(ancestors(%s) and branch(yt))" % str(last_major_tag))
+ first_after_release = client.log(
+ "first(descendants(%s) and branch(yt) and not %s)"
+ % (last_before_release[0][1], last_before_release[0][1]))
+ return str(first_after_release[0][1][:12])
+
+
+def get_branch_tip(repo_path, branch, exclude=None):
+ """Returns the SHA1 hash of the most recent commit on the given branch"""
+ revset = "head() and branch(%s)" % branch
+ if exclude is not None:
+ revset += "and not %s" % exclude
+ with hglib.open(repo_path) as client:
+ change = client.log(revset)[0][1][:12]
+ return change
+
+
+def get_lineage_between_release_and_tip(repo_path, first, last):
+ """Returns the lineage of changesets that were at one point the public tip"""
+ with hglib.open(repo_path) as client:
+ lineage = client.log("'%s'::'%s' and p1('%s'::'%s') + '%s'"
+ % (first, last, first, last, last))
+ return lineage
+
+
+def get_pull_requests_since_last_release(repo_path):
+ """Returns a list of pull requests made since the last tagged release"""
+ r = requests.get(MERGED_PR_ENDPOINT)
+ done = False
+ merged_prs = []
+ with hglib.open(repo_path) as client:
+ last_tag = client.log("reverse(tag())")[0]
+ while not done:
+ if r.status_code != 200:
+ raise RuntimeError
+ data = r.json()
+ prs = data['values']
+ for pr in prs:
+ activity = requests.get(pr['links']['activity']['href']).json()
+ merge_date = None
+ for action in activity['values']:
+ if 'update' in action and action['update']['state'] == 'MERGED':
+ merge_date = action['update']['date']
+ merge_date = merge_date.split('.')[0]
+ timestamp = mktime(strptime(merge_date, "%Y-%m-%dT%H:%M:%S"))
+ merge_date = datetime.fromtimestamp(timestamp)
+ break
+ if merge_date is None:
+ break
+ if merge_date < last_tag[6]:
+ done = True
+ break
+ merged_prs.append(pr)
+ r = requests.get(data['next'])
+ return merged_prs
+
+
+def cache_commit_data(prs):
+ """Avoid repeated calls to bitbucket API to get the list of commits per PR"""
+ commit_data = {}
+ for pr in prs:
+ data = requests.get(pr['links']['commits']['href']).json()
+ if data.keys() == [u'error']:
+ # this happens when commits have been stripped, e.g.
+ # https://bitbucket.org/yt_analysis/yt/pull-requests/1641
+ continue
+ done = False
+ commits = []
+ while not done:
+ commits.extend(data['values'])
+ if 'next' not in data:
+ done = True
+ else:
+ data = requests.get(data['next']).json()
+ commit_data[pr['id']] = commits
+ return commit_data
+
+
+def find_commit_in_prs(needle, commit_data, prs):
+ """Finds the commit `needle` PR in the commit_data dictionary
+
+ If found, returns the pr the needle commit is in. If the commit was not
+ part of the PRs in the dictionary, returns None.
+ """
+ for pr_id in commit_data:
+ commits = commit_data[pr_id]
+ for commit in commits:
+ if commit['hash'] == needle[1]:
+ pr = [pr for pr in prs if pr['id'] == pr_id][0]
+ return pr
+ return None
+
+
+def find_merge_commit_in_prs(needle, prs):
+ """Find the merge commit `needle` in the list of `prs`
+
+ If found, returns the pr the merge commit comes from. If not found, return
+ None
+ """
+ for pr in prs[::-1]:
+ if pr['merge_commit'] is not None:
+ if pr['merge_commit']['hash'] == needle[1][:12]:
+ return pr
+ return None
+
+
+def create_commits_to_prs_mapping(linege, prs):
+ """create a mapping from commits to the pull requests that the commit is
+ part of
+ """
+ commits_to_prs = {}
+ # make a copy of this list to avoid side effects from calling this function
+ my_prs = list(prs)
+ commit_data = cache_commit_data(my_prs)
+ for commit in lineage:
+ cset_hash = commit[1]
+ message = commit[5]
+ if message.startswith('Merged in') and '(pull request #' in message:
+ pr = find_merge_commit_in_prs(commit, my_prs)
+ if pr is None:
+ continue
+ commits_to_prs[cset_hash] = pr
+ # Since we know this PR won't have another commit associated with it,
+ # remove from global list to reduce number of network accesses
+ my_prs.remove(commits_to_prs[cset_hash])
+ else:
+ pr = find_commit_in_prs(commit, commit_data, my_prs)
+ commits_to_prs[cset_hash] = pr
+ return commits_to_prs
+
+
+def invert_commits_to_prs_mapping(commits_to_prs):
+ """invert the mapping from individual commits to pull requests"""
+ inv_map = {}
+ for k, v in commits_to_prs.iteritems():
+ # can't save v itself in inv_map since it's an unhashable dictionary
+ if v is not None:
+ created_date = v['created_on'].split('.')[0]
+ timestamp = mktime(strptime(created_date, "%Y-%m-%dT%H:%M:%S"))
+ created_date = datetime.fromtimestamp(timestamp)
+ pr_desc = (v['id'], v['title'], created_date,
+ v['links']['html']['href'], v['description'])
+ else:
+ pr_desc = None
+ inv_map[pr_desc] = inv_map.get(pr_desc, [])
+ inv_map[pr_desc].append(k)
+ return inv_map
+
+
+def get_last_descendant(repo_path, commit):
+ """get the most recent descendant of a commit"""
+ with hglib.open(repo_path) as client:
+ com = client.log('last(%s::)' % commit)
+ return com[0][1][:12]
+
+def screen_already_backported(repo_path, inv_map):
+ with hglib.open(repo_path) as client:
+ tags = client.log("reverse(tag())")
+ major_tags = [t for t in tags if t[2].endswith('.0')]
+ most_recent_major_tag_name = major_tags[0][2]
+ lineage = client.log(
+ "descendants(%s) and branch(stable)" % most_recent_major_tag_name)
+ prs_to_screen = []
+ for pr in inv_map:
+ for commit in lineage:
+ if commit[5].startswith('Backporting PR #%s' % pr[0]):
+ prs_to_screen.append(pr)
+ for pr in prs_to_screen:
+ del inv_map[pr]
+ return inv_map
+
+def commit_already_on_stable(repo_path, commit):
+ with hglib.open(repo_path) as client:
+ commit_info = client.log(commit)[0]
+ most_recent_tag_name = client.log("reverse(tag())")[0][2]
+ lineage = client.log(
+ "descendants(%s) and branch(stable)" % most_recent_tag_name)
+ # if there is a stable commit with the same commit message,
+ # it's been grafted
+ if any([commit_info[5] == c[5] for c in lineage]):
+ return True
+ return False
+
+def backport_pr_commits(repo_path, inv_map, last_stable, prs):
+ """backports pull requests to the stable branch.
+
+ Accepts a dictionary mapping pull requests to a list of commits that
+ are in the pull request.
+ """
+ pr_list = inv_map.keys()
+ pr_list = sorted(pr_list, key=lambda x: x[2])
+ for pr_desc in pr_list:
+ merge_warn = False
+ merge_commits = []
+ pr = [pr for pr in prs if pr['id'] == pr_desc[0]][0]
+ data = requests.get(pr['links']['commits']['href']).json()
+ commits = data['values']
+ while 'next' in data:
+ data = requests.get(data['next']).json()
+ commits.extend(data['values'])
+ commits = [com['hash'][:12] for com in commits]
+ with hglib.open(repo_path) as client:
+ for com in commits:
+ if client.log('merge() and %s' % com) != []:
+ merge_warn = True
+ merge_commits.append(com)
+ if len(commits) > 1:
+ revset = " | ".join(commits)
+ revset = '"%s"' % revset
+ message = "Backporting PR #%s %s" % \
+ (pr['id'], pr['links']['html']['href'])
+ dest = get_last_descendant(repo_path, last_stable)
+ message = \
+ "hg rebase -r %s --keep --collapse -m \"%s\" -d %s\n" % \
+ (revset, message, dest)
+ message += "hg update stable\n\n"
+ if merge_warn is True:
+ if len(merge_commits) > 1:
+ merge_commits = ", ".join(merge_commits)
+ else:
+ merge_commits = merge_commits[0]
+ message += \
+ "WARNING, PULL REQUEST CONTAINS MERGE COMMITS, CONSIDER\n" \
+ "BACKPORTING BY HAND TO AVOID BACKPORTING UNWANTED CHANGES\n"
+ message += \
+ "Merge commits are %s\n\n" % merge_commits
+ else:
+ if commit_already_on_stable(repo_path, commits[0]) is True:
+ continue
+ message = "hg graft %s\n" % commits[0]
+ print "PR #%s\nTitle: %s\nCreated on: %s\nLink: %s\n%s" % pr_desc
+ print "To backport, issue the following command(s):\n"
+ print message
+ raw_input('Press any key to continue')
+
+
+if __name__ == "__main__":
+ print ""
+ print "Gathering PR information, this may take a minute."
+ print "Don't worry, yt loves you."
+ print ""
+ repo_path = clone_new_repo()
+ try:
+ last_major_release = get_first_commit_after_last_major_release(repo_path)
+ last_dev = get_branch_tip(repo_path, 'yt', 'experimental')
+ last_stable = get_branch_tip(repo_path, 'stable')
+ lineage = get_lineage_between_release_and_tip(
+ repo_path, last_major_release, last_dev)
+ prs = get_pull_requests_since_last_release(repo_path)
+ commits_to_prs = create_commits_to_prs_mapping(lineage, prs)
+ inv_map = invert_commits_to_prs_mapping(commits_to_prs)
+ # for now, ignore commits that aren't part of a pull request since
+ # the last bugfix release. These are mostly commits in pull requests
+ # from before the last bugfix release but might include commits that
+ # were pushed directly to the repo.
+ del inv_map[None]
+
+ inv_map = screen_already_backported(repo_path, inv_map)
+ print "In another terminal window, navigate to the following path:"
+ print "%s" % repo_path
+ raw_input("Press any key to continue")
+ backport_pr_commits(repo_path, inv_map, last_stable, prs)
+ raw_input(
+ "Now you need to push your backported changes. The temporary\n"
+ "repository currently being used will be deleted as soon as you\n"
+ "press any key.")
+ finally:
+ shutil.rmtree(repo_path)
diff -r 2f95674671178a4868f4f6aeff56b041127d189e -r 2e98bc17afff89f7b63aa763b45a82bb8b9fac97 scripts/yt_lodgeit.py
--- a/scripts/yt_lodgeit.py
+++ /dev/null
@@ -1,320 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-"""
- LodgeIt!
- ~~~~~~~~
-
- A script that pastes stuff into the yt-project pastebin on
- paste.yt-project.org.
-
- Modified (very, very slightly) from the original script by the authors
- below.
-
- .lodgeitrc / _lodgeitrc
- -----------------------
-
- Under UNIX create a file called ``~/.lodgeitrc``, under Windows
- create a file ``%APPDATA%/_lodgeitrc`` to override defaults::
-
- language=default_language
- clipboard=true/false
- open_browser=true/false
- encoding=fallback_charset
-
- :authors: 2007-2008 Georg Brandl <georg at python.org>,
- 2006 Armin Ronacher <armin.ronacher at active-4.com>,
- 2006 Matt Good <matt at matt-good.net>,
- 2005 Raphael Slinckx <raphael at slinckx.net>
-"""
-import os
-import sys
-from optparse import OptionParser
-
-
-SCRIPT_NAME = os.path.basename(sys.argv[0])
-VERSION = '0.3'
-SERVICE_URL = 'http://paste.yt-project.org/'
-SETTING_KEYS = ['author', 'title', 'language', 'private', 'clipboard',
- 'open_browser']
-
-# global server proxy
-_xmlrpc_service = None
-
-
-def fail(msg, code):
- """Bail out with an error message."""
- print >> sys.stderr, 'ERROR: %s' % msg
- sys.exit(code)
-
-
-def load_default_settings():
- """Load the defaults from the lodgeitrc file."""
- settings = {
- 'language': None,
- 'clipboard': True,
- 'open_browser': False,
- 'encoding': 'iso-8859-15'
- }
- rcfile = None
- if os.name == 'posix':
- rcfile = os.path.expanduser('~/.lodgeitrc')
- elif os.name == 'nt' and 'APPDATA' in os.environ:
- rcfile = os.path.expandvars(r'$APPDATA\_lodgeitrc')
- if rcfile:
- try:
- f = open(rcfile)
- for line in f:
- if line.strip()[:1] in '#;':
- continue
- p = line.split('=', 1)
- if len(p) == 2:
- key = p[0].strip().lower()
- if key in settings:
- if key in ('clipboard', 'open_browser'):
- settings[key] = p[1].strip().lower() in \
- ('true', '1', 'on', 'yes')
- else:
- settings[key] = p[1].strip()
- f.close()
- except IOError:
- pass
- settings['tags'] = []
- settings['title'] = None
- return settings
-
-
-def make_utf8(text, encoding):
- """Convert a text to UTF-8, brute-force."""
- try:
- u = unicode(text, 'utf-8')
- uenc = 'utf-8'
- except UnicodeError:
- try:
- u = unicode(text, encoding)
- uenc = 'utf-8'
- except UnicodeError:
- u = unicode(text, 'iso-8859-15', 'ignore')
- uenc = 'iso-8859-15'
- try:
- import chardet
- except ImportError:
- return u.encode('utf-8')
- d = chardet.detect(text)
- if d['encoding'] == uenc:
- return u.encode('utf-8')
- return unicode(text, d['encoding'], 'ignore').encode('utf-8')
-
-
-def get_xmlrpc_service():
- """Create the XMLRPC server proxy and cache it."""
- global _xmlrpc_service
- import xmlrpclib
- if _xmlrpc_service is None:
- try:
- _xmlrpc_service = xmlrpclib.ServerProxy(SERVICE_URL + 'xmlrpc/',
- allow_none=True)
- except Exception, err:
- fail('Could not connect to Pastebin: %s' % err, -1)
- return _xmlrpc_service
-
-
-def copy_url(url):
- """Copy the url into the clipboard."""
- # try windows first
- try:
- import win32clipboard
- except ImportError:
- # then give pbcopy a try. do that before gtk because
- # gtk might be installed on os x but nobody is interested
- # in the X11 clipboard there.
- from subprocess import Popen, PIPE
- try:
- client = Popen(['pbcopy'], stdin=PIPE)
- except OSError:
- try:
- import pygtk
- pygtk.require('2.0')
- import gtk
- import gobject
- except ImportError:
- return
- gtk.clipboard_get(gtk.gdk.SELECTION_CLIPBOARD).set_text(url)
- gobject.idle_add(gtk.main_quit)
- gtk.main()
- else:
- client.stdin.write(url)
- client.stdin.close()
- client.wait()
- else:
- win32clipboard.OpenClipboard()
- win32clipboard.EmptyClipboard()
- win32clipboard.SetClipboardText(url)
- win32clipboard.CloseClipboard()
-
-
-def open_webbrowser(url):
- """Open a new browser window."""
- import webbrowser
- webbrowser.open(url)
-
-
-def language_exists(language):
- """Check if a language alias exists."""
- xmlrpc = get_xmlrpc_service()
- langs = xmlrpc.pastes.getLanguages()
- return language in langs
-
-
-def get_mimetype(data, filename):
- """Try to get MIME type from data."""
- try:
- import gnomevfs
- except ImportError:
- from mimetypes import guess_type
- if filename:
- return guess_type(filename)[0]
- else:
- if filename:
- return gnomevfs.get_mime_type(os.path.abspath(filename))
- return gnomevfs.get_mime_type_for_data(data)
-
-
-def print_languages():
- """Print a list of all supported languages, with description."""
- xmlrpc = get_xmlrpc_service()
- languages = xmlrpc.pastes.getLanguages().items()
- languages.sort(lambda a, b: cmp(a[1].lower(), b[1].lower()))
- print 'Supported Languages:'
- for alias, name in languages:
- print ' %-30s%s' % (alias, name)
-
-
-def download_paste(uid):
- """Download a paste given by ID."""
- xmlrpc = get_xmlrpc_service()
- paste = xmlrpc.pastes.getPaste(uid)
- if not paste:
- fail('Paste "%s" does not exist.' % uid, 5)
- print paste['code'].encode('utf-8')
-
-
-def create_paste(code, language, filename, mimetype, private):
- """Create a new paste."""
- xmlrpc = get_xmlrpc_service()
- rv = xmlrpc.pastes.newPaste(language, code, None, filename, mimetype,
- private)
- if not rv:
- fail('Could not create paste. Something went wrong '
- 'on the server side.', 4)
- return rv
-
-
-def compile_paste(filenames, langopt):
- """Create a single paste out of zero, one or multiple files."""
- def read_file(f):
- try:
- return f.read()
- finally:
- f.close()
- mime = ''
- lang = langopt or ''
- if not filenames:
- data = read_file(sys.stdin)
- if not langopt:
- mime = get_mimetype(data, '') or ''
- fname = ""
- elif len(filenames) == 1:
- fname = filenames[0]
- data = read_file(open(filenames[0], 'rb'))
- if not langopt:
- mime = get_mimetype(data, filenames[0]) or ''
- else:
- result = []
- for fname in filenames:
- data = read_file(open(fname, 'rb'))
- if langopt:
- result.append('### %s [%s]\n\n' % (fname, langopt))
- else:
- result.append('### %s\n\n' % fname)
- result.append(data)
- result.append('\n\n')
- data = ''.join(result)
- lang = 'multi'
- return data, lang, fname, mime
-
-
-def main():
- """Main script entry point."""
-
- usage = ('Usage: %%prog [options] [FILE ...]\n\n'
- 'Read the files and paste their contents to %s.\n'
- 'If no file is given, read from standard input.\n'
- 'If multiple files are given, they are put into a single paste.'
- % SERVICE_URL)
- parser = OptionParser(usage=usage)
-
- settings = load_default_settings()
-
- parser.add_option('-v', '--version', action='store_true',
- help='Print script version')
- parser.add_option('-L', '--languages', action='store_true', default=False,
- help='Retrieve a list of supported languages')
- parser.add_option('-l', '--language', default=settings['language'],
- help='Used syntax highlighter for the file')
- parser.add_option('-e', '--encoding', default=settings['encoding'],
- help='Specify the encoding of a file (default is '
- 'utf-8 or guessing if available)')
- parser.add_option('-b', '--open-browser', dest='open_browser',
- action='store_true',
- default=settings['open_browser'],
- help='Open the paste in a web browser')
- parser.add_option('-p', '--private', action='store_true', default=False,
- help='Paste as private')
- parser.add_option('--no-clipboard', dest='clipboard',
- action='store_false',
- default=settings['clipboard'],
- help="Don't copy the url into the clipboard")
- parser.add_option('--download', metavar='UID',
- help='Download a given paste')
-
- opts, args = parser.parse_args()
-
- # special modes of operation:
- # - paste script version
- if opts.version:
- print '%s: version %s' % (SCRIPT_NAME, VERSION)
- sys.exit()
- # - print list of languages
- elif opts.languages:
- print_languages()
- sys.exit()
- # - download Paste
- elif opts.download:
- download_paste(opts.download)
- sys.exit()
-
- # check language if given
- if opts.language and not language_exists(opts.language):
- fail('Language %s is not supported.' % opts.language, 3)
-
- # load file(s)
- try:
- data, language, filename, mimetype = compile_paste(args, opts.language)
- except Exception, err:
- fail('Error while reading the file(s): %s' % err, 2)
- if not data:
- fail('Aborted, no content to paste.', 4)
-
- # create paste
- code = make_utf8(data, opts.encoding)
- pid = create_paste(code, language, filename, mimetype, opts.private)
- url = '%sshow/%s/' % (SERVICE_URL, pid)
- print url
- if opts.open_browser:
- open_webbrowser(url)
- if opts.clipboard:
- copy_url(url)
-
-
-if __name__ == '__main__':
- sys.exit(main())
diff -r 2f95674671178a4868f4f6aeff56b041127d189e -r 2e98bc17afff89f7b63aa763b45a82bb8b9fac97 setup.py
--- a/setup.py
+++ b/setup.py
@@ -164,7 +164,7 @@
config.make_config_py()
# config.make_svn_version_py()
config.add_subpackage('yt', 'yt')
- config.add_scripts("scripts/*")
+ config.add_scripts("scripts/iyt")
return config
Repository URL: https://bitbucket.org/yt_analysis/yt/
--
This is a commit notification from bitbucket.org. You are receiving
this because you have the service enabled, addressing the recipient of
this email.
More information about the yt-svn
mailing list