[yt-dev] Arguments to scripts

[david collins]

> Talking this through leads me to think we might be able to have a
> combination of 4 and 2.  I just tested and what we *can* do is
> actually provide the parser arguments as above, but allow for a
> "leftovers" section for all *positional* arguments to the script.
> This would mean for --something and -s arguments you would need to add
> those to the parser object, but for things like what you do above, you
> wouldn't.
>
> Would that work for your use case?

So is option 4 something like

<myscript>
import yt
if __name__ == "__main__":
   yt_option_parser.add_option("--clown_hair_size_threshold")
   do some stuff using the clow hair size
</myscript>

like a modified/subclassed optparse?  Then my command line is like
mpirun -np 16127 python myscript --clown_hair 14 --parallel
?

In your "leftovers" version of Option 2, could that just modify
sys.argv in place, pop-ing out the bits that yt wants or wants to
raise an error on?  Not strictly necessary, but it would mean a little
less modification of old scripts on my part.  (Though frankly, scripts
where I use sys.argv directly should be thrown in the toilet, so
forcing me to re-write those tools to use some less fragile option
parser is probably good for me.)

Anyhow, as long as it still retains the ability for me to pass in
non-yt options to the script, I'm happy.  And there are certainly
bigger concerns here than my army of dumb scripts.  The first version
of option
2 wouldn't really do that, but the 'leftovers' version would.  I think
I like your new 2 + 4.

d.


>
> -Matt
>
>>
>> Additionally, if I use optparse, similar outcomes?
>>
>> d.
>>
>>
>> On Fri, Dec 2, 2011 at 9:30 AM, Matthew Turk <matthewturk at gmail.com> wrote:
>>> Hi all,
>>>
>>> I have prepared a Pull Request to change how yt processes arguments to
>>> scripts.  I just issued it, but I am emailing because I think
>>> discussion of what it does warrants a bit more public hashing out.
>>> The PR is not done yet, for the reasons I outline below, so please
>>> don't anybody accept it yet.
>>>
>>> https://bitbucket.org/yt_analysis/yt/pull-request/38/overhaul-configuration-system
>>>
>>> This will directly affect you if you have:
>>>
>>> 1) Ever written "from yt.config import ytcfg; ytcfg[...."
>>> 2) Ever put your *own* command-line parser into yt.
>>> 3) Gotten annoyed with configuration files.
>>>
>>> What I've done is create a new file, startup_tasks.py, that gets
>>> imported whenever yt.mods gets imported, and only the first time that
>>> happens.  It sets up an argument parser (using argparse, which is
>>> Python 2.7 only) that parses looking for:
>>>
>>> --parallel
>>> --paste
>>> --paste-detailed
>>> --detailed
>>> --rpdb
>>> --parallel
>>>
>>> One of the things this does is that it also provides --help, so you
>>> can see what is available.  Furthermore, I've added a --config option,
>>> so that from the command line you can set configuration options.  For
>>> instance:
>>>
>>> --config serialize=False
>>>
>>> and so on.  This is pretty cool I think and will go a long way toward
>>> making things nicer.  However, the way this works is still up for a
>>> few more problems.  There are basically two ways this can work:
>>>
>>>  * Parse the entirety of sys.args and accept all arguments that yt
>>> finds, rejecting and throwing an error on unrecognized ones (i.e.,
>>> typos or things you might pass in to a script your write on the
>>> command line).  This will be an exclusive operation.
>>>  * Parse *non-exclusively*, allowing unrecognized arguments to pass
>>> through.  However, the old arguments will still be there: so any
>>> script that has issues with things like --parallel and whatnot will
>>> now see there, whereas it did not before because yt (totally un-cool!)
>>> stripped them out of the sys.args variable.  I don't want to do this
>>> anymore.
>>>
>>> The way I have implemented this for the yt command line tool is to set
>>> a flag that says, "We're also inside the command line, so don't parse
>>> anything, we'll handle adding new options to the parser and then we'll
>>> parse everything at the end."  This way you can pass both --parallel
>>> and whatever option the yt command line utility wants.  This works
>>> because startup_tasks creates a "parser" object, adds arguments to
>>> that parser object, then delays actually conducting the parsing until
>>> all the arguments from teh command line tool have been added.
>>>
>>> There are four ways this can work.  I have presented them in order of
>>> my increasing preference.  (Coincidentally, on the astropy mailing
>>> list they discussed this this week, as I was thinking about my
>>> feelings on it as well, and they are moving away from parsing args in
>>> the library; I think that works for them because AstroPy is designed
>>> to be used much more inside larger frameworks, whereas yt is somewhat
>>> more insular.)
>>> 1) Don't do any argument parsing if not called through a yt-specific
>>> script runner.  This means if you want to pass --parallel, you have to
>>> run with something like "yt run my_script.py --parallel".  Same for
>>> --config and so on.
>>> 2) Parse all arguments any time yt.mods is imported, do not allow for
>>> additional arguments.  This breaks scripts that have their own
>>> parsing.
>>> 3) Parse *some* of the arguments, but not all.  All typos would
>>> succeed and this could lead to confusion for the user.
>>> 4) Provide a yt-specific mechanism for adding new arguments.  So if
>>> you want to add new arguments, you do it at the top of your script,
>>> rather than the bottom, and at the bottom inside the construction "if
>>> __name__ == '__main__'" you'd inspect the values.
>>>
>>> Anyway, I'm inclined to go for #4, simply because it would be the
>>> simplest mechanism for ensuring an explicit method of getting
>>> arguments into user-written scripts.
>>>
>>> Thoughts?
>>>
>>> -Matt
>>> _______________________________________________
>>> yt-dev mailing list
>>> yt-dev at lists.spacepope.org
>>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
>>
>>
>>
>> --
>> Sent from my computer.
>> _______________________________________________
>> yt-dev mailing list
>> yt-dev at lists.spacepope.org
>> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org
> _______________________________________________
> yt-dev mailing list
> yt-dev at lists.spacepope.org
> http://lists.spacepope.org/listinfo.cgi/yt-dev-spacepope.org



-- 
Sent from my computer.