Commit 8d8c97d8 authored by jpic ∞'s avatar jpic ∞ 💾
Browse files

Complete rewrite

Test & Document driven
Fruit of 3 rewrites from scratch
Keeping the best of each ...
parent 5b45cd2c
cli2: unfrustrating python CLI
cli2: Dynamic CLI for Python 3
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Sometimes I just want to execute a python callback and pass args/kwargs on the
CLI, and not have to define any custom CLI entry point of any sort, nor change
any code, typically when automating stuff, cli2 unfrustrates me::
Break free from the POSIX standard for more fluent CLIs, by exposing simple
Python functions or objects with a minimalist argument typing style, or
building your own command try during runtime.
cli2 yourmodule.yourcallback somearg somekwarg=foo
Getting Started
===============
Sometimes I just want to define a new command and expose all callables in a
module and I can't just do it with a one-liner. cli2 unfrustrates me again:
You can either create a Command from a callable that can invoked directly or
via console_script:
.. code-block:: python
console_script = cli2.ConsoleScript(__doc__).add_module('mymodule')
# then i add console_script entrypoint as such: mycmd = mycmd.console_script
def yourcmd():
"""Your own command"""
I also like when readonly commands are in green, writing commands in yellow and
destructive commands in red, I find the commands list in the help output more
readable, and directive for new users of the CLI:
# good enough for your console_script entry_point
console_script = cli2.Command(yourcmd)
# without entry_point, you can call yourself
console_script()
Command group
-------------
In the same fashing, you can create a command Group, and add Commands to it:
.. code-block:: python
# or create a command group group
console_script = cli2.Group()
# and add yourcmd to it
console_script.cmd(yourcmd)
Type-casting
------------
Type hinting is well supported, but you may also hack how arguments are casted
into python values at a per argument level, set the ``cli2_argname`` attribute
to attributes that you want to override on the generated Argument for
``argname``.
You could cast any argument with JSON as such:
.. code-block:: python
def yourcmd(x):
return x
yourcmd.cli2_x = dict(cast=lambda v: json.loads(v))
cmd = Command(yourcmd)
cmd(['[1,2]']) == [1, 2] # same as CLI: yourcmd [1,2]
Or, override ``Argument.cast()`` for the ``ages`` argument:
.. code-block:: python
def yourcmd(ages):
return ages
yourcmd.cli2_ages = dict(cast=lambda v: [int(i) for i in v.split(',')])
cmd = Command(yourcmd)
cmd(['1,2']) == [1, 2] # same as CLI: yourcmd 1,2
If an argument is annotated with the list or dict type, then cli2 will use
json.loads to cast them to Python arguments, but be careful with spaces on your
command line: one sysarg goes to one argument::
yourcmd ["a","b"] # works
yourcmd ["a", "b"] # does not because of the space
However, space is supported as long as in the same sysarg:
.. code-block:: python
subprocess.check_call(['yourcmd', '["a", "b"]')
Typable syntax
--------------
Arguments with the list type annotation are automatically parsed as JSON, if
that fails it will try to split by commas which is easier to type than JSON for
lists of strings::
yourcmd a,b # calls yourcmd(["a", "b"])
Keep in mind that JSON is tried first for list arguments, so a list of ints is
also easy::
yourcmd [1,2] # calls yourcmd([1, 2])
A simple syntax is also supported for dicts by default::
yourcmd a:b,c:d # calls yourcmd({"a": "b", "c": "d"})
The disadvantage is that JSON decode exceptions are swallowed, but by design
cli2 is supposed to make Python types more accessible on the CLI, rather than
being a JSON validation tool. Generated JSON args should always work though.
Boolean flags
-------------
Cast to boolean is already supported by type-hinting, or with json (see above
example), or with simple switches:
.. code-block:: python
@cli2.config(color=cli2.RED)
def challenge(dir):
'''The challenge command dares you to run it.'''
os.exec('rm -rf ' + dir)
def yourcmd(debug=True):
pass
Of course then there's all this code I need to have coverage for and I'm
`still
<https://pypi.org/project/django-dbdiff/>`_ so lazy that I still
`don't write most of my test code myself
<https://pypi.org/project/django-responsediff/>`_, so I throwed an autotest
function in cli2 ("ala" dbunit with a personal touch) that I can use as such:
# prefixing dash not necessary at all
yourcmd.cli2_debug = dict(negate='-no-debug')
# or map this boolean to two simple switches
yourcmd.cli2_debug = dict(alias='-d', negate='-nd')
Edge cases
==========
Simple and common use cases were favored over rarer use cases by design. Know
the couple of gotchas and you'll be fine.
Args containing ``=`` in Mixed ``(*args, **kwargs)``
----------------------------------------------------
It was decided to favor simple use cases when a callable both have varargs and
varkwargs as such:
.. code-block:: python
@pytest.mark.parametrize('name,command', [
('cli2', ''),
('help', 'help'),
('help_debug', 'help debug'),
# ... bunch of other commands
('debug', 'debug cli2.run to see=how -it --parses=me'),
])
def test_cli2(name, command):
cli2.autotest(
f'tests/{name}.txt',
'cli2 ' + command,
)
def foo(*args, **kwargs):
return (args, kwargs)
Call ``foo("a", b="x")`` on the CLI as such::
foo a b=x
**BUT**, to call ``foo("a", "b=x")`` on the CLI you will need to use an
asterisk with a JSON list as such::
foo '*["a","b=x"]'
Admittedly, the second use case should be pretty rare compared to the first
one, so that's why the first one is favored.
For the sake of consistency, varkwarg can also be specified with a double
asterisk and a JSON dict as such::
# call foo("a", b="x")
foo a **{"b":"x"}
Calling with ``a="b=x"`` in ``(a=None, b=None)``
------------------------------------------------
The main weakness is that it's difficult to tell the difference between a
keyword argument, and a keyword argument passed positionnaly which value starts
with the name of another keyword argument. Example:
.. code-block:: python
def foo(a=None, b=None):
return (a, b)
Call ``foo(b='x')`` on the CLI like this::
foo b=x
**BUT**, to call ``foo(a="b=x")`` on the CLI, you need to name the argument::
You should be able tho pip install cli2 and start using the cli2 command, or
cli2.ConsoleScript to make your own commands.
foo a=b=x
.. image:: https://asciinema.org/a/221137.svg
:target: https://asciinema.org/a/221137
Admitadly, that's a silly edge case. Protect yourself from it by always naming
keyword arguments ...
Check `djcli, another cli built on cli2
<https://pypi.org/project/djcli>`_.
... Because the parser considers token that start with a keyword of a keyword
argument prioritary to positional arguments once the positional arguments have
all been bound.
# flake8: noqa
"""cli2 is like click but as laxist as docopts."""
from .colors import * # noqa
from .exceptions import Cli2Exception, Cli2ArgsException
from .parser import Parser
from .introspection import docfile, Callable, Importable, DocDescriptor
from .command import command, option, Option
from .console_script import ConsoleScript, BaseGroup, Group
from .test import autotest
from .cli import debug, docmod, help, run
from .argument import Argument
from .command import Command
from .group import Group
from .node import Node
import re
import json
class Argument:
def __init__(self, cmd, param):
self.cmd = cmd
self.param = param
self.alias = param.name if self.iskw else None
self.negate = None
def __repr__(self):
return self.param.name
@property
def iskw(self):
return self.param.kind in (
self.param.KEYWORD_ONLY,
self.param.POSITIONAL_OR_KEYWORD,
)
@property
def set(self):
return self.param.name not in self.cmd.bound.arguments
@property
def accepts(self):
return (
self.param.name not in self.cmd.bound.arguments
or self.param.kind in (
self.param.VAR_POSITIONAL,
self.param.VAR_KEYWORD,
)
)
@property
def value(self):
return self.cmd.bound.arguments[self.param.name]
@value.setter
def value(self, value):
if self.param.kind == self.param.VAR_POSITIONAL:
self.cmd.bound.arguments.setdefault(self.param.name, [])
self.cmd.bound.arguments[self.param.name].append(value)
elif self.param.kind == self.param.VAR_KEYWORD:
self.cmd.bound.arguments.setdefault(self.param.name, {})
parts = value.split('=')
value = '='.join(parts[1:])
self.cmd.bound.arguments[self.param.name][parts[0]] = value
else:
self.cmd.bound.arguments[self.param.name] = value
def cast(self, value):
if self.param.annotation == int:
return int(value)
if self.param.annotation == bool:
return value.lower() not in ('', '0', 'no', 'false', self.negate)
if self.param.annotation == list:
try:
return json.loads(value)
except json.JSONDecodeError:
return [i.strip() for i in value.split(',')]
if self.param.annotation == dict:
try:
return json.loads(value)
except json.JSONDecodeError:
results = dict()
for token in value.split(','):
parts = token.split(':')
results[parts[0].strip()] = ':'.join(parts[1:]).strip()
return results
return value
def aliasmatch(self, arg):
if arg == self.negate:
return True
if self.iskw and self.param.annotation == bool and arg == self.alias:
return True
return self.alias and arg.startswith(self.alias + '=')
def match(self, arg):
if self.aliasmatch(arg):
if self.param.annotation != bool or '=' in arg:
return arg[len(self.alias) + 1:]
return arg
def take(self, arg):
if self.param.kind == self.param.VAR_KEYWORD:
if arg.startswith('**{') and arg.endswith('}'):
self.cmd.bound.arguments[self.param.name] = json.loads(arg[2:])
return True
elif self.param.kind == self.param.VAR_POSITIONAL:
if arg.startswith('*[') and arg.endswith(']'):
self.cmd.bound.arguments[self.param.name] = json.loads(arg[1:])
return True
# edge case vararg+varkwargs
# priority to varkwargs for word= and **{}
last = self.cmd[[*self.cmd.keys()][-1]]
if last.param.kind == self.param.VAR_KEYWORD:
if re.match('^\\w+=', arg):
return
elif arg.startswith('**{') and arg.endswith('}'):
return
value = self.match(arg)
if value is not None:
self.value = self.cast(value)
return True
"""
cli2 makes your python callbacks work on CLI too !
cli2 provides sub-commands to introspect python modules or callables docstrings
or to execute callables or help working with cli2 itself.
"""
import textwrap
import types
from .console_script import ConsoleScript, BaseGroup
from .command import command, option
from .exceptions import Cli2ArgsException
from .colors import GREEN, RED, RESET, YELLOW
from .introspection import docfile, Callable, Importable
def docmod(module_name):
"""Docstring for a module in dotted path.
import inspect
Example: cli2 docmod cli2
"""
return BaseGroup.factory(module_name).doc
import cli2
@command(color=GREEN)
def help(*args):
def run(dotted_path):
"""
Get help for a command.
Example::
$ cli2 help help
Run a python callable by dotted path, or print the list of callables found.
"""
console_script = ConsoleScript.singleton
if not args:
# show documentation for parsed group
yield console_script.parser.group.doc
else:
# show command documentation if possible
if args[0] in console_script.parser.group:
current = console_script.parser.group
args = list(args)
while args and args[0] in current:
current = current[args.pop(0)]
yield current.doc
else:
importable = Importable.factory(args[0])
if importable.target and not importable.is_module:
yield importable.doc
elif importable.module:
if not importable.target:
yield f'{RED}Cannot import {args[0]}{RESET}'
yield ' '.join([
YELLOW,
'Showing help for',
importable.module.__name__ + RESET
])
yield BaseGroup.factory(importable.module.__name__).doc
@command(color=GREEN)
def debug(callback, *args, **kwargs):
"""
Dump parsed variables.
Example usage::
cli2 debug test to=see --how -it=parses
"""
cs = console_script
parser = cs.parser
yield textwrap.dedent(f'''
Callable: {RED}{callback}{RESET}
Args: {YELLOW}{args}{RESET}
Kwargs: {YELLOW}{kwargs}{RESET}
console_script.parser.options: {GREEN}{parser.options}{RESET}
console_script.parser.dashargs: {GREEN}{parser.dashargs}{RESET}
console_script.parser.dashkwargs: {GREEN}{parser.dashkwargs}{RESET}
console_script.parser.extraargs: {GREEN}{parser.extraargs}{RESET}
''').strip()
@option('debug', help='Also print debug output', color=GREEN, alias='d')
def run(callback, *args, **kwargs):
"""
Execute a python callback on the command line.
To call your.module.callback('arg1', 'argN', kwarg1='foo'):
cli2 your.module.callback arg1 argN kwarg1=foo
You can also prefix arguments with a dash, those that contain equal sign
will end in dict console_script.parser.dashkwargs, those without equal
sign will end up in a list in console_script.parser.dashargs.
For examples, try `cli2 debug`.
For other commands, try `cli2 help`.
"""
if console_script.parser.options.get('debug', False):
print('HELLO')
cb = Callable.factory(callback)
if cb.target and not cb.is_module:
try:
result = cb(*args, **kwargs)
except Cli2ArgsException as e:
print(e)
print(cb.doc)
result = None
console_script.exit_code = 1
except Exception as e:
out = [f'{RED}Running {callback}(']
if args and kwargs:
out.append(f'*{args}, **{kwargs}')
elif args:
out.append(f'*{args}')
elif kwargs:
out.append(f'**{kwargs}')
out.append(f') raised {type(e)}{RESET}')
e.args = (e.args[0] + '\n' + cb.doc,) + e.args[1:]
raise
if isinstance(result, types.GeneratorType):
yield from result
else:
yield result
else:
if '.' in callback:
yield f'{RED}Could not import callback: {callback}{RESET}'
else:
yield f'{RED}Cannot run a module{RESET}: try {callback}.something'
if cb.module:
yield ' '.join([
'However we could import module',
f'{GREEN}{cb.module.__name__}{RESET}',
])
doc = docmod(cb.module.__name__)
if doc:
yield f'Showing help for {GREEN}{cb.module.__name__}{RESET}:'
yield doc
else:
return f'Docstring not found in {cb.module.__name__}'
elif callback != callback.split('.')[0]:
yield ' '.join([
RED,
'Could not import module:',
callback.split('.')[0],
RESET,
])
node = cli2.Node.factory(dotted_path)
if not node.target:
return 'Not found ' + str(node)
print(inspect.getdoc(node.target))
for sub in node.callables:
print(sub)
console_script = ConsoleScript(
__doc__,
default_command='run'
).add_commands(
help,
run,
debug,
command(color=GREEN)(docmod),
command(color=GREEN)(docfile),
)
console_script = cli2.Command(run, name='cli2')
from .introspection import Callable
def command(**config):
def wrap(cb):
if 'cli2' not in cb.__dict__:
name = config.pop('name', cb.__name__)
cb.cli2 = Callable(name, cb, **config)
else:
cb.cli2.__dict__.update(config)
return cb
return wrap
class Option:
def __init__(self, name, help=None, color=None, alias=None,
immediate=False, default=None):
self.name = name
self.help = help or 'Undocumented option'
self.color = color or ''
self.alias = alias
self.immediate = immediate
self.default = default
def option(name, **cfg):
def wrap(cb):
if 'cli2' not in cb.__dict__:
cb = command()(cb)
cb.cli2.options[name] = Option(name, **cfg)
return cb
return wrap
import asyncio
import inspect
import sys
from .argument import Argument
class Command(dict):
def __init__(self, target, name=None, doc=None):
self.target = target
self.name = name or getattr(target, '__name__', type(target).__name__)
self.doc = doc or inspect.getdoc(target)
self.sig = inspect.signature(target)
for name, param in self.sig.parameters.items():
self[name] = Argument(self, param)
overrides = getattr(self.target, 'cli2_' + name, {})
for key, value in overrides.items():
setattr(self[name], key, value)
def help(self, error=None, short=False):
output = []
if error:
output.append(error + '\n')
if self.doc:
output.append(self.doc + '\n')
return '\n'.join(output)
def parse(self, *argv):
self.bound = self.sig.bind_partial()
extra = []
for current in argv:
taken = False
for arg in self.values():
if not arg.accepts:
continue
if arg.iskw:
# we have reached keyword argument sequence
# see if another, further arg matches per alias and takes
reached = False
for _arg in self.values():
if taken:
break
if _arg == arg:
reached = True
continue
if not reached:
continue
if _arg.aliasmatch(current):
taken = _arg.take(current)
if taken:
break
taken = arg.take(current)
if taken:
break
if not taken:
extra.append(current)