aspenweb / state_chain.py Goto Github PK

We should make a new release, two PRs with code and docs improvements have been merged since 1.0.0 (#10 in particular would help for AspenWeb/pando.py#548).

You can add pdb.set_trace to an algorithm list and it will work as expected (you get dropped into the run function when it's hit). However, you have to import pdb separately. We should have an algorithm.set_trace method that aliases pdb.set_trace.

algorithm.on("key", algorithm_func) 

should add algorithm_func to a chain that gets executed after the "key" shows up in the state dict for the first time.

Aspen defined decorators for algorithm functions. Seems like those really belong over here. I'm inlining them here for anyone who wants to copy/paste them to their own projects until we get them fully ported over here.

# Filters
# =======
# These are decorators for algorithm functions.

def by_lambda(filter_lambda):
    """
    """
    def wrap(function):
        def wrapped_function_by_lambda(*args,**kwargs):
            if filter_lambda():
                return function(*args,**kwargs)
        algorithm._transfer_func_name(wrapped_function_by_lambda, function)
        return wrapped_function_by_lambda
    return wrap


def by_regex(regex_tuples, default=True):
    """Only call function if

    regex_tuples is a list of (regex, filter?) where if the regex matches the
    requested URI, then the flow is applied or not based on if filter? is True
    or False.

    For example::

        from aspen.flows.filter import by_regex

        @by_regex( ( ("/secret/agenda", True), ( "/secret.*", False ) ) )
        def use_public_formatting(request):
            ...

    would call the 'use_public_formatting' flow step only on /secret/agenda
    and any other URLs not starting with /secret.

    """
    regex_res = [ (re.compile(regex), disposition) \
                           for regex, disposition in regex_tuples.items() ]
    def filter_function(function):
        def function_filter(request, *args):
            for regex, disposition in regex_res:
                if regex.matches(request.line.uri):
                    if disposition:
                        return function(*args)
            if default:
                return function(*args)
        algorithm._transfer_func_name(function_filter, function)
        return function_filter
    return filter_function


def by_dict(truthdict, default=True):
    """Filter for hooks

    truthdict is a mapping of URI -> filter? where if the requested URI is a
    key in the dict, then the hook is applied based on the filter? value.

    """
    def filter_function(function):
        def function_filter(request, *args):
            do_hook = truthdict.get(request.line.uri, default)
            if do_hook:
                return function(*args)
        algorithm._transfer_func_name(function_filter, function)
        return function_filter
    return filter_function

The StateChain.run() method takes the initial state as keyword arguments, so it's not possible to use an existing dict, because Python creates a copy:

>>> d = {}
>>> def run(**state):
...     return state is d
... 
>>> run(**d)
False

Algorithm is generic. @Changaco had some name for the pattern we're using, started with an em and came from functional programming.

In addition to being somewhat inefficient, the current state chains don't facilitate static type checking. I'm thinking we could solve both problems with a 2.0 version of state chains based on optionally-typed state objects instead of dicts.

Below is a simplified example of what this 2.0 version could look like for a web framework like Pando:

class State:
    website: Website
    environ: dict
    request: Optional[Request] = None
    response: Optional[Response] = None
    exception: Optional[Exception] = None

chain = StateChain(State)

@chain.add
def parse_environ_into_request(state: State):
    state.request = Request.from_wsgi(state.website, state.environ)

@chain.add(exception='required')
def handle_negotiation_exception(state: State):
    if isinstance(state.exception, NotFound):
        state.response = Response(404)
    elif isinstance(state.exception, NegotiationFailure):
        state.response = Response(406, state.exception.message)
    else:
        return
    state.exception = None

state = chain.run(State(website, environ))

The differences between this hypothetical 2.0 and the current 1.x are:

• No more dependency injection, the chain functions now only take one argument: the state. This facilitates static typing and improves performance.
• A new chain.add decorator replaces the clunky StateChain.from_dotted_name constructor.
• The chain functions that handle exceptions are explicitly marked.

Error handling as currently implemented is not entirely intuitive. Essentially we're folding a branching tree into a linear flow, and it's hard to reason about that. What if instead we provided for a list of lists?

normal = [foo, bar, baz, buz]
fallback = [blah, buz]
last_ditch = [bloo]

algo = Algorithm(*normal, _cascade=[fallback, last_ditch])

Relative imports are now a thing. Does this module work given a 'dotted name' of .. ? or .algorithm ? etc, etc.

#20 improved the performance of state chains, but they still have a measurable performance impact. I'm thinking it could be a fun project to write a compiler to aggregate the bytecodes of the chain's functions into a single callable.

Right now insert_{before,after} and remove only take one function argument. They should take multiple function arguments. This way of calling the insert_* variants should result in an algorithm with the additional functions in the order they are passed to the insert_* method. E.g.:

>>> alg.insert_before('foo', bar, baz)
>>> alg.functions
[bar, baz, foo]

>>> alg.insert_after('foo', bar, baz)
>>> alg.functions
[foo, bar, baz]

That's not very good for performance. Upstream issue: AspenWeb/dependency_injection.py#6.

The insert_{before,after} functions should take special markers for their first argument, to indicate that the functions should be inserted at the start or end of the algorithm.

In Gratipay we patch Aspen's website algorithm by setting algorithm.functions. This method implicitly skips functions, so when a function is added to the algorithm upstream and we don't notice the change or forget to propagate it, then we can end up with silent errors like gratipay/gratipay.com#3210.

A method of modifying the algorithm based on explicit skipping would fix the silent error problem.

Here's an instance of me being confused because it wasn't obvious to me that a function was meant to be used in an algorithm. We could have a decorator that adds a __for_algorithm__ to the func or some such:

@algorithm.function
def foo_bar(baz, buz):
    pass

We could also rework from_dotted_name to use this to whitelist instead of blacklisting _foo.