Regex Routing

You can subscribe to arbitrary node’s output by providing a regular expression. In this example, we’ll use Redis’ pubsub capabilities to notify an external receiver of all tasks passing through the graph.

The product of this example is in examples/regex/graph.py.

First, we’ll create a function that yields multiple return values. In this case, we’re going to naively parse a HTTP querystring.

@router.node(('key', 'value'), entry_point=True)
def parse_querystring(msg):
    'parse a querystring into keys and values'
    for part in msg.querystring.strip().lstrip('?').split('&'):
        key, value = part.split('=')
        yield key, value

Now we’re going to count keys and values:

@router.node(('key', 'value', 'count'), prefix('parse_querystring'))
def count_keyval(msg):
    count = redis.zincrby('querystring_count.%s' % msg.key, msg.value, 1)
    return msg.key, msg.value, count

Next, we’ll make a function that publishes to Redis on every message:

@router.node(tuple(), '.+')
def notify_on_emit(msg):
    redis.publish('notify_on_emit.%s' % msg._origin, msg.as_json())
    return NoResult

Now, when you call router(querystring='?a=1&b=2&c=3'), notify_on_emit will publish seven messages: three with origin “graph.parse_querystring”, three with origin “graph.count_keyval”, and one with origin “__entry_point”. The graph ends up looking like this:

You can also specify ignores in Router.node, which can cut a little fat out of an otherwise greedy regex.