Multi-Line Lambdas in Python

I need to preface all of this with a disclaimer:
I love Python, but I am able to see plenty of faults with it. In this article, I attempt to provide a very roundabout way of working around one of those faults: the lack of multi-line lambdas. This is not to say that this is a good solution, but it may be one of the best that we have for certain cases. Try and see if one of the typical workarounds is the best option before settling on this.

Intro

Python has one missing feature that actually hurts my idea that Kotlin is a lot like a statically-typed Python: lack of multi-line lambdas; lambdas in Python, as you probably know, are limited to a single expression or statement. There are numerous ways to get around all of this, but none are very satisfying, since they either obfuscate the code or don’t put it in the place that you want it.

Workarounds

The first and most obvious workaround is to define a function that does what you want then pass that function in instead of a lambda. This is generally your best bet, and if you name the function well, it can help to actually make the code more clear. But, in some instances, having the code right there would be more helpful than any name.
The next workaround is to try and squeeze your code into one line somehow. In this video of a talk given at PyCon 2016, Oneliner-izer: An Exercise in Constrained Coding, Chelsea talks about how it’s technically possible to squeeze any code into a single line (I watched the video a while ago and didn’t actually finish it, so there may be some restrictions that I don’t remember/didn’t see). To be blunt, this is probably a BAD idea. In simple cases, it may work, but it’ll still likely be too confusing to be maintainable.
In certain circumstances you could also try to combine those lines using function piping or composition. If it’s possible to do this, and everyone in your codebase is kind of into functional programming, then this is probably your best bet. It’s generally pretty clean and readable, but it is limited.

The Workaround That is the Topic of This Article

That brings us to the workaround that I came up with for this article. It involves with and context managers. If you’ve followed my blog for a while, you may have read my articles earlier about in Java and Python, in which I showed my initial findings for using context managers to do multiline lambdas. At the time, I placed limitations on them that aren’t technically true. Before I get into that, though, I’ll reshow how they can be used for lambdas.
For a lambda that has no parameters and no return type, a with block is really simple:

with someFunc():
    # do something

For this case, it’s just a simple context manager where __enter__() doesn’t return anything.
For a lambda that has one parameter and no return type, a with block is still pretty simple:

with someFunc() as arg:
    # use arg

To make this, you do a typical context manager, but __enter__() returns a value that is used for arg.
In the older articles, I presented these as the only ways that with blocks could be used for lambdas. Luckily, this actually covers a large majority of use cases, but it’s actually not the extent of how far with block lambdas can go.

More Parameters

Say that we want to provide more parameters than 0 or 1. How do we get more? Well, directly, we can’t, but because of iterable unpacking, we can do essentially the same thing! So, to get two parameters, __enter__() returns a tuple (or list – but a tuple is better) of two objects (i.e. return thing1, thing2).
ASIDE: A tuple is better than a list for a few reasons. First, there are no brackets required to make a tuple, so it looks cleaner. Second, a tuple is immutable, so it can’t be accidentally messed with. If you want to design it so that an argument becomes an out argument, then you could use a list, but there are better, clearer ways to do this.
Let’s make this a little more clear with an example:

class MyContextManager:
    def __enter__(self):
        return 1, 2, 3, 4

    def __exit__(self, type, exc, tb):
        …

with MyContextManager() as args:
    one, two, three, four = args
    print(one)
    print(two)
    print(three)
    print(four)

This context manager is kind of worthless, since it simply gives back the values you gave it as a tuple to use in the with block, but it does demonstrate how multiple arguments can be provided to the “lambdas” in the with blocks. I expect that the name args will generally be used unless a better name can be given for the specific situation. Also, you can see that it’s easy to separate the arguments into something more useful in a single unpacking line.

Return Values

As I mentioned earlier in the aside about using tuples, you could technically pass in a list as the arguments use it to take care of the return value. This could be done by appending a return value to the end of the list or reassigning an item already in the list. This isn’t all that terrible, mechanically, but it’s confusing, error-prone, and doesn’t read well.
So, I’ve got some alternatives, starting with my least favorite.

Return Parameter

With this technique, one of the arguments that is passed in (preferably the last) is a small mutable data store. It probably has a method along the lines of _return(self, value) that can be called within the with block. Calling it will set a value on the object which, if stored on the context manager, can be looked at in the __exit__() method.

class OutParameter:
    def _return(self, value):
        self.return_value = value

class MyContextManager:
    def __enter__(self):
        self.return_value = OutParameter()
        return 1, 2, 3, 4, self.return_value

    def __exit__(self, type, exc, tb):
        do_something_with(self.return_value.return_value)

with MyContextManager() as args:
    one, two, three, four, out = args
    …
    out._return(value)

This works, isn’t too confusing, but seems to take up just a little excess space. Let’s change it up just a little bit.

Function Signature

Instead of passing the object that will take on the return value as one of the arguments, you actually pass it as all the arguments. You see, it’s possible to unpack any sequence object, and making one of those is super easy. I call this class FunctionSignature, since it encapsulates both the arguments as well as the return value, but I’m not too keen on the name. I’d happily accept some suggestions in the comments.

from collections.abc import Sequence
class FunctionSignature(Sequence):
    def arguments(self, *args):
        self.args = args
        return self

    def _return(self, value):
        self.return_value = value

    def __getitem__(self, index):
        return self.value[index]

    def __len__(self):
        return len(self.value)

class MyContextManager:
    def __enter__(self):
        self.sig = FunctionSignature()
        return self.sig.arguments(1, 2, 3, 4)

    def __exit__(self, type, exc, tb):
        do_something_with(self.sig.return_value)

with MyContextManager() as args:
    one, two, three, four = args
    …
    args._return(value)

Now, if I were actually implementing FunctionSignature, I’d override all of the Sequence methods (possibly without even inheriting from Sequence) to redirect to the internal tuple so that the less efficient mixin methods aren’t used. But, for the sake of conciseness, I did the shortest, easiest way to create a sequence.
Overall, this didn’t save us much space. In fact, in total it was longer due to the larger definition of FunctionSignature versus OutParameter, but that length is a one-time thing versus the length of all the with blocks that use it. And we only saved a few characters on one line within the with block. Still, I like this idea more overall, since I feel like it puts a smaller mental burden on the user writing the with block. I could be wrong. What do you think?

Use the contextmanager Decorator

When making a context manager, I highly recommend using contextmanager decorator in the contextlib module. It makes writing the manager much easier, and it still works just fine with OutParameter and FunctionSignature.

Limitations

Now, despite the fact that I’ve removed a few limitations from using with] blocks for multi-line lambdas, there are still plenty of limitations, one of which was actually introduced by adding the ability to 'return' values.

Can Only Be Called Once

Normally, a function that uses a lambda is capable of using that lambda as many times as it wants. When it comes to a with block lambda, the block is only called once; between one call to __enter__() and one call to __exit__().

There is No Function Object Involved

The block of code in a with block is not turned into a callable object that can be passed around. This means it can’t be passed further into more specific functions and such.

Harder to Reuse Existing Functions

with blocks don’t accept functions and lambdas. So you can’t just pass an already-defined function in to be used. To use a function, with arguments and return values, it would look like this:

with cm as args:
    args._return(some_func(*args))

That’s way more than just cm(some_func). Now, you could define the function in such a way that, if it does receive a function or lambda, it doesn’t go into “context manager mode”.

The Highlander Problem

There can be only one. Only one “lambda” can be passed in using a with block. I’m pretty sure there’s no way around this. If there is a way, I’m certain that it’s nearly impossible to read.
Obviously, actual lambdas can be passed in, but that doesn’t help the situation a ton.

Returns That Don’t Return

When you call the _return() method, it doesn’t exit out of the block. You’ll have to use other control flow techniques to make sure that’s the last thing called. Also, please don’t try and be clever by purposely having code run after the return; that will just confuse readers more.

with Statements Aren’t Expressions

Normal functions that take lambdas are able to return something that can be used later. with statements don’t return anything that can be used outside of themselves. This hurts functional programming, since it makes the code obviously impure.

Outro

So that’s the craziness that is multi-line lambdas using with blocks. Remember the disclaimer at the top when it comes to making comments. I’m particularly proud of this one, even if it’s usually a bad idea. Thinking around languages like this is something I particularly enjoy doing, and I hope there are others out there that do, too.