🐍 Functionally Zen

 > “Keep it simple, stupid.” The KISS principle was coined by Kelly Johnson, lead engineer at Lockheed Skunk Works. It is echoed in the third tenet of the Zen of Python: In that same spirit, I propose these additional tenets in our pursuit of Python coding simplicity: Finally: these ideas are generally true. There are likely specific cases where they are not true. Special cases aren’t special enough to break the rules. ## The terminology Specificity is the soul of narrative. Here are some definitions to be more clear and precise as we discuss the rationale for these tenets: 1. Idiomatic: particular to or emblematic of a particular language.‍ 2. Side effects: interactions with external systems, such as the file system, a database, a user (via an interface), or a third-party API.‍ 3. Immutable data structure: a data structure that, once set, never changes. If changes are needed, they are made by replacing that data structure with a new, updated one. Python’s tuple, frozenset, and the upcoming frozendict (in Python 3.15) are examples of immutable data structures‍ 4. Pure function: a function that, given a set of inputs, always produces the same outputs, free of side effects.‍ 5. State: data, but in this case specifically data bound to an instance of a class, which is initialized by invoking the class’s constructor.‍ 6. Methods: a special type of function that is bound to a class and its state.‍ 7. Instance methods: methods that are bound to a constructed/initialized instance of a class; they have access to the instance’s initialized state.‍ 8. Class methods: methods that are bound directly to the class, and thus do not have access to an initialized state.‍ 9. Functional programming: a way of organizing and thinking about code with functions. Though some may see it in opposition to object-oriented programming (OOP), the two ways of organizing code can co-exist. For example, you can write a class whose methods are pure functions and whose state consists of immutable data structures.‍ 10. Functional programming primitives: these are the core concepts that power functional programming, including pure functions and immutable data structures. Other examples: map(), filter(), and reduce().‍ 11. Mocking: used here in a general sense to cover all the various testing methods—spies, test doubles, stubs, etc.—of controlling and/or inspecting dependencies outside the system under test. ## The tenets ### Idiomatic code is simpler than non-idiomatic code All programming languages have idioms that arise from their particular syntax, design goals, standard libraries, etc. Sometimes those idioms can conflict or at least be in tension. For example, many functional (or aspirationally functional) languages have adopted the map/reduce idiom: ```plain text > [1, 2, 3].map(x => x * 2) [2, 4, 6] ``` Python took a different approach, as its creator, Guido van Rossum, once stated: > “I value readability and usefulness for real code. There are some places where ```plain text map() ``` ```plain text filter() ``` Consequently, the idiomatic approach for a simple mapping (transforming one element to another) in Python is not `map()` but rather a list comprehension: ```plain text >>> [x * 2 for x in [1, 2, 3]] [2, 4, 6] ``` Though map() is more typical for other languages, list comprehensions will be more familiar to Python developers, particularly those who have not been exposed to other implementations of map(). ### Data is simpler than functions Data is the virtual equivalent of “no moving parts”. No moving parts is intrinsically simpler than something—that is, a function—which does have moving parts. This tenet is particularly true when the data in question is stored in an immutable data structure. I once worked on a medical app that took in parameters—age, weight, and lab results—and returned the proper dosage. The dosage calculation came from a medical textbook and was moderately complex. My initial thought was to implement the calculation as a function; however, my wiser coworker realized that the textbook also had lookup tables. When the textbook had been written, dosages were calculated manually, so these lookup tables saved physicians from the error-prone work of doing their own calculations. The textbook converted what would have been a function into a data structure. We implemented those lookup tables as a three dimensional dictionary and this: Became this: As the dictionary didn’t have any logic that we’d implemented, we didn’t need to unit test anything. ### Pure functions are simpler than impure functions Since pure functions are consistent in the value they return, we never have the cognitive overload of considering how external systems might change how the function works. For example: ```plain text >>> def double(x):... return x * 2... >>> double(2) 4 ``` We know that invoking double(2) will always result in 4 being returned. On the other hand: ```plain text >>> import requests >>> def fetch_double(x):... resp = requests.post('https://example.com/doubler', data={'x': x}) ... body = resp.json() ... return body["answer"] ... ``` We know that invoking fetch_double(2) will not always return 4. Sometimes it might. Sometimes it might return a 500 HTTP error. Sometimes it may raise a JSONDecodeError. If a developer makes a mistake, the call may even return 3.14! Consequently, pure functions are simpler than impure functions. Furthermore… ### Pure functions are simpler than classes The whole point of classes is to bundle together the state and the behavior that operates on that state. Consequently, a class is by definition more complex than a pure function, which is only behavior. Pure functions do not require the cognitive overhead of tracking how the state has changed, in addition to reading through the behavior. In this somewhat contrived example, compare this: ```plain text >>> def double(x):... return x * 2... ``` With this: ```plain text >>> class Doubler:... def __init__(self, x):... self.x = x ... def double():... return self.x * 2... ``` The class requires a lot of boilerplate code to scaffold out that state, complexity that the pure function form does not need. Furthermore, any developer reading the code will need to remember what x was set to at initialization in order to reason about what a call to `double()` will return. ### Impure functions and/or classes are inescapable, so let’s minimize them At some point, we’ll need to interact with things, be they users, a database, or other APIs. This fundamental truth underlies the idea of “functional core, imperative shell”, as outlined by Gary Bernhardt in his Boundaries talk. (You can read “imperative” here to mean “side effects”.) We should try to keep more complex, impure functions a thin layer around simpler pure functions, thereby keeping the overall system as simple as possible. Note that an important helper here is the Gang of Four’s assertion: Composing functions lets us separate side effects from the logic that needs to happen to them. We see this principle within Python’s standard library: json.load() could have taken in a file path as a string and opened it, but that would have mixed opening a file (side effect) with marshaling JSON into a Python dictionary (core logic). Instead, we can compose two separate functions: ```plain text >>> import json >>> json.load(open('example.json')) ``` (Yes, if we were writing production code, we’d want to use open as a context manager to get automatic closing of file handles, but the approach above does a better job of illustrating the concept at hand, so Python pedants, put your pitchforks paway … er … away!) ### Constructors without side effects are simpler than those with side effects Typically, a constructor initializes the state when creating an instance from a class. For example: ```plain text >>> class Cat:... def __init__(self, sound='roowwwrrrr'):... self.sound = sound ... def meow(self):... return f'Cat says: {self.sound}'... >>> num_num_cat = Cat(sound='num num') >>> num_num_cat.meow() 'Cat says: num num' ``` Constructors with side effects—that is, which interact with external systems as part of initializing that internal state—force a developer to account for that external system when constructing the object. “Is this external system available?” “What happens to the instance being constructed if it’s not available?” For example: ```plain text >>> import requests >>> class ApiClient:... def __init__(self):... self.token = requests.post('https://example.org/token', data={...}) ... ``` In these cases, as described in Impure functions and/or classes are inescapable, so let’s minimize them, it’s simpler to move the interaction with the external system into a separate function, which would live in the imperative shell: ```plain text >>> def get_token(data):... return requests.post('https://example.org/token', data=data) ``` Now we can leverage composition to still initialize the token, but without complicating the constructor with side effects: ```plain text >>> class ApiClient:... def __init__(self, token):... self.token = token ... >>> client = ApiClient(token=get_token(data={...})) ``` One final note on this: constructors with side effects are particularly pernicious because they are poison pills: they don’t just infect their own class, but any classes that depend on the constructor's class. For example: ```plain text >>> class TimeClient:... def __init__(self):... self.token = requests.post('https://example.org/token', data={...}) ... >>> class TimeService:... def __init__(self):... self.client = TimeClient() ... >>> class TimeController:... def __init__(self):... self.service = TimeService() ... ``` Now it’s not just TimeClient() that makes an API call, it’s also TimeService() and TimeController(). (And yes, I’ll allow that dependency injection could also be used to alleviate issues in this particular example.) A practical example of where this “trickle up” effect will cause complexity: tests for both TimeService and TimeController now need to mock the API call. Which brings us to… ### Tests with no mocking are simpler than tests with mocking Mocking requires more code than not mocking. Code is complexity. Therefore, if we can test a unit without any mocks, our code is simpler than if we had to set up mocks. Mocks are required when our unit has dependencies. In other words, when it interacts with external systems. In other words, when it has side effects. In other words… when it is not a pure function. One of the biggest simplicity advantages pure functions have over impure functions and classes is that there are no external dependencies to mock. Going back to our earlier example of a double() pure function versus a fetch_double() impure function, here’s what testing each would look like: ```plain text >>> def test_double():... assert double(2) == 4... ``` ```plain text >>> from unittest import patch >>> def test_fetch_double():... with patch('requests.post') as mock_post: ... mock_post.return_value.status_code = 201... mock_post.return_value.json.return_value = {"answer": 4} ... assert fetch_double(2) == 4... ``` The first test is straightforward, simple, easy to read, and easy to understand. The second test involves knowing a great deal more than 2 × 2 = 4: you must know that successful HTTP requests should return a 2xx status code, that the requests library makes the returned value available via the json() method, the shape of the JSON object returned from the API, and the intricacies of where exactly to patch. Mocking is often a code smell in tests. A large amount of mocking is stinky code; it is an indicator that a test subject’s dependencies have grown too complex. One mitigation strategy—that we've already seen in Constructors without side effects are simpler than those with side effects—would be to pull those dependencies into an imperative shell that could be tested by functional, integration, or end-to-end tests rather than unit tests. ## Wrapping Up Python has a bit of a love/hate relationship with functional programming. Going back to Guido van Rossum's 2013 Slashdot interview: > “So, mostly I don't think it makes much sense to try to add ‘functional’ primitives to Python, because the reason those primitives work well in functional languages don't apply to Python, and they make the code pretty unreadable for people who aren't used to functional languages (which means most programmers).” That said, in the years since that 2013 interview, JavaScript and TypeScript have moved in a more functional-ish direction. As the most-used language(s) in the world, "most programmers" now have some familiarity with those functional programming primitives Guido rejected as “unreadable”. Approached from a pragmatic perspective, those primitives—as demonstrated in the aforementioned tenets—can help steer a Python codebase towards more simplicity. After all: Kyle Adams is a Staff Software Consultant at Test Double who lives for that light bulb moment when a solution falls perfectly in place or an idea takes root.   Letter art spelling out NEAT  Test Double Executive Leadership Team