Managing Dynamic Contexts in Python

Apr 14, 2018 11:14 · 308 words · 2 minutes read Python

In Python we have context managers to deal with concepts like managing the lifecycle of an open resource and closing it automatically when it’s no longer needed. This makes it easier to work with resources like files because you can guarantee that the file is closed properly even if your code encounters some error condition.

with open("some_file.txt", "r") as some_file:
    # Do some operation here

# The file will always be closed correctly

Sometimes, you can get into the situation where you don’t have the list of list of resources available ahead of time. Take the following example: we want to take a list of files from the user, open them, and pass them to some method that requires a list of open files.

list_of_files_provided_by_user = sys.argv[1:-2]

# How to manage the context here?    
method_that_needs_open_files(?)

How can we dynamically generate this list of contexts?

contextlib provides a mechanism for this called ExitStack. Exit stack creates a context from a stack of other contexts, allowing us to dynamically add new contexts to our stack and exit them all at once. Using our example:

list_of_files_provided_by_user = sys.argv[1:-2]
final_file_name = sys.argv[-1]

with ExitStack() as context_stack:
    method_that_needs_open_files([
        context_stack.enter_context(open(filename)) for filename in
            list_of_files_provided_by_user])

When method_that_needs_open_files returns, the context will close all files added to the stack. This utility is also available in contextlib2 if you need Python 2 support.

What about contextlib.nested

This function has been deprecated since 2.7. The preferred functionality if you DO NOT need dynamic context management (as in our above example) is to nest with statements via:

with open(filename_one, "r") as first, open(filenam_two, "r") as second:
    # Woth with both open files at once

Obviously this form does not support the use case of a dynamic list of context managers, thus the need for ExitStack. You can read more about the deprecation of this method in the offical documentation