Nested scopes nits and tidbits

by Uche Ogbuji

Updated. Thanks to "david_given" for pointing out a typo, which is now corrected.



Much of my current late-night hacking (I should be making a couple of big announcements soon) involves pushing the art of XML/SAX processing in Python by using its ever more powerful functional features. In doing so, I've been making more and more use of nested (AKA inner) functions for modularity and some neat approaches to dynamic dispatch. This has also brought me several times into the grey areas of nested scopes. I've come to know the PEP pretty well, but in case it saves anyone time spent probing Python legalese, here is an example that illustrates the behavior of nested scopes. The code can be run as is in Python 2.2 or later. For Python 2.1, add from __future__ import nested_scopes to the top.




g_a = 1 #global scope

def f1():
a = 2
def g():
print "f1/g", a #a is 2
return
g()

def f2():
a = 3
def g():
print "f2/g", a #UnboundLocalError (at runtime)
a = 4
return
g()

def f3():
def g():
global g_a
print "f3/g A", g_a #a is 1
g_a = 5 #Modifying the global
print "f3/g B", g_a #g_a is 5
return
g()
print "f3/g C", g_a #g_a still 5

def f4():
a = 6
def g(a=a): #Yuck. Cumbersome
print "f4/g A", a #No problem. a is 6
a = 7
print "f5/g B", a #Now a is 7
g()
print "f4", a #Back to 6, since int is immutable and

def f5():
a = 8
def g(a):
print "f5/g A", a #No problem. a is 8
a = 9
print "f5/g B", a #Now a is 9
return a
a = g(a)
print "f5", a #a is still 9

f1()
#f2() #Commented out to avoid Exception
f3()
f4()
f5()


When I first ran into the UnboundLocalError problem demoed in f2 I started with the fall-back solution in f4, but 2 considerations turned me off that solution. The main one was ugliness of the keyword stuffing, especially when I wanted several variables in the shared scope. A more minor issue was a need to mutate such variables within the nested function. This is a minor issue because such mutation is rather inelegant and runs counter to the very functional principles underlying nested scopes. Nevertheless, I did have a couple of cases where I wanted to hack in mutation temporarily for some quick and dirtypurpose. It turns out that the f5 approach deals with both issues, with a bonus that mutation is replaced by functional transform. I use tuples to pass back multiple values from the inner function, of course.



I did not show in the listing how using exec or from foo import * can lead to syntax errors, a situation I ran into once, to my great confusion. See the PEP for a terse listing of syntax gotchas. Andrew Kuchling's document mentions the exec gotcha. The usual solution is to use the form exec cmd in globals(), locals(). See the Python Library Ref exec documentation for details (notice: Python 2.4 relaxes the rules a bit on what can be used with exec ... in ...).
This slide mentions the exec gotcha as well as a possible problem with eval.





Side note: in Andrew Kuchling's brief on nested scopes he introduces them by saying:




In Python 2.0, at any given time there are at most three namespaces used to look up variable names: local, module-level, and the built-in namespace. This often surprised people because it didn't match their intuitive expectations. For example, a nested recursive function definition doesn't work:



def f():
...
def g(value):
...
return g(value-1) + 1
...



The function g() will always raise a NameError exception, because the binding of the name "g" isn't in either its local namespace or in the module-level namespace. This isn't much of a problem in practice (how often do you recursively define interior functions like this?)


I read this bit after I'd already used recursive inner functions in several cases, and had been impressed at their expressive power. Just goes to show that the human imagination is not fit to find limits to the usefulness of recursion.



Overall, nested scopes in Python are not as clean as a language purist might wish, but like so much in Python's evolution, they find a comfortable niche between cleanliness and practicality.



2 Comments

david_given
2004-12-15 09:45:44
error in f3 comment?
hola,


I appreciated this article - very simple examples & clear implications of how they can be used.


I am not a python programer, but is this correct:
#begin
def f3():
def g():
global g_a
print "f3/g A", g_a #a is 1
g_a = 5 #Modifying the global
print "f3/g B", g_a #g_a is 5
return
print "f3/g C", g_a #g_a still 5
g()
#end
Specifically, should "#g_a still 5" be "#g_a is 1 becuase we have not called g()"?


thanks
davids

uche
2004-12-15 10:48:42
error in f3 comment?
You're absolutely right. I didn't squint hard enough at the original output:


f1/g 2
f3/g C 1
f3/g A 1
f3/g B 5
f4/g A 6
f5/g B 7
f4 6
f5/g A 8
f5/g B 9
f5 9


Which, of course is not what I was trying to illustrate. I've updated the script (and this article) and verified the correct output:


f1/g 2
f3/g A 1
f3/g B 5
f3/g C 5
f4/g A 6
f5/g B 7
f4 6
f5/g A 8
f5/g B 9
f5 9


Thanks for the correction.