Programming in Python

A Python variable is a name bound to an object; assignment never copies the object. b = a makes two names for one object. Whether that matters depends on the type: immutable types (int, float, bool, str, tuple) can never change in place, so any 'change' rebinds the name to a new object. Mutable types (list, dict, set) can be changed in place, and every name bound to the object sees the change.

Convention: Python 3 semantics throughout. / is true division and // is floor division; integers are unbounded; indexing starts at 0; dictionaries keep insertion order. print separates its arguments with a single space and ends with a newline (sep= and end= change these). The output shown is exactly what appears on screen.

Aliasing versus copying:

a = [1, 2, 3]
b = a            # alias: same object
b.append(4)
print(a, a is b)
c = a[:]         # shallow copy: new list
c.append(5)
print(a, c)

Output:

[1, 2, 3, 4] True
[1, 2, 3, 4] [1, 2, 3, 4, 5]

Strings are immutable, so += rebinds only one name:

s = "abc"
t = s
s += "d"
print(s, t)

Output: abcd abc. A tuple is immutable, but if it holds a mutable object, that inner object can still change:

x = (1, [2, 3])
x[1].append(4)
print(x)

Output: (1, [2, 3, 4]). Assigning x[0] = 9 would raise a TypeError.

== compares values, is compares identity (same object). [1, 2] == [1, 2] is True, but [1, 2] is [1, 2] is False because two list displays make two objects. Whether is is true for two equal numbers or strings depends on the Python implementation, so use is only for None or for objects whose identity the program itself sets up. A shallow copy of a list of lists shares its inner lists:

import copy
q = [[1, 2], [3]]
sh = q.copy()
dp = copy.deepcopy(q)
q[0].append(9)
print(sh, dp)

Output: [[1, 2, 9], [3]] [[1, 2], [3]].

/ is always true division and returns a float (8 / 2 is 4.0, 7 / 2 is 3.5). // is floor division: it rounds toward negative infinity, not toward zero. % is defined so that the identity below always holds, which means the result of % takes the sign of the divisor (or is 0). If either operand is a float, // and % return a float (10.0 // 4 is 2.0, -7.5 % 2 is 0.5). Integers are unbounded in size.

int(x) on a float truncates toward zero, unlike //. round(x) rounds to the nearest integer and, for an exact half, to the even neighbour. divmod(a, b) returns the pair (a // b, a % b):

print(int(-3.5), -7 // 2, round(2.5), round(3.5), divmod(-7, 2), 10.0 // 4)

Output: -3 -4 2 4 (-4, 1) 2.0.

Operators on one level group left to right, except **. ** binds tighter than a unary minus on its left but not on its right (2 ** -1 is 0.5):

print(2 ** 3 ** 2)    # 2 ** 9
print(-2 ** 2)        # -(2 ** 2)
print((-2) ** 2)
print(2 ** -1)
print(1 + 2 * 3 ** 2 % 5)

Output:

512
-4
4
0.5
4

The last line: 3 ** 2 = 9, then 2 * 9 = 18 and 18 % 5 = 3 (same level, left to right), then 1 + 3 = 4.

Comparisons chain: a < b > c means (a < b) and (b > c), with b evaluated once. Shifts and bitwise operators sit below + -:

print(1 < 3 > 2, 3 > 2 == 2, (3 > 2) == 2)
print(2 + 3 << 1, 6 & 3 | 4)

Output:

True True False
10 6

3 > 2 == 2 is (3 > 2) and (2 == 2), which is True; with brackets, (3 > 2) == 2 compares True with 2, which is False. 2 + 3 << 1 is 5 << 1 = 10; 6 & 3 | 4 is 2 | 4 = 6.

Truthiness and short-circuit logic: 0, 0.0, '', [], {}, set(), () and None are false; everything else (including the string '0') is true. a or b returns a if a is true, else b; a and b returns a if a is false, else b. They return one of the operands, not necessarily a bool, and they do not evaluate b when a already decides the result:

print(bool([]), bool(0), bool("0"), 0 or 5, 3 and 0, None or "a")

Output: False False True 5 0 a.

Indexing starts at 0; negative indices count from the end (-1 is the last element). A slice s[i:j:k] takes indices i, i+k, ... up to but not including j. Omitted i, j default to the ends in the direction of k. Out-of-range slice bounds are clipped silently (no error), but an out-of-range single index raises IndexError.

L = [0, 1, 2, 3, 4, 5, 6]
print(L[-1], L[1:4], L[::-1], L[::2])
print(L[-3:], L[5:2], L[2:-2], L[10:])

Output:

6 [1, 2, 3] [6, 5, 4, 3, 2, 1, 0] [0, 2, 4, 6]
[4, 5, 6] [] [2, 3, 4] []

append versus extend, and the None trap:

m = [1, 2]
m.append([3, 4])
print(m, len(m))
n = [1, 2]
n.extend([3, 4])
n.extend("ab")
print(n)
w = [3, 1, 2]
r = w.sort()
print(r, w)

Output:

[1, 2, [3, 4]] 3
[1, 2, 3, 4, 'a', 'b']
None [1, 2, 3]

A string is an iterable of characters, so extend("ab") adds 'a' and 'b' separately.

List comprehension [expr for v in iterable if cond] builds a new list; the condition filters, and expr is evaluated for each kept v:

print([i * i for i in range(6) if i % 2 == 0])

Output: [0, 4, 16]. list("abc") is ['a', 'b', 'c']; tuple([1, 2]) is (1, 2); the one-element tuple needs a comma, so (1,) is a tuple while (1) is just the integer 1. Unpacking: a, *b = (1, 2, 3) gives a = 1, b = [2, 3].

Strings are immutable sequences, indexed and sliced exactly like lists. Methods return new strings and never modify the original:

s = "Hello, World"
print(s[1:5], s[::-1], s.lower(), s.find("o"), s.find("z"), s.count("l"))
print(s.replace("l", "L"), s.split(", "), "-".join(["a", "b", "c"]))
print("  hi ".strip() + "|")

Output:

ello dlroW ,olleH hello, world 4 -1 3
HeLLo, WorLd ['Hello', 'World'] a-b-c
hi|

find returns the index of the first occurrence, or -1 if absent (while index would raise an error). x in s tests for a substring: 'a' in "banana" is True.

* repeats a sequence: "ab" * 3 is 'ababab', [1, 2] * 2 is [1, 2, 1, 2]. Strings, lists and tuples compare lexicographically: element by element from the left, and if one runs out first it is the smaller. Characters compare by code point, so every uppercase letter is smaller than every lowercase letter. "abc" < "abd", "Z" < "a", [1, 2] < [1, 3] and (1, 2) < (1, 2, 0) are all True.

A dictionary maps hashable keys to values: numbers, strings, and tuples whose items are all hashable can be keys; lists, dictionaries and sets cannot. Keys are unique, and in Python 3.7+ a dictionary iterates in insertion order. Assigning to an existing key replaces its value but keeps its original position; deleting a key and inserting it again puts it at the end. d[k] raises KeyError if k is missing, whereas d.get(k) returns None and d.get(k, default) returns default. k in d tests keys, not values.

The standard counting idiom:

d = {}
for ch in "banana":
    d[ch] = d.get(ch, 0) + 1
print(d, list(d.keys()), d.get("z"), d.get("z", -1))
for k, v in d.items():
    print(k, v)

Output:

{'b': 1, 'a': 3, 'n': 2} ['b', 'a', 'n'] None -1
b 1
a 3
n 2

The order b, a, n is the order in which each letter first appeared.

Changing the values of existing keys while looping over a dictionary is safe. Adding or deleting a key, so that the dictionary's size changes during the loop, raises RuntimeError: dictionary changed size during iteration. To add or delete keys, loop over a copy of the keys, for k in list(d):.

d = {"a": 1, "b": 2}
for k in d:
    d[k] += 10
print(d)
print(list(d.items()))

Output:

{'a': 11, 'b': 12}
[('a', 11), ('b', 12)]

A set is an unordered collection of distinct hashable elements; duplicates vanish (len({1, 2, 2, 3, 3, 3}) is 3). The empty set must be written set(), since {} is an empty dictionary. s.add(x) adds an element (no effect if present); s.remove(x) raises KeyError if x is absent, while s.discard(x) does not. Set operations:

if / elif / else runs the first branch whose condition is true. for v in iterable visits the items in order; range(start, stop, step) produces start, start+step, ... while strictly before stop (after it in the negative direction if step < 0). while cond repeats while cond is true. break leaves the innermost loop; continue skips to its next iteration. A loop's optional else block runs only if the loop finished without break.

Trace this loop with break, continue and else:

for i in range(5):
    if i == 1:
        continue
    if i == 4:
        break
    print(i, end=" ")
else:
    print("done")
print()

Output: 0 2 3 followed by a newline. The else is skipped because break fired at i = 4. (end=" " replaces the newline that print normally adds with a space.)

while also has an else that runs when the condition becomes false normally:

i = 0
while i < 3:
    i += 1
else:
    print("else", i)

Output: else 3. And for i in range(2, 10, 3) visits 2, 5, 8.

Simultaneous assignment. In a, b = b, a + b the whole right side is evaluated first, using the old values, and only then are both names bound:

a, b = 0, 1
for _ in range(5):
    a, b = b, a + b
print(a, b)

Output: 5 8. In the same way, x, y = y, x swaps two variables.

Arguments can be passed by position or by keyword. A parameter with a default may be omitted. Parameters after a bare * are keyword-only. A function without return returns None:

def g2(a, b=2, *, c=3):
    return (a, b, c)

print(g2(1), g2(1, c=5), g2(b=7, a=0))

Output: (1, 2, 3) (1, 2, 5) (0, 7, 3).

*args collects extra positional arguments into a tuple, and **kw collects extra keyword arguments into a dict. In a call, * unpacks an iterable into positional arguments and ** unpacks a dict into keyword arguments:

def h(a, *args, **kw):
    return a, args, kw

print(h(1, 2, 3, x=4))
print(h(*[5, 6], **{"y": 7}))

Output:

(1, (2, 3), {'x': 4})
(5, (6,), {'y': 7})

The mutable-default trap. Default values are evaluated once, when the def statement runs, not on each call. A mutable default is therefore shared by all calls that omit it:

def f(x, l=[]):
    l.append(x)
    return l

print(f(1), f(2), f(3, [9]), f(4))

Output: [1, 2, 4] [1, 2, 4] [9, 3] [1, 2, 4]. The three calls that omit l all return the same list object, which ends as [1, 2, 4] before print runs. f(3, [9]) uses a fresh list passed in, so it is unaffected.

Scope (LEGB). A name is looked up in the Local scope, then any Enclosing function, then Global, then Built-in. Assigning to a name inside a function makes it local to that function unless declared global or nonlocal. Reading a global needs no declaration. global x rebinds the module-level name; nonlocal x rebinds the name in the nearest enclosing function:

x = 10
def p():
    x = 5
    def q():
        nonlocal x
        x += 1
        return x
    return q()
print(p(), x)

Output: 6 10. q changes p's x (5 to 6); the global x stays 10.

Closures. An inner function keeps access to the enclosing variables it uses, and it sees their current value when it is called, not the value when it was defined:

def counter():
    c = 0
    def inc():
        nonlocal c
        c += 1
        return c
    return inc

k = counter()
print(k(), k(), k())

Output: 1 2 3. The late-binding effect:

def mk():
    fs = []
    for i in range(3):
        fs.append(lambda: i)
    return [f() for f in fs]
print(mk())

Output: [2, 2, 2], because all three lambdas read the same i, which is 2 after the loop ends. Writing fs.append(lambda i=i: i) instead stores the current value as a default (defaults are evaluated when the lambda is created), and the output becomes [0, 1, 2].

lambda args: expr is a one-expression function. enumerate(it, start) pairs each item with a counter; zip pairs items and stops at the shortest input; map and filter are lazy, so wrap them in list(...) to see the values. sorted(it, key=f, reverse=True) sorts by f(item) and is stable (items with equal keys keep their original order). min and max accept a key and return the first of equally ranked items. sum(it, start) adds from start. any is true if some item is true; all is true if every item is true.

print(list(enumerate("ab", 1)), list(zip([1, 2, 3], "ab")))
print(list(map(lambda v: v * v, [1, 2, 3])), list(filter(lambda v: v % 2, range(6))))
print(sorted(["bb", "a", "ccc", "dd"], key=len), sorted([3, 1, 2], reverse=True))
print(max(["apple", "fig", "kiwi"], key=len), sum([1, 2, 3], 10), any([0, "", 3]), all([1, []]))

Output:

[(1, 'a'), (2, 'b')] [(1, 'a'), (2, 'b')]
[1, 4, 9] [1, 3, 5]
['a', 'bb', 'dd', 'ccc'] [3, 2, 1]
apple 16 True False

'bb' stays before 'dd' because both have length 2 and the sort is stable.

Generators. A function containing yield returns a generator: each next() runs the body up to the next yield and hands back that value. A generator can be consumed only once:

def gen(n):
    for i in range(n):
        yield i * i

gg = gen(3)
print(next(gg), list(gg), list(gg))

Output: 0 [1, 4] []. next took 0, the first list took the rest, and the second list found the generator exhausted.

Classes. self.name is first looked up on the instance, then on the class. A class attribute is shared by all instances. Mutating it through self (self.items.append(v)) changes the shared object, but assigning through self (self.count += 1) creates a separate instance attribute and leaves the class attribute alone:

class C:
    items = []
    count = 0
    def __init__(self, v):
        self.v = v
        self.items.append(v)
        self.count += 1

p = C(1)
q = C(2)
print(C.items, p.items is q.items, C.count, p.count, q.count)

Output: [1, 2] True 0 1 1.

A recursive function needs a base case that returns without recursing, and a recursive case that moves toward it. Each call gets its own local variables. Statements placed before the recursive call run on the way down; statements placed after it run on the way back up, in reverse order of the calls.

Pre-order versus post-order printing:

def r(n):
    if n <= 0:
        return 0
    print(n, end=" ")
    return n + r(n - 2)
print(r(7))

def h2(n):
    if n == 0:
        return
    h2(n - 1)
    print(n, end=" ")
h2(4)
print()

Output:

7 5 3 1 16
1 2 3 4 

For r(7): it prints 7, 5, 3, 1, then r(-1) returns 0, and the return values add up as 7 + 5 + 3 + 1 + 0 = 16. That sum is printed after the numbers, on the same line, because each of them ended with a space instead of a newline. In h2, the print comes after the recursive call, so the numbers appear in increasing order.

For a function with two recursive calls, such as

def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)
print(fib(6))

build a table of values from the base cases up: fib(0)=0, fib(1)=1, fib(2)=1, fib(3)=2, fib(4)=3, fib(5)=5, fib(6)=8. The output is 8. Working bottom-up is faster and safer than expanding the whole call tree.