ted@tedneward.com | Blog: http://blogs.tedneward.com | Twitter: tedneward | Github: tedneward | LinkedIn: tedneward
In this talk, we shall...
Talk about how to get started with Python
Go over some of the key syntax and semantics
Explore just tiny bits of the Python universe
"Python is a programming language that lets you work quickly and integrate systems more effectively."- http://www.python.org
"Python is an easy to learn, powerful programming language. It has efficient high-level data structures and a simple but effective approach to object-oriented programming. Python's elegant syntax and dynamic typing, together with its interpreted nature, make it an ideal language for scripting and rapid application development in many areas on most platforms."
https://docs.python.org/3/tutorial/index.html
(mostly) portable
open-source
extensive platform
"scripting" emphasis
many different ways to obtain bits
get source and build
prepackaged distributions
often ships with other tools
Python comes in two distinct flavors
Python2 (2.7) and Python3 (3.9)
Python2 was deprecated in 2008 (!)
NOTE: Python 3.10 introduced some new features!
grab binary installers from python.org
or build from source
the most barebones option
uses "pip" as the package manager
package and environment manager
Python distribution
collection of OSS packages
uses "conda" as the package manager
"python" or "python3"
-h: Help
-i: Enter interactive mode after script execution
-v: Print message on each module initialization
PYTHONHOME
alternative location to use for Python's install directory
PYTHONPATH
paths to search for modules
PYTHONSTARTUP
file containing Python to execute before executing anything else
object (class)-oriented
dynamically-typed
preferential to words over symbols
moderately convention-based
significant whitespace
standard operator custom definitions
often called "interpreted"
not entirely accurate
either feed scripts to executable directly
arguments passed after script name
or (on *nix) use "she-bang" style at top of script
or use REPL directly to experiment
Hello, Python
#!/usr/bin/env python
# Say hello, Python msg = "Hello, Python!" print(msg)
"#"-prefixed, run to end-of-line
no multi-line comments at all
just use lots of "#"s
introduced upon initialization
not on first reference; must be initialized first
variable names can be any alphanumeric symbol
variables and functions are snake_cased
classes are CamelCased
multiple declaration, assignment permitted
Variables
num = 5 message = "This is a string" this_is_a_really_long_variable = 0 x,y = 0,0 #uninitialized_variable # error!
use multiple assignment for identical initialization
numbers.Number: four subtypes
numbers.Integral (int
): unlimited range
Boolean (bool
) is a subtype of int; True
/False
numbers.Real (float
): machine-dependent range
numbers.Complex (complex
): complex numbers
Sequences: finite ordered sets
Strings (str
): Unicode
Tuples
Bytes (bytes
): immutable array of bytes
Objects (object
): object references
all the usual mathematical operations
/
always yields floating-point division
//
always yields integral ("floor rounding") division
**
exponentiation operator
Numbers
div1 = 5 / 2 # 2.5 div2 = 5 // 2 # 2 square = 2 ** 2 # 4
single-quoted, double-quoted
r
-prefixed (either-quoted) "raw" strings
f
-prefixed "formatted" strings
"""..."""
"here-doc" style
adjacent string literals automatically concat
strings can be indexed
positive integer: position from left
negative integer: position from right
strings can be "sliced"
using [x:y]
syntax, where x or y could be empty
strings are immutable
Strings
str1 = "hello" str2 = 'hello' str3 = r"Long live metal \m/" str4 = f"{str1}. I would like to say {str3}" str5 = """ This is a here-doc. It can span lines. It will continue going until we run into the triple-quote again. It is often used for documentation and simple formatting purposes. """ str6 = "This" "is" "several" "strings" str7 = str4[6:8] # "is"
sequences of arbitrary elements, collected
think of class with fields
but no field names
positionally ordered
destructuring is often done via multiple assignment
idiomatic/stylistic: use _
for ignored/unused elements
Tuples
ted = "Ted", "Neward", 46 print(ted) # ("Ted", "Neward", 46) charlotte = ("Charlotte", "Neward", 39) first, last, age = ted print(first) # "Ted"
ordered sequence of arbitrary (polymorphic) elements
square-bracket [...]
surrounded, comma-separated
mutable, indexable, sliceable
concatenatable via +
Lists
list1 = [1, 2, 3, 4, 5] list2 = [1, "two", 3.0] list3 = list1 + list2 firstlistelement = list1[0] print(firstlistelement) lastlistelement = list1[-1] print(lastlistelement)
unordered sequence of unique arbitrary elements
initialize by...
{}
surrounded, comma-separated
or initialize via set()
function
supports mathematical set operations
-
(diff), |
(or), &
(and), ^
(xor)
Sets
set1 = {1, 2, 3, 4, 5} set2 = {1, 1, 2, 2, 3, 3, 4, 4} print(set2) set3 = {2, 4, 6} set4 = {1, 4, 9, 16} print(set1 & set2) print(set1 - set2) print(set1 | set3)
unordered sequence of key/value pairs
keys are immutable
values are mutable
initialize by...
{}
surrounded, comma-separated
key : value
pairs as arguments
or dict()
function
key=value
pairs as arguments
access:
elements via []
-indexing using key
keys()
- sequence of keys
values()
- sequence of values
items()
- sequence of key:value tuples
Dictionaries (Hashes)
dict1 = { "captain":"Antilles" } dict1["admiral"] = "Ackbar" k = dict1.keys() v = dict1.values() kv = dict1.items() another_dict = dict(name="Ted",age=47) print(another_dict) # {'name':'Ted','age':47}
if
condition :
colon not optional
condition cannot contain assignment
block is indented; no "scope markers"
elif
condition :
else:
If
age = 45 if age > 50: print("You old!") elif age < 18: print("Juvie!") else: print("You not old!") insult = "Boomer!" if age > 50 else "Millennial!"
chain if
conditions for brevity
avoid placing branch on the same line as the condition
use if
/else
as ternary operator
NOTE Only in Python 3.10 or later
match
value :
followed by case
condition :
blocks
literal (case 400:
)
several literals (case 400 | 401 | 402:
)
tuple bindings (case (x, y):
)
lists (!)
wildcard (case _:
)
... and can be "guarded" by if
conditions
Match
http_message = "" match status: case 400: http_message = "Bad request" case 404: http_message = "Not found" case 418: http_message = "I'm a teapot" case _: http_message = "Something's wrong with the internet"
If
# point is tuple match point: case (0, 0): print("Origin") case (0, y): print(f"Y={y}") case (x, 0): print(f"X={x}") case (x, y) if x == y: print(f"Y=X at {x}") case (x, y): print(f"X={x}, Y={y}") case _: raise ValueError("Not a point")
looking for a more "functional" way to do this?
... or still using Python 3.9 or earlier?
option: use a dictionary of functions
Dictionaries of functions
import operator as op calculator = { '+': op.add, '-': op.sub, '*': op.mul, '/': op.truediv } left = int(input()) op = input() right = int(input()) print(calculator[op](left, right))
while
condition :
condition cannot contain assignment
block is indented
break
terminates loop
continue
immediately restarts loop
else:
block executes when condition becomes false
not executed in case of break
While
x = 3 while x > 0: print("x is still more than 0") x = x -1 else: print("x is now " + str(x))
for can only operate on "iterables"
objects that are iteratable
for
iterator in
iterable :
block is indented
break
terminates, continue
moves to next
else:
block executes when iterable exhausts
not executed in case of break
For
for c in range(10): print(c) else: print("Can we see c?", c) print("Can we still see c?", c) message = "Python is interesting" for ch in message: print(ch) attendees = ["Marcel", "Roy", "Jeremy"] for at in attendees: print(at) my_first_tuple = "Ted", "Neward", 47 for t in my_first_tuple: print(t)
use enumerate()
instead of an index variable
use in
to iterate over an iterable
raise
object
throws the object as an exception
exceptions continue up the stack until handled
or the program terminates
try:
sets up guarded block
must have one or more except
s and/or finally
clause
handler clauses evaluate in top-down order
except:
all exceptions
except
type :
only exceptions of type or subtype
except
type "as" value :
value holds reference to exception instance
except
(type, type, ...) :
any of these types
except
(t1, t2, ...) as
value :
any type and hold reference
else:
block executed once try:
completes without error
finally:
block always executed
Exceptions
try: bad_div = 1/0 except ZeroDivisionError: print("Silly boy; you can't do that") else: print("Else!") finally: print("Finally!") try: print("In a new try block") raise Exception() except Exception: print("We just raised this!") print("Onwards we execute...")
exceptions are used throughout Python code; follow suit!
little to no performance penalty to their use
avoid "swallowing" exceptions with bare except
clause
with
expression as
variable :
expression result must support a "custom context manager"
this is a convention that assures the expression result will always be cleaned up after use
most common use is for open()
-returned file objects
part of the "builtins" module
complete list is in Python documentation
automatically always in top-level scope
use dir(__builtins__)
to see complete list
use help()
to discover documentation about any particular one
Conversion:
bin()
, hex()
, oct()
, bool()
, str()
, bytes()
, bytearray()
, int()
, float()
, complex()
, chr()
, list()
, map()
, dict()
, set()
, tuple()
Math operations:
abs()
, divmod()
, pow()
, round()
, sum()
I/O operations:
print()
, input()
, open()
Iterable functions:
all()
, any()
, filter()
, iter()
, len()
, max()
, min()
, range()
, sorted()
Language/platform functions:
id()
, classmethod()
, staticmethod()
, property()
, type()
, breakpoint()
, compile()
, eval()
, exec()
, globals()
, locals()
Full list:
https://docs.python.org/3/library/functions.html
Built-in Functions
print("Hello, Python!") print(dir(__builtins__))
def
name (
parameter(s) ):
indent entire body of function
first line of function is a string -- documentation ("doc-string")
use return
to hand back return value
invoke functions using parens, passing in comma-separated list of arguments
Defining functions
def sayHowdy(): """This function says howdy and returns the same""" print("Howdy!") return "I said howdy" response = sayHowdy() print(response) # "I said howdy" print(sayHowdy.__doc__) # "This function says..."
parameter list can have default values assigned
default values must come after non-defaulted parameters
note that (non-literal) default values are evaluated at time of definition and only once
Default argument values
def sayGoodMorning(language="english"): if (language == "english"): return "Good morning!" else: return f"Sorry, I don't speak {language}" print(sayGoodMorning()) print(sayGoodMorning("french"))
arguments can be "passed-by-name" at point of call
allows for reordering of arguments
positional and keyword can be mixed, if positional arguments come first
Keyword arguments
def saySomething(message="Hello", language="English", times=1): """Prints a message in a language a number of times""" for x in range(times): print(message + " (in " + language + ")") saySomething(language="German", times=3, message="Good Morning")
final formal parameter of the form **name
will be a dictionary instance containing all parameter arguments by name
name can be anything desired; the two ** are the signal
Formal parameters
def speak(**args): print(args) for key, value in args.items(): print("argument " + str(key) + " = " + str(value)) speak(one="1", two="2", three="3")
parameter of the form "*name" is a catch-all list for any arguments after
provides variadic parameter list capabilities
Aribtrary argument lists
def myPrint(prefix, *rest): message = prefix for r in rest: message += " " + r print(message) myPrint("Salutations") myPrint("Salutations", "planet")
can be captured in variables
can be passed as arguments
can be returned from functions
Function literals
fn_lit = saySomething fn_lit("Hello", "English", 2)
syntax:
lambda
argument(s) :
body
keep lambdas short
use full function definitions for more than a single expresison or two
Lambdas (aka closures)
messages = ["Hello", "Guten morgen", "Bonjour"] def capIt(x): return x.upper() print(capIt("hello")) # "HELLO" up_messages = map(capIt, messages) # "HELLO", "GUTEN MORGEN", "BONJOUR" up_messages2 = map(lambda x: x.upper, messages) # same as previous add_one = lambda x: x+1 # same as def add_one(x): return x+1 result = (lambda x: x+1)(2) # 3
succinct way to initialize a sequence via code
relies heavily on lambda-style syntax
considered "higher form" among Pythonistas
Three ways to initialize a list of squares
squares1 = [] for x in range(10): squares1.append(x**2) # or... squares2 = list(map(lambda x: x**2, range(10))) # or... squares3 = [x**2 for x in range(10)]
More Comprehensions
x = [2, 3, 4, 5, 6] y = [] for v in x : y += [v * 5] # y == [10, 15, 20, 25, 30] y2 = map(lambda v : v * 5, x) y3 = [v * 5 for v in x] # y3 == [10, 15, 20, 25, 30]
More Comprehensions
for v in x : if v % 2 : y += [v * 5] y2 = map(lambda v : v * 5, filter(lambda u : u % 2, x)) y3 = [v * 5 for v in x if v % 2]
define a set of results; do not materialize the list
on demand, each element is generated
range()
: function that creates a generator for use
list comprehension is a generator
lightweight compared to lists
Generators
def generator_function(): for i in range(10): yield i for i in generator_function(): print(i) def fibonacci(n): a = b = 1 for i in range(n): yield a a,b = b, a+b
Generally, global variables are bad
but sometimes they're useful and/or necessary
Global variables have interesting semantics
globals can be read from anywhere
... but cannot be modified
... unless declared using the global
keyword
Globals
player = "Fred" def doSomething(): #global player # without this, it's an error # "UnboundLocalError: local variable 'player' referenced before assignment" player = player + " and Jed" print(player) doSomething()
runtime-defined
runtime-accessible
implementation inheritable
all methods dynamically dispatched
class
name :
prefer CamelCase
class can contain zero or more "attributes"
"data attributes" are fields
"function attributes" are methods
fields are simply introduced
methods are just functions defined in the class
Basic syntax
class Person: """Someday this class will represent a human being""" # count is a "class variable" (static) count = 0 # sayHello is a "class method" (static) def sayHello(): print("Hello, world!") print(Person.count) Person.sayHello()
to instantiate, "invoke" the classname
if an __init__
method is present, this will initialize the instance
takes one or more arguments; first is the instance itself ("self")
introduce any instance-specific fields here
fields and methods are on class object (singleton/"static")
instance methods must take explicit "self" parameter
by convention, always called self
fields do not exist unless initialized in initializer
in essence, you're "scripting" the object into existence
__init__
class Person: def __init__(self, fn, ln, a): self.firstName = fn self.lastName = ln self.age = a def code(self): print(self.firstName + " is coding, coding, coding...") ted = Person("Ted", "Neward", 48) ted.code()
property()
function takes getter, setter, and deleter member function
optional docstring
returns an object that "is" the property (which is just a wrapper)
can also be used as a "decorator" on each member function
Using property()
class Monster: def __init__(self, name): self._name = name def getname(self): return self._name def setname(self, value): self._name = value def delname(self): del self._name name = property(getname, setname, delname, "The Monster's name") orc = Monster("Orc") print("The " + orc.name + " attacks you!")
The property decorator
class Item: def __init__(self, name): self._name = name @property def name(self): return self._name @name.setter def name(self, value): self._name = value @name.deleter def name(self): del self._name magicSword = Item("Sword +1") print(magicSword.name + " is such a boring name") magicSword.name = "Sting" # Much better
classes and objects are "open"
meaning, we can add members after definition
use dir()
on class or instance to see all members
capturing method reference into variable captures both instance and method
execute directly as desired
Class manipulation
intern = Person("Joe", "Intern", 20) intern.code() # "Joe is coding, coding, coding..." print(dir(intern)) # (Prints out a lot of methods, most of which we didn't define) internCodeFunction = intern.code internCodeFunction() # "Joe is coding, coding, coding..."
any "special" method is surrounded by double-underscores
__init__
, __eq__
, __hash__
, etc
these are all understood by the interpreter -- do not change semantics
provides features found in other languages
initialization/deinitialization
operator overloading
comparison
interception/resolution
full list is found in Python Language Reference
https://docs.python.org/3/reference/datamodel.html#special-method-names
Method conventions
class Example: def __init__(self): print("Object constructor") def __del__(self): print("Object finalizer/destructor") def __repr__(self): print("Stringified representation of this object") def __eq__(self, other): print("Does this == other?") # See also __lt__, __le__, __gt__, __ge__, __ne__ def __hash__(self): print("Generate a hash code for this object") def __bool__(self): print("returns True or False")
define __str__
for human-readable representation
define __eq__
and __hash__
together
to inherit, put base class(es) after class name before colon
class Student(Person):
multiple base classes are permitted
not common, however
to override a method, simply define a new one
to call up to a base class method, use classname.method()
Python3 classes always inherit from object
some style guides suggest making that explicit
Inheritance
class Student(Person): def study(self, subject): print(self.firstName + " is studying " + subject)
pretty much nothing; everything is public by default
convention:
prefix members with...
one underscore for "protected" (client-inaccessible)
two underscores for "private" (subclass-inaccessible)
the interpreter actually recognizes this convention internally
this can help immensely with subclasses
Python's dynamic nature
everything is an object
dir()
on an object yields a lot of info
Python's functional nature
functions as first class objects
But Python clearly has some additional power buried within
property()
can somehow modify classes?
Dynamic and functional nature
string = "This is a string" print(dir(string)) number = 5 print(dir(number)) # do it on a builtin function, even! print(dir(dir)) class FakeNumber: def __add__(self, other): return 0 fake = FakeNumber() print(fake + 5) # prints '5' #print(fake - 5) # error!
Dynamic and functional nature
# The dictionary: A poor man's class ... person = { 'first': 'Ted', 'last' : 'Neward' } # ... as long as it can take functions as values person["speak"] = lambda: print(person['first'] + " " + person['last']) # ... but it works! person["speak"]()
... and in Python3, every object has a type
can either use __class__
or type()
to get hold of the type
this is a "type object" or "class object"
types are metaclasses: classes about classes
Getting hold of the type object
print(type(3)) # <class 'int'> print(type(['a', 'b', 'c'])) # <class 'list'> print(type((1, 2, 3, 4))) # <class 'tuple'> class Foo: pass obj = Foo() print(obj.__class__) # <class '__main__.Foo'> print(type(obj)) # <class '__main__.Foo'> print(obj.__class__ is type(obj)) # True
... are functions
... that take functions (or classes)
... to do "interesting" things to it
If you're familiar with such terms, decorators allow for method interception
Custom decorator
def restricted(func): def wrapper(*args, **kwargs): if 7 <= datetime.now().hour < 22: return func(*args, **kwargs) else: pass # Hush, the neighbors are asleep return wrapper @restricted def say_whee(): print("Whee!") # Only between 7 and 10!
Custom decorator: Timing functions
import functools import time def timer(func): """Print the runtime of the decorated function""" @functools.wraps(func) def wrapper_timer(*args, **kwargs): start_time = time.perf_counter() # 1 value = func(*args, **kwargs) end_time = time.perf_counter() # 2 run_time = end_time - start_time # 3 print(f"Finished {func.__name__!r} in {run_time:.4f} secs") return value return wrapper_timer @timer def waste_some_time(num_times): for _ in range(num_times): sum([i**2 for i in range(10000)])
use the 3-argument version of type()
name
(string): the name of the class
bases
(tuple of type): the base class(es) of the class
dct
(dict): the class body definitions (__dict__
)
Writing a class at runtime
RFoo = type('RFoo', (), {}) obj = RFoo() print(obj) # <__main__.RFoo object at ...?> RBar = type('RBar', (RFoo,), dict(scotches=100)) obj = RBar() print(obj.scotches) # 100 print(dir(obj)) print(obj.__class__) print(obj.__class__.__bases__) Person = type('Person', (), { 'first' : 'Ted', 'last' : 'Neward', 'speak' : lambda self: self.first + " " + self.last }) ted = Person() print(ted.speak())
Consider object construction (f = Foo()
):
__call__()
of Foo's parent (which is type
)is called
type.__call__()
in turn invokes __new__()
and then __init__()
Foo can define overrides of either of those methods
... or replace the existing implementations
Overriding __new__
class Foo: pass def new(cls): x = object.__new__(cls) x.randomAttribute = 100 return x Foo.__new__ = new f = Foo() print(f.randomAttribute) # 100
We can't change default behavior of type
itself
But we can put a new type
-derivative as the parent of a class's class object
In other words, normally type(type(object)) is type
; we can change that to be our own type
And then we can override/replace that __new__
if we choose
Providing custom metaclass
class Meta(type): def __new__(cls, name, bases, dct): x = super().__new__(cls, name, bases, dct) x.randomAttribute = 100 return x class Foo(metaclass=Meta): pass print(Foo.randomAttribute) # 100
Python Tutorial
https://docs.python.org/3/tutorial/index.html
Python Language Reference
https://docs.python.org/3/reference/index.html
Python Library Reference
https://docs.python.org/3/library/index.html
Real Python website
https://realpython.com
"A Guide to Python's Magic Methods"
https://github.com/RafeKettler/magicmethods/blob/master/magicmethods.pdf
"Writing Idiomatic Python3"
Jeff Knupp
"Effective Python"
Slatkin Brett (Addison-Wesley)
Who is this guy?
Architect, Engineering Manager/Leader, "force multiplier"
Principal -- Neward & Associates
http://www.newardassociates.com
Educative (http://educative.io) Author
Performance Management for Engineering Managers
Author
Professional F# 2.0 (w/Erickson, et al; Wrox, 2010)
Effective Enterprise Java (Addison-Wesley, 2004)
SSCLI Essentials (w/Stutz, et al; OReilly, 2003)
Server-Based Java Programming (Manning, 2000)