Agenda¶
- Motivation
- Core Ideas
- Basic Application
- Common Patterns
- Takeaways
Motivation¶
Computers do exactly what you program them to do.
If you aren’t getting the results you expect, look for a simpler model.
Three Targets For Simpler Programs¶
- Make software easier to understand by breaking it into pieces that can be understood separately
Three Targets For Simpler Programs¶
Make software easier to understand by breaking it into pieces that can be understood separately
Enable the ability to maintain, reuse and extend software (or bits of software)
Three Targets For Simpler Programs¶
Make software easier to understand by breaking it into pieces that can be understood separately
Enable the ability to maintain, reuse and extend software (or bits of software)
Avoid extra work from lack of forethought towards methods of engagement with software
Three Targets For Simpler Programs¶
Make software easier to understand by breaking it into pieces that can be understood separately
Make it easier to collaborate with others by making your code understandable OR make it easier for yourself when you ultimately need to revisit your code after six months, a year, or more working on a different project.
Enable the ability to maintain, reuse and extend software (or bits of software)
Avoid extra work from lack of forethought towards methods of engagement with software
Three Targets For Simpler Programs¶
Make software easier to understand by breaking it into pieces that can be understood separately
Enable the ability to maintain, reuse and extend software (or bits of software)
By breaking software into pieces that can be understood and operate independently, we hope they are not only easier to maintain, but portable from project to project. If your research continues to build off prior work, why doesn't your code?
Avoid extra work from lack of forethought towards methods of engagement with software
Three Targets For Simpler Programs¶
Make software easier to understand by breaking it into pieces that can be understood separately
Enable the ability to maintain, reuse and extend software (or bits of software)
Avoid extra work from lack of forethought towards methods of engagement with software
Less work good 👍👍
Core Ideas¶
Core Ideas¶
- Decomposable – can be broken down into modules to reduce complexity and allow teamwork
Core Ideas¶
- Decomposable – can be broken down into modules to reduce complexity and allow teamwork
- Composable – enable code to be reused in many places
Core Ideas¶
- Decomposable – can be broken down into modules to reduce complexity and allow teamwork
- Composable – enable code to be reused in many places
- Understandable – one module can be examined, reasoned about, developed, etc. in isolation
Core Ideas¶
- Decomposable – can be broken down into modules to reduce complexity and allow teamwork
- Composable – enable code to be reused in many places
- Understandable – one module can be examined, reasoned about, developed, etc. in isolation
- Continuity – a small change in the requirements should affect a small number of modules
Core Ideas¶
- Decomposable – can be broken down into modules to reduce complexity and allow teamwork
- Composable – enable code to be reused in many places
- Understandable – one module can be examined, reasoned about, developed, etc. in isolation
- Continuity – a small change in the requirements should affect a small number of modules
- Isolation – an error in one module should be as contained as possible
Core Ideas¶
Cohesion – internal consistency¶
Generally, we want to create separation of concerns
Methods / Functions¶
- Compute a value, let client decide what to do with it
- Observe or mutate; don’t do both
- Hard to follow? Refactor into multiple simpler methods
Data Containers (abstract data types)¶
- Provide a single abstraction, represent a single concept
- Redesign your class if it violates this principal
from copy import deepcopy
def observe(data: dict[str, str]) -> dict[str, str]:
# No modifications to the dataframe
updated = deepcopy(data)
updated["hello"] = "world"
return updated
def mutate(data: dict[str, str]) -> None:
# Modify the original
data["hello"] = "world"
# Create empty dict for testing
empty = {"name": "eva"}
# Demonstrate observe
new_data = observe(empty)
print("observed")
print("original is still the same", empty)
print("new data includes the addition", new_data)
print()
# Demonstrate mutate
print("mutated")
print("original was", empty)
mutate(empty)
print("original now is", empty)
observed original is still the same {'name': 'eva'} new data includes the addition {'name': 'eva', 'hello': 'world'} mutated original was {'name': 'eva'} original now is {'name': 'eva', 'hello': 'world'}
def new_dataset_from_additions(
existing_data: dict[str, str],
new_data: dict[str, str],
) -> dict[str, str]:
# copy existing and new
# add them together into a new object
# return new object
return {}
def update_dataset_with_additions(
existing_data: dict[str, str],
new_data: dict[str, str],
) -> None:
# add new data to existing
pass
Well designed software should try to minimize the mental effort required to reason about it.
If there are too complex of functions, break them out to think about and develop one at a time. If there is too much coupling between functions or modules, try to reduce coupling.
Basic Application¶
Basic Application¶
You have {some process} in which data is monitored by different sensors. Data collected from sensors stored in CSV files. You want to collect all the data in a single Google Sheet for plot creation and other post-processing. Only keep data columns that correspond to headers specified in the Google sheet.
Basic Application¶
Planned Work¶
- Define a basic generalized specification for reading and writing data for both CSV and Google Sheets.
- Implement both “file handling” specifications (i.e. two separate functions / classes)
- Write a function to remove duplicate data points.
- Write a function to subset / select the specific data points we want to save.
Basic Application¶
Decomposable - split the reading and writing of files out to their own handlers in case you need to add or change storage locations or data formats later.
Composable -- this is a toy problem but you might imagine that the same file reading and writing code can be used in other projects with minimal change.
Understandable -- each part of the process is separate (reading separate from processing)
Continuity -- if you need to change which data points are selected, there is a single place to update
Isolation -- an error in Python will always tell you which function it came from, these separate functions and handlers will help debug when things go wrong.
Basic Application¶
Let's step through an existing notebook and talk about how we might convert a notebook to a collection of independent functions.
Common Patterns¶
Common Patterns¶
For the next ten minutes, with a partner, look through:
https://github.com/faif/python-patterns¶
Think about how any of these patterns might be useful to a problem you recently encountered.
A first one to look at might be the factory
which is very useful when trying to have a standard interface to create different objects or functions using a parameter.
Ex: you are processing a bunch of different image file formats and you need to find the appropriate file reader for each
As you are doing this, make note of patterns you find interesting or particularly useful and why. Or take note of patterns you have never seen before.
Takeaways¶
Three Targets For Simpler Programs¶
Make software easier to understand by breaking it into pieces that can be understood separately
Enable the ability to maintain, reuse and extend software (or bits of software)
Avoid extra work from lack of forethought towards methods of engagement with software
Well designed software should try to minimize the mental effort required to reason about it.
If there are too complex of functions, break them out to think about and develop one at a time. If there is too much coupling between functions or modules, try to reduce coupling.
Learn and apply coding patterns. There are many resources to look up common patterns.
A good way to learn them is to use the example code commonly given and change, add, or remove values and see what happens.
Credit¶
Much of the content of this talk comes from: