Agenda¶
- Motivation
- Core Ideas
- Basic Application
- Common Patterns
- Takeaways
Motivation¶
Computers do exactly what you program them to do.
If you aren’t getting the results you expect, look for a simpler model.
Three Targets For Simpler Programs¶
- Make software easier to understand by breaking it into pieces that can be understood separately
Three Targets For Simpler Programs¶
Make software easier to understand by breaking it into pieces that can be understood separately
Make it easier to collaborate with others by making your code understandable OR make it easier for yourself when you ultimately need to revisit your code after six months, a year, or more working on a different project.
Three Targets For Simpler Programs¶
Make software easier to understand by breaking it into pieces that can be understood separately
Enable the ability to maintain, reuse and extend software (or bits of software)
Three Targets For Simpler Programs¶
Make software easier to understand by breaking it into pieces that can be understood separately
Enable the ability to maintain, reuse and extend software (or bits of software)
By breaking software into pieces that can be understood and operate independently, we hope they are not only easier to maintain, but portable from project to project. If your research continues to build off prior work, why doesn't your code?
Three Targets For Simpler Programs¶
Make software easier to understand by breaking it into pieces that can be understood separately
Enable the ability to maintain, reuse and extend software (or bits of software)
Avoid extra work from lack of forethought towards methods of engagement with software
Three Targets For Simpler Programs¶
Make software easier to understand by breaking it into pieces that can be understood separately
Enable the ability to maintain, reuse and extend software (or bits of software)
Avoid extra work from lack of forethought towards methods of engagement with software
Less work good 👍👍
Core Ideas¶
Core Ideas¶
- Decomposable – can be broken down into modules to reduce complexity and allow teamwork
Core Ideas¶
- Decomposable – can be broken down into modules to reduce complexity and allow teamwork
- Composable – enable code to be reused in many places
Core Ideas¶
- Decomposable – can be broken down into modules to reduce complexity and allow teamwork
- Composable – enable code to be reused in many places
- Understandable – one module can be examined, reasoned about, developed, etc. in isolation
Core Ideas¶
- Decomposable – can be broken down into modules to reduce complexity and allow teamwork
- Composable – enable code to be reused in many places
- Understandable – one module can be examined, reasoned about, developed, etc. in isolation
- Continuity – a small change in the requirements should affect a small number of modules
Core Ideas¶
- Decomposable – can be broken down into modules to reduce complexity and allow teamwork
- Composable – enable code to be reused in many places
- Understandable – one module can be examined, reasoned about, developed, etc. in isolation
- Continuity – a small change in the requirements should affect a small number of modules
- Isolation – an error in one module should be as contained as possible
Core Ideas¶
Cohesion – internal consistency¶
Generally, we want to break down our code into smaller pieces but we should have consistency in design and pattern across all of those pieces.
For example, how you handle errors, how you name variables, how you process data, etc. Changing your coding style from function to function can make it harder to understand and maintain your code.
Well designed software should try to minimize the mental effort required to reason about it.
If there are too complex of functions, break them out to think about and develop one at a time.
Basic Application¶
Basic Application¶
You want to write a program to gather data from a number of sources (databases, web sites, etc.), all of this data has similar structure but may have slightly different formats.
Following our "separate code into small pieces" idea, our first thought might be to write a single function that can gather data from any our sources -- at the very least, you have separated out the data gathering from the data processing.
Basic Application¶
A single function separated out is already an improvement! But, if you have many sources of data, you may want to define and structure your code in a way that makes it easy to add new sources of data. In Python, one way to do this is via "AbstractBaseClasses" (ABCs).
Basic Application¶
Lets start with a toy example where you want to pull price data for a number of items from a number of sellers.
from abc import ABC, abstractmethod
import pandas as pd
class DataSource(ABC):
@abstractmethod
def pull_data(self) -> pd.DataFrame:
pass
@staticmethod
def validate_data(data: pd.DataFrame) -> None:
# Check that all the columns we need are present
if any(
feature not in data.columns
for feature in ["name", "description", "price", "category"]
):
raise ValueError("The dataset does not contain all the necessary columns")
# Check that the price is non-negative
if (data["price"] < 0).any():
raise ValueError("The price cannot be negative")
# etc. etc.
class AmazonDataSource(DataSource):
def pull_data(self) -> pd.DataFrame:
# You likely going to pull data from their API or service
# keep that contained here
# ...
# before you return the data, validate it
DataSource.validate_data(data)
return data
class TargetDataSource(DataSource):
def pull_data(self) -> pd.DataFrame:
# You likely going to pull data from their API or service
# keep that contained here
# ...
# before you return the data, validate it
DataSource.validate_data(data)
return data
Basic Application¶
Abstract Base Classes ABCs are great! They allow you to define a common interface for a group of classes. As well as enforce that the classes that inherit from the ABC implement the methods you define.
As with validate_data you can also attach utility functions that each sub-class may want to use to keep everything together.
But, there are lots of other ways to structure code and other design patterns.
Common Patterns¶
Common Patterns¶
For the next ten minutes, with a partner, look through:
https://github.com/faif/python-patterns¶
Think about how any of these patterns might be useful to a problem you recently encountered.
A first one to look at might be the factory which is very useful when trying to have a standard interface to create different objects or functions using a parameter.
Ex: you are processing a bunch of different image file formats and you need to find the appropriate file reader for each
As you are doing this, make note of patterns you find interesting or particularly useful and why. Or take note of patterns you have never seen before.
Takeaways¶
Three Targets For Simpler Programs¶
Make software easier to understand by breaking it into pieces that can be understood separately
Enable the ability to maintain, reuse and extend software (or bits of software)
Avoid extra work from lack of forethought towards methods of engagement with software
Well designed software should try to minimize the mental effort required to reason about it.
If there are too complex of functions, break them out to think about and develop one at a time. If there is too much coupling between functions or modules, try to reduce coupling.
Learn and apply coding patterns. There are many resources to look up common patterns.
A good way to learn them is to use the example code commonly given and change, add, or remove values and see what happens.
Credit¶
Much of the content of this talk comes from: