Software Design and Modularity¶

Eva Maxfield Brown

UW Information School

Agenda¶

  1. Motivation
  2. Core Ideas
  3. Basic Application
  4. Common Patterns
  5. Takeaways

Motivation¶

Computers do exactly what you program them to do.

If you aren’t getting the results you expect, look for a simpler model.

Three Targets For Simpler Programs¶

  1. Make software easier to understand by breaking it into pieces that can be understood separately

Three Targets For Simpler Programs¶

  1. Make software easier to understand by breaking it into pieces that can be understood separately

    Make it easier to collaborate with others by making your code understandable OR make it easier for yourself when you ultimately need to revisit your code after six months, a year, or more working on a different project.

Three Targets For Simpler Programs¶

  1. Make software easier to understand by breaking it into pieces that can be understood separately

  2. Enable the ability to maintain, reuse and extend software (or bits of software)

Three Targets For Simpler Programs¶

  1. Make software easier to understand by breaking it into pieces that can be understood separately

  2. Enable the ability to maintain, reuse and extend software (or bits of software)

    By breaking software into pieces that can be understood and operate independently, we hope they are not only easier to maintain, but portable from project to project. If your research continues to build off prior work, why doesn't your code?

Three Targets For Simpler Programs¶

  1. Make software easier to understand by breaking it into pieces that can be understood separately

  2. Enable the ability to maintain, reuse and extend software (or bits of software)

  3. Avoid extra work from lack of forethought towards methods of engagement with software

Three Targets For Simpler Programs¶

  1. Make software easier to understand by breaking it into pieces that can be understood separately

  2. Enable the ability to maintain, reuse and extend software (or bits of software)

  3. Avoid extra work from lack of forethought towards methods of engagement with software

    Less work good 👍👍

Core Ideas¶

Core Ideas¶

  1. Decomposable – can be broken down into modules to reduce complexity and allow teamwork

Core Ideas¶

  1. Decomposable – can be broken down into modules to reduce complexity and allow teamwork

  2. Composable – enable code to be reused in many places

Core Ideas¶

  1. Decomposable – can be broken down into modules to reduce complexity and allow teamwork

  2. Composable – enable code to be reused in many places

  3. Understandable – one module can be examined, reasoned about, developed, etc. in isolation

Core Ideas¶

  1. Decomposable – can be broken down into modules to reduce complexity and allow teamwork

  2. Composable – enable code to be reused in many places

  3. Understandable – one module can be examined, reasoned about, developed, etc. in isolation

  4. Continuity – a small change in the requirements should affect a small number of modules

Core Ideas¶

  1. Decomposable – can be broken down into modules to reduce complexity and allow teamwork

  2. Composable – enable code to be reused in many places

  3. Understandable – one module can be examined, reasoned about, developed, etc. in isolation

  4. Continuity – a small change in the requirements should affect a small number of modules

  5. Isolation – an error in one module should be as contained as possible

Core Ideas¶

Cohesion – internal consistency¶

Generally, we want to break down our code into smaller pieces but we should have consistency in design and pattern across all of those pieces.

For example, how you handle errors, how you name variables, how you process data, etc. Changing your coding style from function to function can make it harder to understand and maintain your code.

Well designed software should try to minimize the mental effort required to reason about it.

If there are too complex of functions, break them out to think about and develop one at a time.

Basic Application¶

Basic Application¶

You want to write a program to gather data from a number of sources (databases, web sites, etc.), all of this data has similar structure but may have slightly different formats.

Following our "separate code into small pieces" idea, our first thought might be to write a single function that can gather data from any our sources -- at the very least, you have separated out the data gathering from the data processing.

Basic Application¶

A single function separated out is already an improvement! But, if you have many sources of data, you may want to define and structure your code in a way that makes it easy to add new sources of data. In Python, one way to do this is via "AbstractBaseClasses" (ABCs).

Basic Application¶

Lets start with a toy example where you want to pull price data for a number of items from a number of sellers.

In [1]:
from abc import ABC, abstractmethod

import pandas as pd


class DataSource(ABC):
    @abstractmethod
    def pull_data(self) -> pd.DataFrame:
        pass

    @staticmethod
    def validate_data(data: pd.DataFrame) -> None:
        # Check that all the columns we need are present
        if any(
            feature not in data.columns
            for feature in ["name", "description", "price", "category"]
        ):
            raise ValueError("The dataset does not contain all the necessary columns")

        # Check that the price is non-negative
        if (data["price"] < 0).any():
            raise ValueError("The price cannot be negative")

        # etc. etc.
In [2]:
class AmazonDataSource(DataSource):
    def pull_data(self) -> pd.DataFrame:
        # You likely going to pull data from their API or service
        # keep that contained here
        # ...
        # before you return the data, validate it
        DataSource.validate_data(data)
        return data


class TargetDataSource(DataSource):
    def pull_data(self) -> pd.DataFrame:
        # You likely going to pull data from their API or service
        # keep that contained here
        # ...
        # before you return the data, validate it
        DataSource.validate_data(data)
        return data

Basic Application¶

Abstract Base Classes ABCs are great! They allow you to define a common interface for a group of classes. As well as enforce that the classes that inherit from the ABC implement the methods you define.

As with validate_data you can also attach utility functions that each sub-class may want to use to keep everything together.

But, there are lots of other ways to structure code and other design patterns.

Common Patterns¶

Common Patterns¶

For the next ten minutes, with a partner, look through:

https://github.com/faif/python-patterns¶

Think about how any of these patterns might be useful to a problem you recently encountered.

A first one to look at might be the factory which is very useful when trying to have a standard interface to create different objects or functions using a parameter.

Ex: you are processing a bunch of different image file formats and you need to find the appropriate file reader for each


As you are doing this, make note of patterns you find interesting or particularly useful and why. Or take note of patterns you have never seen before.

Takeaways¶

Three Targets For Simpler Programs¶

  1. Make software easier to understand by breaking it into pieces that can be understood separately

  2. Enable the ability to maintain, reuse and extend software (or bits of software)

  3. Avoid extra work from lack of forethought towards methods of engagement with software

Well designed software should try to minimize the mental effort required to reason about it.

If there are too complex of functions, break them out to think about and develop one at a time. If there is too much coupling between functions or modules, try to reduce coupling.

Learn and apply coding patterns. There are many resources to look up common patterns.

A good way to learn them is to use the example code commonly given and change, add, or remove values and see what happens.

Credit¶

Much of the content of this talk comes from:

  1. UW CSE 331 Lecture 1
  2. Previous URSSI Winter School
  3. Software Design for Research Software