4.1 What is backward design?

When writing code, be it a helper function, a data processing script, or a whole package, there’s a common logical order we follow.

1. Define function output
2. Write unit test
3. Write function code

Writing the code itself ideally comes last. We can only write the code once we know what it’s supposed to do, and how we’ll verify that. If you started writing code before you decided what it was supposed to do, the code would go nowhere!

Backward design takes that same logic and applies it to teaching. The “usual” method of planning lessons starting from activities and demonstrations can be referred to as “forward design.” If planning lessons starting from activities is “forward design”, then backward design involves the following steps:

1. Define learning outcomes
2. Define evidence of learning
3. Build lesson

This looks a lot like the logical process of coding. Both coding logic and backward lesson design aim to answer the following questions, in order:

1. What should code/learners be able to do?
2. How will we know code/learners can do that?
3. How will we equip code/learners with the ability to do that?

You already know this logic! Lesson planning is conceptually similar to writing code. You don’t need to acquire a whole new knowledge base from scratch to be able to plan R lessons. You only need to learn how to relate concepts from coding logic to backward lesson design, and that’s exactly what we’re going to do.

Next, we’ll dig more into each of these three steps of backward design as they might manifest in an R coding lesson.

4.2 Define outcomes

Defining the learning outcome, or objective (used interchangeably from here on out), is conceptually similar to defining the output of a function, script, or entire package that one might write. It’s the answer to the following question:

What will learners be able to do when the lesson is finished?

You have to be specific when spelling out learning goals for them to be most effective. Just like it’s harder to write a function when you haven’t delineated what you want it to do, it’s harder to prepare a lesson when you haven’t outlined what you want it to teach.

Less specific (worse):

Learners will understand how tidyr, dplyr, and ggplot2 help them clean and explore their data

More specific (better):

Learners will be able to: - use tidyr’s pivot_wider() and pivot_longer() to reshape data for easier analysis - use dplyr’s group_by(), count(), and summarize() functions to generate summary statistics - use ggplot2 to generate exploratory scatterplots of data

A few tips on making your R lesson objectives more specific and effective:

• Use active verbs. “Learners will be able to use … to …” is a good starter
• If you’re focusing on a specific package, name that package
• If you’re focusing on specific functions, name those functions
• Make it more verbose (but don’t go overboard!)

Note: Most learning goals in lessons about R will be to “use” or “implement” certain functions or techniques. However, depending on the topic, the goal may be something lower, like to “recognize” or “recall” which function does what, or something higher, like to “evaluate” or “critique” code for style and speed. If you’re interested in reading more about the different types of learning goals one might set for a lesson, and which one best fits the level of learning you want, Bloom’s Revised Taxonomy is a helpful framework.

4.3 Define evidence of learning

Once you’ve outlined your desired learning objectives, the next step is to define acceptable evidence of learning. How will you determine that learners can indeed accomplish what the learning objective says they can?

Again, in programming and teaching, we must test to confirm that our code (or our learners) have accomplished the objective.

In coding, this can be as simple as running your code multiple times under different conditions to make sure the output looks right, or as elaborate as a full unit testing system (we’ll touch on this later!).

In a semester-long programming course, this would be the final assignment or exam. In a 5-minute tutoring session, this might involve watching your colleague write code that solves the problem they came to you for. In every case, the only way we can know that learners have reached the objective is by testing their ability to demonstrate it.

Note: the final assessment example below was omitted from the talk for time.

For example, the final test for a lesson with the learning objectives shown above might be:

The Orange dataset in R’s datasets package contains data for the ages (in days) and circumferences (in mm) of several orange trees, each measured at different timepoints.

1. Use the appropriate pivot (_wider or _longer) function to pivot the data to have one row for each measurement, one column for the age of each tree, and one column for the circumference of each tree.
2. Using the long form of the data, generate a dataframe showing the number of measurements for each tree.
3. Using the long form of the data, generate a dataframe showing the minimum and maximum age and circumference for each tree.
4. Using appropriate ggplot2 functions, generate a scatterplot of tree circumference by age. Plot lines connecting the observations for each tree. The points and lines for each tree should be different colors. Add an informative title and axis labels.

There doesn’t have to be one test question for each objective, but it’s a good place to start to make sure you have appropriate coverage. Just like you want high code coverage when testing your own code, you want high objective coverage with your final assessment, to ensure that you’re testing learners’ ability to execute every objective that you want them to accomplish. No matter how many prompts you include in your assessment, make sure that tests are written specifically enough that a learner who has reached the objective would be able to answer every question acceptably. Vague prompts don’t “give away the answer,” they give enough information for learners to get to the answer.

There can (and often should) be a back-and-forth process between defining learning outcomes and defining tests of learning. Sometimes, you might find that your test isn’t testing exactly what you think learners should be able to do. You might add more to your test, or change what’s already in it, so that it better fits the learning outcome. Other times, you might have a sense that the test itself correctly expresses what you want learners to come away with. In those cases, you might re-write the learning outcome so that it better fits the test. The key is to plan with intention either way.

iris %>%
  rename_with(tolower) %>% 
  pivot_longer(cols = -species,
               names_to = c("part", ".value"),
               names_sep = "\\.") %>% 
  group_by(species, part) %>% 
  summarize(across(everything(), list(mean = mean, sd = sd)))

## `summarise()` has grouped output by 'species'. You can override using the `.groups` argument.

## # A tibble: 6 x 6
## # Groups:   species [3]
##   species    part  length_mean length_sd width_mean width_sd
##   <fct>      <chr>       <dbl>     <dbl>      <dbl>    <dbl>
## 1 setosa     petal        1.46     0.174      0.246    0.105
## 2 setosa     sepal        5.01     0.352      3.43     0.379
## 3 versicolor petal        4.26     0.470      1.33     0.198
## 4 versicolor sepal        5.94     0.516      2.77     0.314
## 5 virginica  petal        5.55     0.552      2.03     0.275
## 6 virginica  sepal        6.59     0.636      2.97     0.322

iris %>%
  rename_with(tolower) %>% 
  pivot_longer(cols = -species,
               names_to = c("part", ".value"),
               names_sep = "\\.") %>% 
  group_by(Species, part) %>% 
  summarize(across(everything(), list(mean = mean, sd = sd)))

## Error: Must group by variables found in `.data`.
## * Column `Species` is not found.

iris_long <- iris %>% 
  rename_with(tolower) %>% 
  pivot_longer(cols = -species,
               names_to = c("part", ".value"),
               names_sep = "\\.") %>% 
  group_by(species, part)

# CODE GOES HERE

iris_long %>% 
  summarize(across(everything(), list(mean = mean, sd = sd)))

## `summarise()` has grouped output by 'species'. You can override using the `.groups` argument.

## # A tibble: 6 x 6
## # Groups:   species [3]
##   species    part  length_mean length_sd width_mean width_sd
##   <fct>      <chr>       <dbl>     <dbl>      <dbl>    <dbl>
## 1 setosa     petal        1.46     0.174      0.246    0.105
## 2 setosa     sepal        5.01     0.352      3.43     0.379
## 3 versicolor petal        4.26     0.470      1.33     0.198
## 4 versicolor sepal        5.94     0.516      2.77     0.314
## 5 virginica  petal        5.55     0.552      2.03     0.275
## 6 virginica  sepal        6.59     0.636      2.97     0.322

iris_long %>% 
  summarize(across(everything(), list(mean = avg, sd = sd)))

iris_long %>% 
  summarize(across(everything(), list(avg = mean, sd = sd)))

4.4 Build lesson

The beauty of backwards design is that once you’ve spelled out your learning objectives, and the tests you’ll use to verify that learners have reached those objectives, the lesson itself is nearly done. Ideally, once you’ve laid out your assessments, you will be able to see exactly what you need to demonstrate and explain to equip learners with enough knowledge to complete those assessments.

The core principle I try to rely on when crafting lessons is to teach what you need to teach: no more and no less. What does this mean? This encompasses several things, including:

Calibrate to learners’ incoming skill level. What do you expect learners to be able to do already before they start your lesson? Start where learners currently are (don’t re-teach), and teach everything new that learners need to know to reach the objective (think through all the pre-reqs!).

Allow your formative assessments to break the lesson into manageable chunks. If you have implemented the right number and scope of checkpoints, each check-in question should serve as the end to a bite-sized (5-10 min) piece of lesson. These bites are a good size for you, because you get to take breaks from speaking and survey learners’ progress, and a good size for learners, because they get to take a breather from new information and use the check-in question to practice what they’ve just learned.

Teach less than you think you have time for. An instructor teaching a particular lesson for the first time is likely to find that the lesson simply takes way longer than originally planned. Running out of time before you’ve finished the lesson guarantees that learners won’t reach the objective! Plan for less time than you’ll actually have. The first time I teach a new lesson, I plan it for 2/3 of the allotted class time, and it usually comes out right.

Teach one way to do things. One of the great (and terrible) features of R is that there’s often many, many different ways to solve a problem. While some might be less verbose, more generalizable, etc., than others, ultimately sometimes you get to two different techniques that differ only on personal taste. For example, there might be two different ways to teach the pivot_longer() technique demonstrated earlier.

Using names_sep to break up column names:

iris %>% 
  rename_with(tolower) %>% 
  pivot_longer(cols = -species,
               names_to = c("part", ".value"),
               names_sep = "\\.")

## # A tibble: 300 x 4
##    species part  length width
##    <fct>   <chr>  <dbl> <dbl>
##  1 setosa  sepal    5.1   3.5
##  2 setosa  petal    1.4   0.2
##  3 setosa  sepal    4.9   3  
##  4 setosa  petal    1.4   0.2
##  5 setosa  sepal    4.7   3.2
##  6 setosa  petal    1.3   0.2
##  7 setosa  sepal    4.6   3.1
##  8 setosa  petal    1.5   0.2
##  9 setosa  sepal    5     3.6
## 10 setosa  petal    1.4   0.2
## # … with 290 more rows

Using names_pattern to break up column names:

iris %>% 
  rename_with(tolower) %>% 
  pivot_longer(cols = -species,
               names_to = c("part", ".value"),
               names_pattern = "(.*)\\.(.*)")

## # A tibble: 300 x 4
##    species part  length width
##    <fct>   <chr>  <dbl> <dbl>
##  1 setosa  sepal    5.1   3.5
##  2 setosa  petal    1.4   0.2
##  3 setosa  sepal    4.9   3  
##  4 setosa  petal    1.4   0.2
##  5 setosa  sepal    4.7   3.2
##  6 setosa  petal    1.3   0.2
##  7 setosa  sepal    4.6   3.1
##  8 setosa  petal    1.5   0.2
##  9 setosa  sepal    5     3.6
## 10 setosa  petal    1.4   0.2
## # … with 290 more rows

In practice, there are valid arguments for using either method. Using names_sep to split along the period is less verbose, but is theoretically less flexible than spelling out the capture groups with names_pattern, in case there are future columns in the data that delimit with periods AND underscores.

What is the learning goal here? If indeed a primary objective of the lesson is to be able to pivot many different arrangements of data, then students do need to learn about the similarities and differences between using names_sep and names_pattern to break up column names in service of the objective. However, if the objective is solely to be able to pivot data with multiple value columns using the ".value" placeholder, then being able to compare, contrast, and choose between names_sep and names_pattern is not part of the learning goal. In this case, you as the instructor need to decide which technique you would rather students come away using, and teach only that technique.

Related to this point: don’t teach first principles unless the learning goal is first principles. I was guilty of this for a while! For example, when I used to teach vectorized operations like apply(), and later map(), I would lead with a whole section on for loops and why they were slow and unwieldy. When I learned how to take full advantage of R’s vectorized functions, I felt like I had woken up from a fog of for loops. I assumed this experience generalized to other people, and so when I taught those functions I would spend a lot of time making for loops sound like the black-and-white “before” section of an infomercial before the life-changing new product is revealed. However, I eventually received feedback that learners didn’t understand why I was spending so much time talking about for loops when I didn’t teach them, and some learners didn’t even know what for loops were before I mentioned them, so they were getting confused.

While it is true that a first-principles comparison of for loops and vectorized functions is important to a full understanding of when vectorizing is and isn’t the best solution to a code problem, understanding first-principles isn’t necessary to be able to correctly use a vectorized function. Once I realized this, I was able to let go of a lesson module that was more nostalgic to me than instructive to learners.

(I actually heard the above from Hadley Wickham! At Columbia I was lucky to be able to take Dr. Andrew Gelman’s Communicating Data course, where we heard from guest speakers like Hadley.)