{"id":29,"date":"2026-09-10T13:05:43","date_gmt":"2026-09-10T20:05:43","guid":{"rendered":"https:\/\/blogs.reed.edu\/datalab\/?p=29"},"modified":"2026-09-11T15:13:49","modified_gmt":"2026-09-11T22:13:49","slug":"commons-common-meals","status":"publish","type":"post","link":"https:\/\/blogs.reed.edu\/datalab\/2026\/09\/10\/commons-common-meals\/","title":{"rendered":"Commons’ Common Meals"},"content":{"rendered":"\n
The most commonly served meal was Fish & Chips at 14 times! It was closely followed by Fried Tofu & Chips at 12 times. <\/div>\n\n\n\n

This week we wanted to know what types of meals were being served at commons. We found out that a ton of data on that is available and the chart in the Quest just skims the surface. If you want to learn more about the data, or use it yourself, keep reading for a walk-through or email data@reed.edu<\/a>. <\/p>\n\n\n\n

\"plot<\/figure>\n\n\n\n

Where did we find the data?<\/h2>\n\n\n\n

Every day Bon Appetit’s menu is posted on the Commons homepage<\/a>. Since we wanted to get access to past menus, the first thing to do was see if that information was cached somewhere. When you go to any Commons’ menu that is not todays, you will see the URL look something like this: https:\/\/reed.cafebonappetit.com\/cafe\/commons-cafe\/2026-09-11\/<\/a> The important thing there is that it ends in the specific date. That means each menu was theoretically stored as its own page that we could access using a web-scraping script. <\/p>\n\n\n\n

How did we web scrape the data?<\/h2>\n\n\n\n

The individual URL dates indicate that each day’s data exists as an inline JS object in the page’s HTML code. So we can write a script to access that and pull out the code. I prefer working in R, so I wrote this script in R. But it could have been written in Python or a number of other languages. <\/p>\n\n\n\n

There is actually a lot of data contained in each day’s page. To make the graph simpler (because it has to print in black and white and be legible when small), I decided to only pull dinner menus and only from the SimplyOasis and Classics stations. But more data than this is available if you want to use it!<\/p>\n\n\n\n

Here’s the script. The main things it relies on are the packages {httr}<\/a> and {jsonlite}<\/a>. {httr} handles the HTTP request \u2014 downloading each day’s page \u2014 while {jsonlite} takes the JSON-formatted menu data embedded in that page and turns it into something R can actually work with.<\/p>\n\n\n\n

\nShow the script<\/summary>\n
\n\n# load libraries\nlibrary(httr)\nlibrary(jsonlite)\nlibrary(stringr)\nlibrary(readr)\n\n####  set selections for scraping\n###############################\n\n# get url for menu\ncafe_url <- \"https:\/\/reed.cafebonappetit.com\/cafe\/commons-cafe\/%s\/\"\n\n# select what meal (can be single value or vector)\ndaypart <- \"Dinner\"\n# select station (can be single value or vector)\nstation_wanted <- c(\"SimplyOASIS\", \"Classics\")\n# select date range\ndates <- seq(as.Date(\"2026-01-26\"), as.Date(\"2026-05-14\"), by = \"day\")\n\n\n####  create functions to:\n####  fetch webpage,  \n####  extract the menu,\n####  extract the correct part of day\n\n# downloads one date's menu page as raw HTML\nfetch_page <- function(d) {\n  url <- sprintf(cafe_url, format(d, \"%Y-%m-%d\"))\n  resp <- GET(url, timeout(30))\n  stop_for_status(resp)\n  content(resp, as = \"text\", encoding = \"UTF-8\")\n}\n\n# pulls the menu_items from the JavaScript and parses it as JSON\nextract_menu_items <- function(html) {\n  pattern <- regex(\"Bamco\\\\.menu_items\\\\s*=\\\\s*(\\\\{.*?\\\\});\", dotall = TRUE)\n  match <- str_match(html, pattern)\n  fromJSON(match[1, 2], simplifyVector = FALSE)\n}\n\n# pulls the relevant daypart and returns as a named list\nextract_dayparts <- function(html) {\n  pattern <- regex(\n    \"Bamco\\\\.dayparts\\\\['(\\\\d+)'\\\\]\\\\s*=\\\\s*(\\\\{.*?\\\\});\\\\s*\\\\n\\\\s*\\\\}\\\\)\\\\(\\\\);\",\n    dotall = TRUE\n  )\n  matches <- str_match_all(html, pattern)[[1]]\n  dayparts <- list()\n  for (i in seq_len(nrow(matches))) {\n    dayparts[[matches[i, 2]]] <- fromJSON(matches[i, 3], simplifyVector = FALSE)\n  }\n  dayparts\n}\n\n\n####  create function extract the actual meals\n###############################\n\n# helper function for pulling menu\n# if a is NULL, return b; otherwise return a \n# (can't get combinable rows if NULL values are present)\n`%||%` <- function(a, b) if (is.null(a)) b else a\n\n# extracts meals based on previous parameters\nget_meal_items <- function(d) {\n  html <- fetch_page(d)\n  menu_items <- extract_menu_items(html)\n  dayparts <- extract_dayparts(html)\n\n  # create empty list for output\n  rows <- list()\n  for (dp in dayparts) {\n    # only do this for the correct meal time\n    if (!(dp$label %in% daypart)) next\n\n    # find every station that matches the station(s)_wanted\n    # using Filter because it's lists not dataframes\n    stations <- Filter(\\(s) str_trim(str_remove_all(s$label, \"<[^>]+>\")) %in% station_wanted, dp$stations)\n\n    # make sure there's something there to find\n    if (length(stations) == 0) next\n\n    # for every matching station, grab data for each of its items\n    for (station in stations) {\n      for (item_id in station$items) {\n        # get the item's full list of things\n        item <- menu_items[[item_id]]\n        # make a dataframe out of the following\n        rows[[length(rows) + 1]] <- data.frame(\n          date = as.character(d),\n          meal = dp$label,\n          item_name = item$label,\n          description = item$description %||% \"\", # put \"\" instead of NULL\n          price = as.character(item$price %||% \"\"), # put \"\" instead of NULL\n          dietary_tags = paste(unlist(item$cor_icon), collapse = \", \")\n        )\n      }\n    }\n  }\n  # bind everything together\n  do.call(rbind, rows)\n}\n\n\n####  actually pull the data\n###############################\n\n# create empty list for things to go in\nrows <- list()\n# get meal for each date\nfor (i in seq_along(dates)) {\n  d <- dates[i]\n  rows[[i]] <- get_meal_items(d) # this is the line that actually runs everything\n\n  # prints each date just to show progress \n  cat(as.character(d), \"\\n\")\n  # small pause between requests (technically don't need if it's too slow)\n  Sys.sleep(0.1)\n}\n\nall_meals <- do.call(rbind, rows)\n\n\n####  write file\n###############################\n\nwrite_csv(all_meals, \"data\/dinner_jan26_may14.csv\")\n\n<\/details>\n<\/code><\/pre>\n<\/details>\n<\/ br>\n\n\n\n
<\/p>\n\n\n\n
Out of this script, I get a csv file that looks like this: <\/p>\n\n\n\n
\"\"<\/figure>\n\n\n\n
Wrangling the data and graphing<\/h2>\n\n\n\n
The way I designed the scraping leaves the data pretty clean, so not a lot needs to be done to work with it. The only thing I need to do is decide what things I want to pull from it. After looking through it, I decided that the best thing would be to look at main ingredients from the item name. I decided to focus on proteins (and mushrooms). Full disclosure, I used Claude to generate this list. I have strong and mixed feelings about AI, but this is one of the uses where it excels and it was much faster than trying to just identify things by most common word because things like “pasta” or “seasoned” came up a lot.<\/p>\n\n\n\n