Setting Up Python

Unit 3 uses Python, in Jupyter notebooks (documents that mix code, its output, and explanatory text) (Kluyver et al. 2016). You write and run code in small cells, one at a time, and see the result immediately below each one, which makes them forgiving for learning: run a cell, see what happens, change it, run it again.

This page covers two things: how much Python you need, and how to get a working setup.

Required Python Knowledge

You won’t write Python from scratch in this unit. You’ll read, run, and adapt a notebook that has already been written, and ask an LLM to explain anything that isn’t clear (Coding with LLMs). But you need to be able to follow what the code is doing. Pick the route that best fits you:

  • You already use Python: skip ahead to the setup options.
  • You’ve used some Python, or another programming language, before: read the refresher below, and check that each idea in it looks familiar.
  • You’ve never programmed: work through the first two units of this self-paced introduction to Python before you start the case study. Then read the refresher below, as a check.

Python reads relatively close to plain English. Five ideas are enough to follow the case study:

  • Variables store a value under a name:

    text = 'MY DEAR COLVIN'

  • Lists hold a collection of items, in order:

    letters = ['wikisource_vol1_ch1_letter1.txt', 'wikisource_vol1_ch1_letter2.txt']

  • Loops do something for each item in a list:

    for letter in letters:
    print(letter)

  • Functions are reusable, named blocks of code. They take inputs and return a result:

    def count_words(text):
    return len(text.split())

count_words(text)  # 3

  • Libraries are code that someone else has written, which you import and use. The case study uses pandas for tables and transformers for the model:

    import pandas as pd

letters = pd.read_csv('../data/letters_metadata.csv')

Notice the indentation: in Python, the lines that belong to a loop or a function are indented under it. If any of this is unfamiliar, the self-paced introduction covers it properly.

Setup Options

There are three ways to get set up, each in its own tab below. If you’re not sure, start with Google Colab.

Option Install? Good for
Google Colab No, runs in the browser Getting started quickly, or a computer where you can’t install software
Anaconda Yes, one installer A permanent local setup, with a graphical launcher
Plain Python Yes, command line People comfortable with a terminal

Google Colab runs notebooks on Google’s computers, and you see the results in your browser. You only need a Google account.

  1. Open the case study notebook on GitHub and click the Open in Colab badge at the top.
  2. Run the first cell. It downloads a copy of this repository into your Colab session, so the notebook can find the data.
  3. Carry on from there. The libraries the notebook needs are already installed in Colab.
WarningAbout Colab

Two things to know about Colab:

  • Your session is temporary. When it ends, the downloaded data and anything you created (such as your annotations) are deleted. Download anything you want to keep (File → Download), or save a copy of the notebook to your Google Drive (File → Save a copy in Drive).
  • Annotations: to use your own annotations, upload your my_annotations.csv file to understanding-ai/data/annotations/ in Colab’s file browser (the folder icon on the left) before running Section 9.

The Anaconda distribution installs Python, Jupyter, and many common libraries in one go, on Windows, macOS, or Linux.

Anaconda is free for individuals, for teaching, and for organizations with fewer than 200 employees, but its terms may require a paid licence for other uses at larger organizations, research included. If that might apply to you, check with your institution, or install Miniforge instead: it’s free for everyone, and the steps below are the same, with Miniforge Prompt in place of Anaconda Prompt.

  1. Download and run the installer from anaconda.com/download. You can install it like any other program (the defaults are fine).

  2. Get a copy of this repository: on its GitHub page, click Code → Download ZIP and unzip it (or use git clone if you know git).

  3. Open Anaconda Prompt (Windows) or Terminal (macOS and Linux), move into the repository folder, and install the libraries the notebooks need (the . means this folder, the list of libraries is in pyproject.toml):

    cd path/to/understanding-ai
pip install .

  4. Start Jupyter from the same window:

    jupyter notebook

    Your browser opens with a list of files. Open code/ner_pipeline.ipynb.

You can also start Jupyter from the Anaconda Navigator app instead of the command line, but you still need to run step 3 once.

If you’re comfortable with a terminal, you can use a standard Python installation and a virtual environment (an isolated set of libraries for this project, so it doesn’t interfere with anything else on your computer).

  1. Install Python 3.12 or later.

    • macOS: with Homebrew: brew install python@3.12, or use the installer from python.org.
    • Windows: install the Python Install Manager, from python.org or the Microsoft Store. Then, in Command Prompt, run py install 3.12.
    • Linux: use your distribution’s package manager (for example, sudo apt install python3 python3-venv).

  2. Get this repository: git clone https://github.com/gpizzorno/understanding-ai.git, or download and unzip it from GitHub.

  3. Create and activate a virtual environment in the repository folder:

    macOS and Linux:

    cd understanding-ai
python3 -m venv .venv
source .venv/bin/activate

    Windows (Command Prompt):

    cd understanding-ai
py -m venv .venv
.venv\Scripts\activate

  4. Install the libraries listed in pyproject.toml:

    pip install .

  5. Start Jupyter:

    jupyter notebook

Each time you come back, activate the environment again (step 3’s last line) before starting Jupyter.

Checking Your Setup

Whichever option you chose:

  1. Open code/ner_pipeline.ipynb.
  2. Run the first three code cells (the setup cell, the imports, and loading the metadata).
  3. If the third cell prints Total letters in corpus: 462, you’re ready.

If something goes wrong:

  • ModuleNotFoundError (for example, No module named 'transformers'): a library is missing. Make sure you ran pip install . from the repository folder, in the same environment Jupyter is using. In a notebook cell, you can also run %pip install transformers (replacing transformers with the missing module): %pip always installs into the environment the notebook is running in.
  • FileNotFoundError for ../data/...: the notebook can’t find the data. Make sure you opened the notebook from inside the repository’s code/ folder, not a copy saved somewhere else. Coding with LLMs explains why this matters.
  • Anything else: copy the full error message into an LLM and ask what it means, as described in Coding with LLMs.

The first time the notebook loads a model, it downloads a few hundred megabytes, so you’ll want to do it from somewhere with a reasonably fast internet connection. Installing the libraries also takes a while, and several gigabytes of disk space.

References

Kluyver, Thomas, Benjamin Ragan-Kelley, Fernando Pérez, et al. 2016. “Jupyter Notebooks—a Publishing Format for Reproducible Computational Workflows.” In Positioning and Power in Academic Publishing: Players, Agents and Agendas.