All in One View

Content from Getting Started with the Shell

Last updated on 2026-08-04 | Edit this page

Overview

Questions

“How do I install and open a shell on my computer?”
“What is a shell, and why should I use it instead of only a graphical interface?”
“How do I move around directories and inspect files from the command line?”
“How do I do common file and folder operations in Bash?”
“How can I get help when I do not remember a command?”

Objectives

“Install a shell (Bash or equivalent) and open a terminal on your operating system.”
“Explain what a shell is and why it is useful for scientific and data workflows.”
“Open a terminal and run simple commands safely.”
“Navigate the filesystem with pwd, ls, and cd.”
“Create, move, copy, rename, and remove files and directories with core shell commands.”
“Use tab completion and built-in help (--help, man) to discover command usage.”

Install and open a shell

If you do not already have shell software installed, follow the Carpentries installation instructions:

https://carpentries.github.io/workshop-template/install_instructions/#shell

After installation, open a new terminal window.

Windows: open Git Bash, WSL terminal, or another Bash-compatible terminal.
macOS: open Terminal (Applications -> Utilities -> Terminal).
Linux: open your system Terminal application.

Once the terminal opens, type:

BASH

cd

This returns you to your home directory and gives everyone a consistent starting point for the lesson.

If your terminal is already configured and opens correctly, you can continue directly to the next section.

Why use the shell?

Many scientific workflows rely on command-line tools, including remote systems, HPC environments, and cloud processing pipelines. The shell helps you:

Automate repetitive tasks.
Combine simple tools into larger workflows.
Work efficiently on local and remote machines.

In this course we use Bash, one of the most common Unix shells. A shell reads the commands you type, runs programs, and prints the output.

For prep work, think of the shell as a universal interface: even when tools differ across institutions, the basic command-line workflow is often the same. Learning this now makes later lessons smoother because you can focus on data concepts instead of terminal basics.

Opening a shell and running first commands

Open a terminal and try:

BASH

echo "Hello World"
whoami
date

Each line is a command. You are always working from a current directory.

These first commands help learners build confidence:

echo prints text, so you can confirm command syntax quickly.
whoami shows your current user, which is useful on shared systems.
date confirms system time and demonstrates command output.

Navigating files and directories

Print working directory

BASH

pwd

pwd prints the path of your current directory. It is good practice to run pwd whenever you are unsure where a command will run.

List directory contents

BASH

ls
ls -F
ls -l
ls -la

ls lists items in the current directory.
-F marks directory names with /.
-l shows a detailed list.
-a includes hidden files.

Together, these options help you answer two quick questions before changing anything: “Where am I?” and “What is here?”

Change directory

BASH

cd some_directory
cd ..
cd

cd some_directory enters a directory.
cd .. moves one level up.
cd without arguments returns to your home directory.

A useful mental model is a tree of folders. cd moves your location in that tree, and every relative path depends on your current location.

Paths can be:

Absolute: start from /, such as /home/username/data.
Relative: start from your current location, such as data/file.txt.

Show me the solution

BASH

pwd
ls -la
cd some_directory
cd ..
cd

Working with files and directories

In command-line workflows, file and directory operations are the foundation for reproducible analysis. Before running scripts or notebooks, you usually create a clean folder structure and move files into predictable locations.

Create directories and files

BASH

mkdir my_project
cd my_project
touch file1.txt file2.txt
ls

mkdir creates a new directory. touch creates an empty file if it does not exist, and updates a timestamp if it does.

In the example above, we created a new directory called my_project, entered it, and created two empty text files. The output of ls confirms the files exist.

Copy, move, and rename

BASH

cp file1.txt file1_copy.txt
mv file1_copy.txt renamed.txt
mv renamed.txt ../renamed.txt

Use the cp command to copy files (it will create a duplicate). Use the mv command to move or rename files.

If you try ls after the last command, you will see that renamed.txt is no longer in the current directory because it was moved to the parent directory.

Remove files and directories

BASH

rm file2.txt
rm -r some_directory

The rm command removes files. With the -r option, it removes directories and their contents recursively.

Be careful with rm and especially rm -r, since removal is usually permanent. For beginners, a safe habit is to run ls before and after removal so you can verify exactly what changed.

Viewing and editing text files

Write text to files with echo, >, and >>

You can use echo together with redirection operators to create and update text files directly from the shell.

BASH

echo "Sea surface temperature notes" > notes.txt
echo "Added a second line" >> notes.txt
cat notes.txt

How this works:

> writes output to a file (and overwrites the file if it already exists).
>> appends output to the end of a file (keeps existing content).

This pattern is very common in command-line workflows and is used again in the Git lesson when creating and updating tracked files.

View file contents with cat

Use cat to print a text file in the terminal:

BASH

cat data1.txt
cat data1.txt data_main.txt

This is useful for quickly checking file contents. For very large files, tools like less are often more practical than cat, but cat is perfect for small text files:

BASH

less data1.txt

Edit files with nano

nano is a simple terminal text editor:

BASH

nano notes.txt

In nano, type your text, then:

Press Ctrl+O to save.
Press Enter to confirm the filename.
Press Ctrl+X to exit.

nano is a good starter editor because it shows keyboard shortcuts directly in the interface.

Challenge

Exercise 2: cat and nano basics

Create a file called notes.txt using nano.
Add two short lines of text and save the file.
Display the file contents using cat.

Show me the solution

BASH

nano notes.txt
cat notes.txt

Challenge

Exercise 3: echo and redirection

Create a file called log.txt with one line using echo and >.
Add a second line using echo and >>.
Display the final file content with cat.

Show me the solution

BASH

echo "First log line" > log.txt
echo "Second log line" >> log.txt
cat log.txt

Getting help and using tab completion

Use built-in help:

BASH

ls --help
man ls

Use tab completion by typing part of a filename or directory name, then pressing Tab. This reduces typos and speeds up command entry.

When learning new commands, combine these habits:

Check options with --help.
Try commands in a small practice directory first.
Use tab completion to avoid path mistakes.

Challenge

Exercise 4: File operations practice

Create a directory called shell_practice.
Enter it and create data1.txt and data2.txt.
Create a subdirectory called backup.
Copy data1.txt into backup.
Rename data2.txt to data_main.txt.
List the contents of the main directory and backup.
Remove data1.txt from the main directory.

Show me the solution

BASH

mkdir shell_practice
cd shell_practice
touch data1.txt data2.txt
mkdir backup
cp data1.txt backup/
mv data2.txt data_main.txt
ls
ls backup
rm data1.txt

Summary

This episode introduced the shell as a practical interface for navigating files, running programs, and performing common file operations. These skills are foundational for later lessons where we work with scripts, remote systems, and larger scientific datasets.

Key Points

“The shell is a powerful interface for scientific workflows and automation.”
“pwd, ls, and cd are core commands for navigation.”
“mkdir, touch, cp, mv, and rm cover most basic file operations.”
“echo with > and >> lets you create files and append text from the command line.”
“cat displays text file contents quickly in the terminal.”
“nano is a beginner-friendly terminal editor for creating and updating text files.”
“Use --help, man, and tab completion to work faster and more safely.”

Content from Working with data in Xarray

Last updated on 2026-08-14 | Edit this page

Overview

Questions

“How do I load data with Xarray?”
“How does Xarray index data?”
“How do I apply operations to the whole or part of an array?”
“How do I work with time series data in Xarray?”
“How do I visualise data from Xarray?”

Objectives

“Understand the concept of lazy loading and how it helps work with datasets bigger than memory.”
“Understand whole-array operations and the performance advantages they bring.”
“Apply Xarray operations to load, manipulate, and visualise data.”

Introducing Xarray

For this lesson, you need to have the setup completed and the example data downloaded. See the setup instructions for details.

Xarray is a Python library for working with multi-dimensional array data. Many concepts are inspired by Pandas, but Xarray is designed to work well with very large array-based scientific datasets. It integrates with core scientific Python libraries such as NumPy and Matplotlib, and supports data larger than memory through lazy evaluation and chunked workflows.

Xarray can read and write NetCDF and also supports other formats such as GRIB and Zarr.

A useful way to think about Xarray is:

NumPy gives fast arrays.
Pandas gives labeled 1D/2D tables.
Xarray gives labeled N-dimensional arrays.

Those labels (for example valid_time, latitude, longitude) are the key feature that makes code easier to read and less error-prone.

Datasets and DataArrays

Xarray has two core data types:

DataArray: a single n-dimensional array with named dimensions and coordinates.
Dataset: a collection of multiple DataArray objects plus metadata.

Because Xarray follows duck-typing conventions with NumPy-like APIs, many NumPy-style operations can be applied directly to Xarray objects.

As you work through this lesson:

Use a Dataset when you have multiple related variables in one file.
Use a DataArray when you want to work on one variable at a time.

Most operations in this lesson start from dataset["sst"], which returns a DataArray.

Opening a NetCDF Dataset

For this lesson, use the example dataset provided during setup:

data/era5_sst/ocean_temperature.nc
Main variable: sst (sea surface temperature)

To open the dataset, use xarray.open_dataset:

PYTHON

import xarray as xr

dataset = xr.open_dataset("data/era5_sst/ocean_temperature.nc")

open_dataset reads metadata first and delays loading full data values until needed. This is called lazy loading. It is important for large datasets because you can inspect structure before using memory for computations.

You can also pass optional arguments such as:

engine=... to choose a backend reader.
chunks=... to prepare Dask-backed parallel operations.

To inspect the dataset, simply type its name in a Jupyter notebook cell:

PYTHON

dataset
print(dataset.attrs)
print(dataset.dims)
print(dataset.variables)

The attrs attribute contains metadata about the dataset, such as title, source, and history. The dims attribute lists dimension names and sizes. The variables attribute lists all variables, including coordinates and data variables.

To understand the file structure more clearly, inspect these explicitly:

PYTHON

print(dataset.data_vars)
print(dataset.coords)

data_vars are the main scientific variables (such as sst).
coords are coordinate variables used for indexing (such as time and latitude).

Accessing data variables

Access a variable:

PYTHON

print(dataset["sst"])

As you can see, the sst variable has dimensions of valid_time, latitude, and longitude.

dataset["sst"] is usually the safest style because it is explicit and still works when variable names are not valid Python identifiers, i.e., when it has special characters or space.

Access dimensions and elements:

PYTHON

print(dataset["sst"]["valid_time"]) # prints the valid_time coordinate
print(dataset["sst"]["valid_time"][0]) # prints the first valid_time value

You can also use dot-style access to retrieve variables that have valid Python identifiers, as mentioned above:

PYTHON

print(dataset.sst)

Dot-style access is convenient, but bracket style is safer and more general.

Indexing and slicing

There are two ways to index data in Xarray: by label and by integer index.

To index by label, use the sel method:

PYTHON

dataset["sst"]["valid_time"].sel(valid_time="2025-01-01T00:00:00")

This will return the sea surface temperature at the specified timestamp.

sel matches coordinate labels, not integer positions. It is the best choice when you care about physical meaning (specific dates, latitudes, longitudes).

To index by integer position, use the isel method:

PYTHON

dataset["sst"]["valid_time"].isel(valid_time=0)

This will return the sea surface temperature at the first timestamp in the dataset.

isel is useful when you want “first”, “last”, or “every nth” item, independent of coordinate values.

You can also combine indexing methods to select data by both label and integer position:

PYTHON

dataset["sst"].sel(valid_time="2025-01-01T00:00:00").isel(latitude=0, longitude=0)

You can also use standard Python slicing syntax to select ranges of data:

PYTHON

dataset["sst"][:12] # or dataset.sst[:12]
dataset["sst"][::2] # every other time step

For label-based slices with sel, remember:

The slice endpoints are coordinate labels.
The stop label is usually included when present.

On the other hand, for standard Python slicing syntax:

The stop index is usually NOT included in the slice.

Slice by labels:

PYTHON

dataset["sst"].sel(valid_time=slice("2025-01-01", "2025-01-02"))

To retrieve raw values, use .values (otherwise, Xarray returns a DataArray object):

PYTHON

dataset["sst"].sel(valid_time=slice("2025-01-01", "2025-01-02")).values

This will return a NumPy array of the sea surface temperature values for the specified time range.

Use .values only when you really need a NumPy array, which is rare, because we can do data operations using DataArray. Keeping data as Xarray objects preserves coordinate labels and metadata, which is often useful for later steps.

Challenge

Exercise 1: Slicing

Write a slicing command to get every other time step from the sea surface temperature dataset.
Write a slicing command to get the first 12 time steps from the sea surface temperature dataset, using isel.

Show me the solution

PYTHON

dataset["sst"]["valid_time"][::2]
dataset["sst"].isel(valid_time=slice(0, 12))

Nearest-neighbour lookups

A direct lookup fails when the timestamp is not present in the coordinate:

PYTHON

dataset["sst"].sel(valid_time="2025-01-01T01:10:00")

This may raise an error because that exact timestamp is not present.

Use nearest-neighbour matching instead:

PYTHON

dataset["sst"].sel(valid_time="2025-01-01T01:10:00", method="nearest")

You can add tolerance limits:

PYTHON

dataset["sst"].sel(valid_time="2025-01-01T01:10:00", method="nearest", tolerance="30min")
dataset["sst"].sel(valid_time="2025-01-01T01:10:00", method="nearest", tolerance="2h")

tolerance is important in scientific workflows because it prevents accidental matching to points that are too far from your requested value.

Plotting Xarray data

Plot a time series at one location:

PYTHON

dataset["sst"].sel(latitude=-30, longitude=320, method="nearest").plot()

Xarray automatically uses metadata to label axes and variables.

DataArray.plot() chooses a sensible default plot type based on dimensions:

1D data -> line plot
2D data -> image/pseudocolor plot
1D flattened values -> histogram with plot.hist()

Plotting two-dimensional data

Plot a 2D field:

PYTHON

dataset["sst"].sel(valid_time="2025-01-01T00:00:00").plot()

Use Matplotlib options, such as a grayscale colormap:

PYTHON

import matplotlib.pyplot as plt

dataset["sst"].sel(valid_time="2025-01-01T00:00:00").plot(cmap=plt.cm.Grays)

You can pass most Matplotlib-like keyword arguments (cmap, vmin, vmax, figsize) directly through Xarray plotting helpers.

Plotting histograms

You can also plot a histogram of values:

PYTHON

dataset["sst"].sel(valid_time="2025-01-01T00:00:00").plot.hist()

Histograms are useful for quick quality checks (for example, unrealistic value ranges or skewed distributions).

Challenge

Exercise 2: Slicing and plotting

Using a slice of the array, plot a transect of sea surface temperature across the Atlantic at 23 degrees North between 70 and 17 degrees West on:

2025-01-01 00:00
2025-01-03 00:00

Remember that the longitude values are in degrees East, so you will need to convert the West values to East.

Show me the solution

PYTHON

dataset["sst"].sel(valid_time="2025-01-01T00:00:00", method="nearest").sel(longitude=slice(290, 343), latitude=23).plot()
dataset["sst"].sel(valid_time="2025-01-03T00:00:00", method="nearest").sel(longitude=slice(290, 343), latitude=23).plot()

Array operations

Map operations

Xarray supports vectorised, whole-array operations, which are usually faster and clearer than manual loops.

These operations are applied element-wise and preserve array alignment by dimension names. This is one of Xarray’s main safety advantages compared with manual indexing.

Apply an offset:

PYTHON

dataset_corrected = dataset["sst"] - 1.0

This subtracts 1.0 from every element in the array.

Apply a linear correction:

PYTHON

dataset_corrected = dataset["sst"] * 1.1 - 1.0

Chaining operations like this is common and keeps code concise.

Apply a custom function with apply_ufunc:

PYTHON

def apply_correction(x):
    return x * 1.01 + 0.1

corrected_sst = xr.apply_ufunc(apply_correction, dataset["sst"])

Use apply_ufunc when you want to apply a custom function while still working with labeled data. For many pure NumPy-style functions, direct operations (+, -, *, /) are simpler.

Apply NumPy functions:

PYTHON

import numpy as np

dataset_clipped = xr.apply_ufunc(np.clip, dataset["sst"], 282, 291)

np.clip limits all values to a fixed range. This can help control outliers before plotting or statistics. We can see below the impact of applying the function to our SST results.

PYTHON

dataset["sst"].sel(valid_time="2025-01-01T00:00:00").plot() # original data
dataset_clipped.sel(valid_time="2025-01-01 00:00").plot() # clipped data

Reduce operations

Reduce operations aggregate data to fewer values.

Typical reducers include mean, sum, min, max, std, and quantile.

PYTHON

sst_mean = dataset["sst"].mean()
print(sst_mean.values)

Without a dim=... argument, mean() reduces all dimensions. To reduce along one axis only, specify dimensions explicitly, for example:

PYTHON

dataset["sst"].mean(dim="valid_time")

On a slice:

PYTHON

transect_mean = dataset["sst"].sel(
    valid_time="2025-01-01T00:00:00",
    longitude=slice(290, 343),
    latitude=23,
).mean()
print(transect_mean.values)

Note that this dataset uses longitudes in degrees East (0 to 360), so West longitudes must be converted.

Conditionally selecting and replacing data

Mask negatives to NaN:

PYTHON

dataset["sst"].where(dataset["sst"] >= 0.0)

where keeps values where the condition is True and sets other values to NaN (unless other= is provided).

Conditional replacement with xr.where:

PYTHON

xr.where(dataset["sst"] < 0.0, 0, dataset["sst"] * 2.0)

xr.where(cond, x, y) is a full if/else expression over arrays:

where cond is true -> use x
where cond is false -> use y

Mask by coordinate condition (for example, keeping eastern hemisphere values):

PYTHON

dataset["sst"].where(dataset["sst"].longitude > 0)

Challenge

Exercise 3: Map, reduce, and where

Using the example dataset:

Calculate the 95th percentile using quantile.
Remove data above the 95th percentile with where.
Multiply remaining values by a correction factor of 0.90.
Plot both original and corrected data for 2025-01-01 00:00.

Show me the solution

PYTHON

threshold = dataset["sst"].quantile(0.95)
lower_95th = dataset["sst"].where(dataset["sst"] < threshold)
lower_95th = lower_95th * 0.90

lower_95th.sel(valid_time="2025-01-01T00:00:00").plot()
# Run this in a separate cell to view both clearly
dataset["sst"].sel(valid_time="2025-01-01T00:00:00").plot()

Xarray patterns

Xarray provides common computational patterns including resampling, grouping, rolling windows, and coarsening.

These are high-level APIs for common scientific time-series and gridded-data workflows.

Resampling

PYTHON

resampled = dataset["sst"].sel(latitude=53, longitude=330, method="nearest").resample(valid_time="1D")

resample creates a resampler object; it does not compute values until you call an aggregation like .mean().

Apply a reducer and compare against original data:

PYTHON

import matplotlib.pyplot as plt

dataset["sst"].sel(latitude=53, longitude=330, method="nearest").plot(label="Original Data")
resampled.mean().plot(label="Resampled Daily Data", marker="o")
plt.legend()

Groupby

PYTHON

grouped = dataset["sst"].sel(latitude=53, longitude=330, method="nearest").groupby("valid_time.day")
grouped_mean = grouped.mean()
plt.bar(grouped_mean.day, grouped_mean)

groupby splits data into groups based on coordinate-derived labels, then applies operations per group.

Rolling windows

PYTHON

rolling = dataset["sst"].rolling(valid_time=12, center=True)
ds_rolling = rolling.mean()

dataset.sst.sel(longitude=330, latitude=53, method="nearest").plot(label="SST")
ds_rolling.sel(longitude=330, latitude=53, method="nearest").plot(label="12-step rolling mean")
plt.legend()

Rolling operations smooth short-term variability and highlight trends. center=True centers each window on the output coordinate instead of right-aligning.

Coarsening

Spatial coarsening:

PYTHON

coarse = dataset.coarsen(latitude=5, longitude=5, boundary="trim")
coarse.mean()["sst"].sel(valid_time="2025-01-01T00:00:00").plot()

coarsen aggregates fixed-size blocks and is often used to reduce spatial resolution. boundary="trim" drops leftover cells that do not fill a complete window.

Temporal coarsening:

PYTHON

coarse = dataset.coarsen(valid_time=12)
coarse.mean()["sst"].sel(latitude=53, longitude=330, method="nearest").plot()
dataset["sst"].sel(latitude=53, longitude=330, method="nearest").plot()

Writing data

Write processed output to NetCDF:

PYTHON

dataset_corrected = dataset["sst"] * 1.1 - 1.0
dataset_corrected.to_netcdf("corrected.nc")

to_netcdf writes a new NetCDF file to disk. In real workflows, choose output filenames that describe processing steps clearly, for example sst_bias_corrected_2025.nc.

Function reference: what each command does

This quick reference summarises the core functions used in this lesson.

xr.open_dataset(path): open a file as an Xarray Dataset (metadata first, data lazily).
dataset["var"]: select one variable as a DataArray.
sel(...): select by coordinate labels (dates, lat/lon values, named categories).
isel(...): select by integer positions (first, last, nth items).
where(condition, other=...): keep values where condition is true, replace others.
mean, sum, min, max, quantile: reduce over one or more dimensions.
resample(...): regroup data to a new time frequency (then aggregate, for example .mean()).
groupby(...): split data into groups by labels (then aggregate per group).
rolling(...): define moving windows for smoothing/trend analysis.
coarsen(...): aggregate fixed-size blocks across dimensions.
plot(), plot.hist(): quick visualisations directly from labeled arrays.
to_netcdf(path): write arrays back to NetCDF.

A useful workflow to follow is:

Open and inspect metadata.
Select a subset (sel/isel).
Apply operations (where, map/reduce).
Visualise results (plot).
Save outputs (to_netcdf).

Challenge

Exercise 4: Xarray tutorial dataset workflow

Using Xarray’s ersstv5 tutorial dataset (available with xr.tutorial.load_dataset("ersstv5")), complete the following workflow:

Select data before 2000.
Resample to annual means.
Compute global annual mean.
Plot the annual global mean time series.
Save results to NetCDF.

Show me the solution

PYTHON

import xarray as xr

sst = xr.tutorial.load_dataset("ersstv5")
sst_20c = sst.sel(time=slice("1970-01-01", "1999-12-31"))
sst_annual = sst_20c.resample(time="1YE").mean()
sst_global = sst_annual.mean(dim=["lat", "lon"])

sst_global["sst"].plot()
sst_global.to_netcdf("global-mean-sst.nc")

Key Points

“Xarray can load NetCDF files (and other formats such as GRIB and Zarr).”
“We can address dimensions by name with dot syntax, bracket syntax, sel, and isel.”
“With lazy loading, data are only loaded into memory when needed.”
“Whole-array math operations are usually more efficient than explicit Python loops.”
“Custom functions can be applied across arrays with tools such as apply_ufunc.”
“Xarray can plot directly through Matplotlib-backed plotting methods.”
“Hvplot enables interactive visualisations.”
“Xarray includes built-in patterns such as resampling, grouping, rolling, and coarsening.”

Content from Introduction to Git

Last updated on 2026-08-10 | Edit this page

Overview

Questions

“How do I install and configure Git on my machine?”
“Why do I need a GitHub account for collaborative workflows?”
“How do add, status, diff, and commit work together?”
“How do I inspect what changed and when?”
“How do branches help me work safely on new changes?”
“How do I merge branch work back into main?”

Objectives

“Install Git locally and verify it is available in the terminal.”
“Create a GitHub account and understand why it is useful for collaboration.”
“Use core Git commands: add, status, diff, and commit.”
“Track file changes over time with Git history.”
“Create and switch branches for feature work.”
“Merge a feature branch into main.”

Why version control?

Version control helps us keep a reliable history of changes to files. It allows us to:

Recover earlier versions.
See who changed what and when.
Work on new ideas in branches without breaking stable work.

This lesson is a simplified path through key ideas from the Software Carpentry Git novice material.¹ It focuses on the minimum skills learners need before collaborative coding and data workflows in later episodes.

If Git is new to you, do not worry about memorising every command immediately. The main goal is to understand the workflow: inspect changes, stage intentionally, and commit meaningful checkpoints.

Core concepts before commands

Before using Git commands, it helps to understand four terms:

Working directory: the files you are editing right now.
Staging area: the “next commit” area, where you choose exactly what to record.
Repository: the full Git history and metadata (stored in the .git directory).
Commit: a named snapshot in history, including author, date, and message.

A beginner mental model:

Edit in working directory.
Move selected changes to staging area (git add).
Save staged snapshot (git commit).

This two-step save process is one of Git’s strengths because it lets you build clean, logical history.

Install Git locally

Install Git for your operating system using the Carpentries setup page:

https://carpentries.github.io/workshop-template/install_instructions/#git

After installation, verify in a terminal:

BASH

git --version

If this prints a version number, Git is installed correctly and available on your terminal path.

If the command is not found, the shell cannot locate Git yet. In that case, complete installation first, then open a new terminal and run the check again.

Set your identity once:

BASH

git config --global user.name "Your Name"
git config --global user.email "your.email@example.com"

Check the values:

BASH

git config --global --list

These identity settings are written into commit metadata so teammates can see authorship clearly. Use your real name and an email address you want associated with your commits.

If you want to inspect one value only:

BASH

git config --global user.name
git config --global user.email

Create a GitHub account

Create an account at:

https://github.com

Recommended setup:

Verify your email.
Enable two-factor authentication.
Add a profile name that collaborators can recognise.

For this episode, a GitHub account is mainly needed so learners are ready for remote collaboration in the “Cloud-Native Architectures and Modern Data Formats for Geoscience” course. Git itself works locally without GitHub, but GitHub is the most common place to share repositories, open pull requests, and review changes.

In other words:

Git tracks versions on your machine.
GitHub is a hosting and collaboration platform for Git repositories.

Start a repository and track changes

Create a project folder and initialise Git:

BASH

mkdir git-practice
cd git-practice
git init

git init creates a hidden .git directory where Git stores project history and metadata.

At this point, your repository has no commits yet. You can verify that with:

BASH

git log

which will report that no commits exist.

Create a file and check status:

BASH

echo "Sea surface temperature notes" > notes.txt
git status

git status is your main safety command. It tells you what has changed, what is staged, and what is still untracked.

For beginners, running git status frequently is the best way to avoid confusion. If you are unsure what state the repository is in, run git status first.

Stage and commit:

BASH

git add notes.txt
git commit -m "Add initial notes file"

git add places content in the staging area, and git commit records that staged snapshot in history.

Important detail: git commit only records what is staged. If you edited a file but did not git add it, that edit will not be included in the commit.

Check history:

BASH

git log --oneline

Each commit gets a unique identifier and message, forming a timeline of your project.

The timeline helps with reproducibility: you can always return to a specific commit and understand what changed.

Understand add, status, diff, and commit

These four commands work together as a cycle:

Edit files in your working directory.
Inspect changes with status and diff.
Select what to include with add.
Save a logical checkpoint with commit.

This helps avoid accidental commits and makes history easier to read.

Think of staging as preparing slides for a presentation: you choose what belongs in this specific story.

Edit the file and inspect changes:

BASH

echo "Added a second line" >> notes.txt
git status
git diff

git diff shows unstaged edits line by line, so you can review before staging.

In git diff output:

Lines with - were removed.
Lines with + were added.
Context lines are shown to help you locate the change.

Stage then inspect staged changes:

BASH

git add notes.txt
git diff --staged

git diff --staged confirms exactly what will be committed next.

This is especially useful before large commits, when it is easy to accidentally stage too much.

Commit staged work:

BASH

git commit -m "Update notes with second line"

Good commit messages are short, specific, and action-oriented.

Useful message pattern:

Start with a verb: Add, Update, Fix, Refactor.
Describe what changed, not what you did while changing it.
Keep the first line concise.

Working pattern:

Edit files.
Check git status.
Review with git diff.
Stage with git add.
Commit with a clear message.

Challenge

Exercise 1: First tracked change

Create a file workflow.txt.
Add one line describing your workflow.
Stage and commit the file.
Confirm the commit appears in the log.

Show me the solution

BASH

echo "Use status, diff, add, commit" > workflow.txt
git add workflow.txt
git commit -m "Add workflow summary"
git log --oneline

Track changes over time

Tracking changes is one of Git’s biggest advantages for research and engineering work. You can always inspect how a file evolved and recover context for decisions.

This is valuable in scientific work, where you may need to explain how a figure, table, or derived dataset was produced weeks later.

Show what changed between the working tree and last commit:

BASH

git diff

Show history compactly:

BASH

git log --oneline --decorate

--decorate shows branch and HEAD labels, helping you see where you are in history.

Show a commit and its patch:

BASH

git show

By default, git show displays the most recent commit and its exact patch. You can also show a specific commit with:

BASH

git show <commit-id>

This is the core of change tracking: you can inspect both timeline and content deltas.

Create and use branches

A branch is an independent line of development. Using branches means you can experiment safely without destabilising main.

In collaborative projects, a common pattern is one branch per feature or fix. This keeps main stable and reviewable.

Create a feature branch and switch to it:

BASH

git switch -c improve-notes

This creates the branch and moves you to it in one step.

You can list branches at any time with:

BASH

git branch

The current branch is marked with *.

Make a change and commit it:

BASH

echo "Branch-specific update" >> notes.txt
git add notes.txt
git commit -m "Add branch-specific update"

Switch back to main:

BASH

git switch main

At this point, main does not yet include branch-only commits until you merge.

This separation is intentional: it allows testing and review before integration.

Merge a branch into main

Merging combines histories. In simple cases Git performs a fast-forward merge automatically; if the same lines changed in both branches, you may need to resolve a conflict manually.

If conflicts occur, Git will pause the merge and mark affected files. Typical workflow is:

Open conflicted files and edit to the final desired version.
Stage resolved files with git add.
Complete merge with git commit (if required).

Merge your feature branch:

BASH

git merge improve-notes

Verify history now includes merged work:

BASH

git log --oneline --decorate --graph

If the project uses master instead of main, replace main in the commands accordingly.

Many modern repositories default to main, but older repositories may still use master.

Challenge

Exercise 2: Branch and merge

Create a new branch add-reference.
Add one new line to notes.txt and commit.
Switch back to main.
Merge add-reference into main.
Confirm the merged history with git log --oneline --graph.

Show me the solution

BASH

git switch -c add-reference
echo "Include data file reference" >> notes.txt
git add notes.txt
git commit -m "Add reference note"
git switch main
git merge add-reference
git log --oneline --graph

Key Points

“Install Git locally and configure your identity before starting a project.”
“Create a GitHub account so you are ready for collaboration and remote repositories.”
“git status, git diff, git add, and git commit form the core local workflow.”
“Git history (git log, git show) lets you track what changed and when.”
“Branches isolate work; merging brings reviewed changes back into main.”
“The staging area is central to Git: it lets you build clean, intentional commits.”
“Frequent git status checks help you stay oriented and avoid mistakes.”

Software Carpentry. Version Control with Git. https://swcarpentry.github.io/git-novice/↩︎

All in One View

Overview

Questions

Objectives

Install and open a shell

BASH

Why use the shell?

Opening a shell and running first commands

BASH

Navigating files and directories

Print working directory

BASH

List directory contents

BASH

Change directory

BASH

Exercise 1: Basic navigation

Show me the solution

BASH

Working with files and directories

Create directories and files

BASH

Copy, move, and rename

BASH

Remove files and directories

BASH

Viewing and editing text files

Write text to files with echo, >, and >>

BASH

View file contents with cat

BASH

BASH

Edit files with nano

BASH

Exercise 2: cat and nano basics

Show me the solution

BASH

Exercise 3: echo and redirection

Show me the solution

BASH

Getting help and using tab completion

BASH

Exercise 4: File operations practice

Show me the solution

BASH

Summary

Overview

Questions

Objectives

Introducing Xarray

Datasets and DataArrays

Opening a NetCDF Dataset

PYTHON

PYTHON

PYTHON

Accessing data variables

PYTHON

PYTHON

PYTHON

Indexing and slicing

PYTHON

PYTHON

PYTHON

PYTHON

PYTHON

PYTHON

Exercise 1: Slicing

Show me the solution

PYTHON

Nearest-neighbour lookups

PYTHON

PYTHON

PYTHON

Plotting Xarray data

PYTHON

Plotting two-dimensional data

PYTHON

PYTHON

Plotting histograms

PYTHON