Intro
I just got a new disk, installed it physically into my Ubuntu Server, and realized that I’ve forgotten everything to do with disk formatting/partitioning, etc. Here I remind myself as to what exactly all of these are concepts are and how to work with them.
Identifying Disks
When you’ve got a disk installed for the first time, run lsblk -o name,fstype,size,mountpoint,partuuid. If your disk is in a blank state, then you will probably only get this sort of info:
├─sdb1 vfat 512M /boot/efi 79033c3d-...
└─sdb2 ext4 931G / 5c6b1ad5-...
...
nvme0n1 3.6TIn this case, I can see my disk has been detected and given the handle /dev/nvme0n1, and that it has a size of 3.6T, but that is it.
Partition Tables
Every disk needs to have some space dedicated to a partition table, that is, a space that enables interfacing systems to determine how the disk is partitioned. There are different standards (i.e. conventions) for how such tables are to be organized and read.
A widely-supported type of partition table is the “GUID Partition Table” (GPT). GUID stands for “globally unique identifiers”.
To see what partition table is used by a given disk, you can use ‘print’ within the wizard launched by parted in the following manner:
❯ sudo parted /dev/nvme0n1
GNU Parted 3.4
Using /dev/nvme0n1
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Error: /dev/nvme0n1: unrecognised disk label
Model: CT4000P3PSSD8 (nvme)
Disk /dev/nvme0n1: 4001GB
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:In this case, we are told that we have an “unrecognised disk label”, and that the Partition Table is “unknown”. These two issues are one and the same: the ‘disk label’ is just another name for the partition table, and we don’t have one.
Typing “help” will show the actions that we can take, in particular, it tells us we can create our partition table using either mklabel TYPE or mktable TYPE. Type gpt is ‘the’ solid and universally supported type, so I just went with that: mklabel gpt. Now when I type print I do not get the earlier error messages.
Note: if you want to have a table of type ‘Master Boot Record’ (MBR), then you enter mklabel msdos. I have read that “GPT is a more modern standard, while MBR is more widely supported among older operating systems. For a typical cloud server, GPT is a better option.” Also, if you erase a disk on a Mac with Disk Utility, it gives you the option of an ‘Apple Partition Map’ (APM) for its ‘Scheme’ (as Mac calls the partition table). I did some light googling on this Scheme, and concluded that it is outdated — used for PowerPC Macs — and that the most recent Apple disks use GPT. In short, whenever you have to decide what Scheme to use, use GUID.
Creating Partitions
Now that we have a partition table, we can insert information into it to establish partitions. Still using parted, we can run mkpart as a wizard of sorts. This will prompt you for information about the partition you want to create.
The first thing it asked me for was a “Partition name? []”. This was confusing, because the arguments for mkpart given by ‘help’ are: mkpart PART-TYPE [FS-TYPE] START END.
So the argument is calling it a ‘type’, and the wizard is asking for a ‘name’. What’s going on? The confusion stems from different conventions used by different partition-table types. For partition tables of type msdos or dvh, you can specify a ‘type’ from one of these three options: ‘primary’, ‘extended’ or ‘ logical ‘. The wizard for parted mkpart is geared towards setting up partitions governed by msdos partition tables, hence why its documentation calls it a ‘type’. However, these categories do not apply to GUID partition tables. Instead, GPTs have a category that msdos does not have — a ‘name’ — which you have to specify (though it can just be an empty string). Hence when the wizard detects that you are filling in a GUID table, it prompts you for a ‘name’ instead of a ‘type’.
What is the GPT ‘name’ used for? It can be used to boot the drive from /etc/fstab (see below). It does not determine the name of the handle in /dev/* for the partition (which, from what I can tell, is determined solely by the OS).
Next, it asks for a “File system type? [ext2]”. The default is ext2. Note that parted does not go ahead and create a filesystem on this partition; this is just a ‘hint’ being recorded in the table as to the intended use of the partition, and certain programs might use this hint when e.g. looking to auto-mount the partition. Here I chose ‘ext4’ — the standard file system for linux.
Next it asks for the ‘Start’ and then ‘End’ of the partition. Because I am only making one partition, and I want it to take up all the available space, I could just put 0% and 100% respectively. If you have to work around other partitions, then you need to note their start and end positions first using parted print, etc.
To summarize so far, every disk needs to have it’s first few blocks set aside for a partition table. Partition tables are, no doubt, designed by strict conventions to convey to any interface as to what sort of partition table it will be dealing with (I’m not sure how exactly, but I’d guess it’s something like the very first byte tells you the type from 256 options established by the comp sci community). The partition table then in turn records where each partition begins, ends, its type, its name, etc. The only role of the program parted is to set/adjust/erase the contents of the partition table. (As far as I can tell, it does not read-write to any part of the disk outside of the partition table.)
Note: if you delete a partition with parted, I expect that all that that does is remove the corresponding entries from the partition table, without erasing the information within the partition table itself. I therefore expect — but can’t be bothered to confirm — that if you were to then recreate a partition with the exact same bounds, that you would be able to re-access the file system and its contents.
File Systems
Now that the partition table has been created, our disk has a well-defined range of addresses to which we can read/write data. Now, in order to store data into the partition in an organized and efficient manner, we need another set of conventions by which programs acting on the data in the partition can operate. Such conventions are called the ‘file system’. As with the partition table being the very first thing on the disk, modern ‘hierarchical’ file systems (i.e. ones that allow for folders within folders) work by reserving the very first section of the partition a space that describes the contents and properties of the root directory, which in turn points to the location of data files and other directory files within it. Those directory files in turn point to the location of files and directories within them, etc. For an excellent, more-detailed overview of these concepts, see this video.
Now, having quit parted, we can create a file system within our partition with:
sudo mkfs.ext4 /dev/nvme0n1p1In practice, there can only be one file system per partition, so you don’t need to think about what sort of space the file system takes up — it is designed to work within the boundaries of the partition it finds itself in.
Mounting
To mount the partition temporarily, you use sudo mount /dev/nvme0n1p1 /mnt/temp where the dir /mnt/temp already exists. To have the partition mounted automatically on boot, add the following line to /etc/fstab:
PARTLABEL=NameYouChose /mnt/data ext4 defaults 0 2…where:
- PART
LABEL=NameYouChoseis the criterion by which the OS will select from all detected partitions /mnt/datais the path where the partition is being mountedext4signals to the OS what sort of file system to expect to find on the partitiondefaultsmeans that this partition should be mounted with the default options, such as read-write support0 2signifies that the filesystem should be validated by the local machine in case of errors, but as a2nd priority, after your root volume
To put this line into effect without rebooting, run sudo mount -a.
Note: just to make things a little more confusing, you can also use mkfs.ext4 with the -L flag to set yet another kind of ‘label’ within the partition table. If you use this label, then you can use it to mount the partition in the /etc/fstab file using LABEL=FlagYouChose (instead of PARTLABEL=NameYouChose using parted above).
Volumes, Containers, etc.
As a final note, sometimes the term ‘Volume’ is used in the context of disks and partitions. The important thing to note is that the term is used differently on different platforms. In Appleland, the APFS has ‘Containers’, ‘Volumes’ and ‘Partitions’ as distinct constructs. From what I can tell, a Volume in Appleland is synonomous with Logical Volume in other lands. An example of a Logical Volume is a RAID1 set up where you have two disks, your data is duplicated on one of the disks, but you interact with that data as though it is in one place (i.e. the fact that the data has been spread across two disks has been abstracted away and hidden from you). In general, a LV can be spread across multiple physical disks, but is presented you as though you were dealing one old school physical disk.
It’s not clear to me at this time what a Mac ‘container’ really is.
