A hard disk can be divided into several partitions. Each partition functions as if it were a separate hard disk. The idea is that if you have one hard disk, and want to have, say, two operating systems on it, you can divide the disk into two partitions. Each operating system uses its partition as it wishes and doesn't touch the other ones. This way the two operating systems can co-exist peacefully on the same hard disk. Without partitions one would have to buy a hard disk for each operating system.
Floppies are not usually partitioned. There is no technical reason against this, but since they're so small, partitions would be useful only very rarely. CD-ROMs are usually also not partitioned, since it's easier to use them as one big disk, and there is seldom a need to have several operating systems on one.
The MBR, Boot Sectors and Partition Table
The information about how a hard disk has been partitioned is stored in its first sector (that is, the first sector of the first track on the first disk surface). The first sector is the master boot record (MBR) of the disk; this is the sector that the BIOS reads in and starts when the machine is first booted. The master boot record contains a small program that reads the partition table, checks which partition is active (that is, marked bootable), and reads the first sector of that partition, the partition's boot sector (the MBR is also a boot sector, but it has a special status and therefore a special name). This boot sector contains another small program that reads the first part of the operating system stored on that partition (assuming it is bootable), and then starts it.
The partitioning scheme is not built into the hardware, or even into the BIOS. It is only a convention that many operating systems follow. Not all operating systems do follow it, but they are the exceptions. Some operating systems support partitions, but they occupy one partition on the hard disk, and use their internal partitioning method within that partition. The latter type exists peacefully with other operating systems (including Linux), and does not require any special measures, but an operating system that doesn't support partitions cannot co-exist on the same disk with any other operating system.
Primary Partition Rules and Conventions
You can only create four primary partitions on any single physical hard drive. This partition limit extends to the Linux Swap partition as well as for any Operating System installation or extra special purpose partitions, such as separate /root, /home, /boot, etc., that you might want to create.
If you attempt to create more than four primary partitions, you will see a warning: "It is not possible to create more than 4 primary partitions." You do NOT have to have 4 primary partitions. If you decide to create an extended partition, be aware that it is counted as one of the 4 partitions.
"5.9. Partitions" by Multiple contributors, The Linux Documentation Project is in the Public Domain, CC0
"How to Partition / Partitioning Basics" by Ben64, Community Help Wiki is licensed under CC BY-SA 4.0