Valence band holes confined in silicon quantum dots are attracting significant attention for use as spin qubits. However, experimental studies of single-hole spins have been hindered by challenges in fabrication and stability of devices capable of confining a single hole. To fully utilize hole spins as qubits, it is crucial to have a detailed understanding of the spin and orbital states. Here we show a planar silicon metal-oxide-semiconductor-based quantum dot device and demonstrate operation down to the last hole. Magneto-spectroscopy studies show magic number shell filling consistent with the Fock–Darwin states of a circular two-dimensional quantum dot, with the spin filling sequence of the first six holes consistent with Hund’s rule. Next, we use pulse-bias spectroscopy to determine that the orbital spectrum is heavily influenced by the strong hole–hole interactions. These results provide a path towards scalable silicon hole-spin qubits.