Strong spin-orbit interactions make hole quantum dots central to the quest for electrical spin qubit manipulation enabling fast, low-power, scalable quantum computation. Yet it is important to establish to what extent spin-orbit coupling exposes qubits to electrical noise, facilitating decoherence. Here, taking Ge as an example, we show that group IV gate-defined hole spin qubits generically exhibit optimal operation points, defined by the top gate electric field, at which they are both fast and long-lived: the dephasing rate vanishes to first order in the electric field noise along with all directions in space, the electron dipole spin resonance strength is maximized, while relaxation is drastically reduced at small magnetic fields. The existence of optimal operation points is traced to group IV crystal symmetry and properties of the Rashba spin-orbit interaction unique to spin-3/2 systems. Our results overturn the conventional wisdom that fast operation implies reduced lifetimes and suggest group IV hole spin qubits as ideal platforms for ultra-fast, highly coherent scalable quantum computing.