As commonly understood, the noise spectroscopy problem—characterizing the statistical properties of a noise process affecting a quantum system by measuring its response—is mathematically ill-posed, in the sense that no unique noise process leads to a set of responses unless extra assumptions are taken into account. Ad-hoc solutions assume an implicit structure, which is often never determined. Thus, it is unclear when the method will succeed or whether one should trust the solution obtained. Here, we propose to treat the problem from the point of view of statistical estimation theory. We develop a Bayesian solution to the problem which allows one to easily incorporate assumptions which render the problem solvable. We compare several numerical techniques for noise spectroscopy and find the Bayesian approach to be superior in many respects.