This work offers a necessary and sufficient condition for a stationary and ergodic process to be $\ell_p$-compressible.The condition reduces to check that the p-moment of the invariant distribution of the process is well defined, which contextualizes and extends the result presented by Gribonval, Cevher and Davies in 2012. Furthermore, for the scenario of non-compressible ergodic sequences, we provide a closed-form expression for the best k-term relative approximation error when only a fraction (rate) of the most significant sequence coefficients are kept as the sequence-length tends to infinity. We analyze basic properties of this rate-approximation error curve, which is again a function of the invariant measure of the process. Revisiting the case of i.i.d. sequences, we completely identify the family of compressible processes, which reduces to look at a polynomial order decay (heavy-tail) property of the distribution.