We revisit the impossibility of a variety of cryptographic tasks including privacy and differential privacy with imperfect randomness. For traditional notions of privacy, such as security of encryption, commitment or secret sharing schemes, dramatic impossibility results are known [MP90,DOPS04] for several concrete sources R, including a (seemingly) very “nice and friendly” Santha-Vazirani (SV) source. Somewhat surprisingly, Dodis et al. [DLMV12] showed that non-trivial differential privacy is possible with the SV sources. This suggested a qualitative gap between traditional and differential privacy, and left open the question of whether differential privacy is possible with more realistic (i.e., less structured) sources than the SV sources. Motivated by this question, we introduce a new, modular framework for showing strong impossibility results for (both traditional and differential) privacy under a general imperfect source R. As direct corollaries of our framework, we get the following new results: (1) Existing, but quantitatively improved, impossibility results for traditional privacy, but under a wider variety of sources R. (2) First impossibility results for differential privacy for a variety of realistic sourcesR(including most “block sources”, but not the SV source). (3) Any imperfect source allowing (either traditional or differential) privacy under R admits a certain type of deterministic bit extraction from R.