Spatial frequency is an elementary aspect of visual perception. Moreover, faces elicit distinct responses by the human visual system when compared to other visual objects. This review examines the factors influencing spatial frequency processing in faces. Visual perception of a face results from the interaction between the physical properties of the stimulus and the human visual system. We first review the methodology of visual stimulus production and presentation in the assessment of spatial frequency processing in faces. Image properties interact with the physical constraints of the visual system. Other cognitive phenomena also drive the processing of spatial frequencies in faces. Finally, the observer’s characteristics may further influence this spatial processing. Overall, the studies indicate that spatial frequency processing in faces is not a fixed process, conditioned by physical constraints alone, but a flexible process, dependent of various cognitive constraints, developmental, and health conditions. Finally, limitations and new challenges are discussed.