It has become increasingly clear that even occasional exposure to loud sounds in occupational or recreational settings can cause irreversible damage to the hair cells of the cochlea and the auditory nerve fibers, even if the resulting partial loss of hearing sensitivity, usually accompanied by tinnitus, disappears within hours or days of the exposure. Such exposure may explain at least some cases of poor speech intelligibility in noise in the face of a normal or near-normal audiogram. Recent findings from our laboratory suggest that long-term changes to auditory brain function—potentially leading to problems with speech intelligibility—can be effected by persistent, passive exposure to more moderate levels of noise (in the 70 dB SPL range) in the apparent absence of damage to the auditory periphery (as reflected in normal distortion product otoacoustic emissions and auditory brainstem responses). Specifically, passive exposure of adult cats to moderate levels of band-pass-filtered noise, or to band-limited ensembles of dense, random tone pips, can lead to a profound decrease of neural activity in the auditory cortex roughly in the exposure frequency range, and to an increase of activity outside that range. This can progress to an apparent reorganization of the cortical tonotopic map, which is reminiscent of the reorganization resulting from hearing loss restricted to a part of the hearing frequency range, although again, no hearing loss was apparent after our moderate-level sound exposure. Here, we review this work focusing specifically on the potential hearing problems that may arise despite a normally functioning auditory periphery.