A water softener reduces the calcium or magnesium ion concentration in hard water. These "hardness ions" cause three major kinds of problems. The metal ions react with soaps and calcium sensitive detergents, hindering their ability to lather properly and forming an unsightly precipitate— the familiar scum or "bathtub ring". Presence of "hardness ions" also inhibits the cleaning effect of detergent formulations. More seriously, calcium and magnesium carbonates tend to adhere to the surfaces of pipes and heat exchanger surfaces. The resulting scale build-up can restrict water flow in pipes. In boilers, the deposits act as thermal insulation that impedes the flow of heat into the water; this not only reduces heating efficiency, but allows the metal to overheat which, in a pressurized system, can lead to failure. The presence of ions in an electrolyte can also lead to galvanic corrosion, in which one metal will preferentially corrode when in contact with another type of metal. The use of water softeners can aggravate this and cause sacrificial anodes in hot water heaters to corrode more quickly.

Conventional water-softening depend on an ion-exchange resin in which "hardness" ions trade places with sodium ions that are electrostatically bound to the anionic functional groups of the polymeric resin. A class of minerals known as zeolites also exhibits ion-exchange properties; these minerals were widely used in earlier water softeners.

Water softeners are typically used when water is supplied from wells. Public water systems are also susceptible to hard water, although this is much less common.