Gluey mashed potatoes are more than just unfortunate—they're usually a lost cause. Overcooked or insufficiently drained potatoes can become sticky, as can the wrong kind of potato. But the main problem is overworked spuds. The science is simple: Boiled potatoes develop swollen starch cells. When ruptured during mashing, the cells release starch. The more cells are ruptured, the gummier the mashed potatoes. So if you use an electric mixer or food processor to mash your potatoes, you'll probably beat them mercilessly and end up with wallpaper paste. Instead, use a potato masher, or even better, pass the potatoes through a ricer or food mill before mixing them with butter and hot milk—these devices are gentler on the starch cells, and they'll also prevent lumps.
Low-starch (or waxy) red potatoes hold their shape well after boiling, so they require more effort to mash. Hence, you're likely to overwork them. Try mashing them just partway, as in our Herbed Smashed Potatoes. By contrast, high-starch (mealy or floury) baking potatoes, also called russets, break down more readily, yielding light and fluffy mashed potatoes (or, with a little more milk and butter, smooth and creamy).