They don’t, actually. Not immediately, anyway. In some cases, pokémon simply get rusty with old ones. After all, a trainer is most likely to use the most recent moves they’ve taught their pokémon, and wild pokémon will tend to use the strongest moves at their disposal. Pokémon simply have no reason to use any other move, and thus, they “forget” them in the sense that they ignore they exist until those moves become less familiar to them than the newer ones. It’s a little like learning languages, actually; unless you practice daily, you will very likely forget certain words, even though you’re still somewhat adept at speaking it.
In other cases, the pokémon in question has excellent memory; it’s just that they refuse to use older moves. So unlike the above case, where older moves become rusty, the pokémon has the ability to use any move they’ve ever learned; they simply don’t because it’s rather pointless to do so.
And in other cases, the new move is the older move, just with an increase in power. For example, if you “overwrite” Bite with Crunch, you’re not teaching the pokémon a new move. You’re simply using the principles behind Bite and adding more technique until it becomes Crunch. The pokémon still doesn’t see a point in ever using Bite again, so every time you attempt to order it to use Bite, what it will do instead is basically execute a slightly weaker Crunch than usual—as in, same principle but with more technique and less power.
In short, pokémon don’t literally forget moves. They simply don’t see a point in using anything else once you’ve taught them new techniques, which is a stance that sometimes results in the pokémon becoming rusty at using the older moves.
Unless, of course, you take them to a Move Deleter. In that case, the pokémon’s knowledge of the move is psychically suppressed, so they literally do forget the move in question.
The Pokédex According to Bill 