He thinks he is talking about sex, but there are some important point he overlooks. Most people would usually want their button pressed by members of the other sex. Most people naturally close to you (parents, siblings, cousins) would be out of the question as pushers of your button.
Also, over millions and millions of years, certain aspects of hominine reproduction (large heads, long gestation times, preposterously long childhood, staggering maternal mortality) and simple statistics would have conspired to make women very, very picky about when and by whom they want to press their buttons. At the same time they would have conspired to make men want their buttons pressed as often as possible, and by pretty much any girl with a neck pulse.
The availability of people of the desired sex willing to press your button would be your default main source of serenity and self-respect. It would take experience and hard work for you to create other sources of fulfillment for yourself. It would be natural for immature, stupid, or bitter people to spend enormous amounts of energy trying to control access to other people's buttons.
If you could press your neighbor's button without risk of adverse side effects (killing months of their productivity, having them lose one of their four of five shots at producing healthy offspring, making them die in childbirth) offering someone to pleasure of a press of their button would be as harmless as offering them the pleasure of a smile, or of a glass of drink. There would be certain rules of etiquette, but there wouldn't usually be much debilitating drama.
At first I thought that was the idea, but I honestly can't believe someone so successful at the satire of modern social interaction could so tremendously oversimplify the emotional and biological labyrinth of human sexuality into a mere "button pressing" that instantly makes the recipient "happy."
I think Adams' point is simply that it's not that hard to make someone's day better.
Yes, he's talking about sex and it's pretty sad to see how a nerd sees sex. He's missing SO many aspects and simplifying everything to a button. "Engineering" at its finest.