Festinger's Dissonance theory explains the result. The “Dissonance” is between the actions of the participants and their beliefs about themselves. Here they are, nice guys, lying to an innocent woman. Admittedly there are lots of other social forces at work – obligation, authority, even attraction. Festinger's interpretation is that these things may play a role in how the participants act, but they can't be explicitly relied upon as reasons for acting. So there is a tension between their belief that they are a nice person and the knowledge of how they acted. This is where the cash payment comes in. People paid $20 have an easy rationalisation to hand. "Sure, I lied", they can say to themselves, "but I did it for $20". The men who got paid the smaller amount, $1, can't do this. Giving the money as a reason would make them look cheap, as well as mean. Instead, the story goes, they adjust their beliefs to be in line with how they acted. "Sure, the experiment was kind of interesting, just like I told that girl", "It was fun, I wouldn't mind being in her position" and so on.
So this is cognitive dissonance at work. Normally it should be a totally healthy process – after all, who could object to people being motivated to reduce contradictions in their beliefs (philosophers even make a profession of out this), but in circumstances where some of our actions or our beliefs exist for reasons which are too complex, too shameful, or too nebulous to articulate, it can lead to us changing perfectly valid beliefs, such as how boring and pointless a task was.
Fans of cognitive dissonance will tell you that this is why people forced to defend a particular position – say because it is their job – are likely to end up believing it. It can also suggest a reason for why military services, high school sports teams and college societies have bizarre and punishing initiation rituals. If you've been through the ritual, dissonance theory predicts, you're much more likely to believe the group is a valuable one to be a part of (the initiation hurt, and you're not a fool, so it must have been worth it right?).
For me, I think dissonance theory explains why some really long books have such good reputations, despite the fact that they may be as repetitive and pointless as Festinger's peg task. Get to the end of a three-volume, several thousand page, conceptual novel and you're faced with a choice: either you wasted your time and money, and you feel a bit of a fool; or the novel is brilliant and you are an insightful consumer of literature. Dissonance theory pushes you towards the latter interpretation, and so swells the crowd of people praising a novel that would be panned if it was 150 pages long.
Changing your beliefs to be in line with how you acted may not be the most principled approach. But it is certainly easier than changing how you acted.