There is a known psychological effect where giving people a reward for doing an activity makes them do it much less once the reward is gone, compared to if they never got the reward at all. See the wikipedia writeup about extrinsic motivation:
Extrinsic incentives sometimes can weaken the motivation as well. In one classic study done by green & lepper, children who were lavishly rewarded for drawing with felt-tip pens later showed little interest in playing with the pens again.
Perhaps the contest holders are willing to accept this negative hit of user retention in exchange for getting their name out, as people tell their friends about the contest. In that case, it's important to design your competition for maximal reach. Make sure the goal aligns with spreading the word about your site, and not about encouraging heavier activity amongst your existing users.
You could take this to its natural extreme and just have a contest of who can refer the most number of users to your site.