What to consider when designing an online community rep system like SO's?

Question

I'm designing a website and want a good reputation system like they built into StackOverflow.

When I started to design my reputation system, I realized the SO Q/A model doesn't quite fit into my app (since mine is not Q/A), and that perhaps I should do some research first.

So my question is; What should I consider when designing a community rep system?

Answer 1

You first need to decide whether your system should more democratic (ala SO - one voce one vote) or more expert driven (some pigs... errr voters are more equal than others).

An example of the latter is giving the voter more weight based on either:

• Their reputation, or

• Their "accuracy" - e.g. they voted similarly with other voters. This one is VERY tricky to get right and likely requires NOT showing a total post vote until AFTER someone voted.

Decide which features are most important to you. Prevention of abuse? Avoiding certain types of biases?

The important functionality that differentiates things is:

• "Herd-blind" voting (you don't get to see the posts's ranking until you vote or state "I will not vote"). This avoids the pressure to vote with the herd.

• Anonymous votee. You don't get to see the author of the item till after you vote or decline to vote. Prevents "Jon Skeet" bias, for lack of a better term (or zillion upvotes on Joe's Turtle question).

• Anonymous vs. roll-call voting. SO obviouly has the former - it prevents retaliation. OTOH, roll-call (e.g. identifing who cast which vote) promotes more responsible voting and curbs SOME abuses to certain extent.

• Limitations on voting (# of votes per voter, per votee, per voter-per-post, etc...)

• Ability to take back/change the vote

• Ability to vote on attributes as opposed to post as a whole (think technical score vs. artistic score in figure skating or gymnastics)

• Ability to give configurable amount of vote, either by user's decision or by context.

  As an example of the latter, think +5 for Q/+10 for A/+0 for CW on StackOverflow these days.

• Granularity of rankings. E.g. SO's exact point system vs. "/." 1-5 Karma vs. something else.

• Vote sharing. E.g. on SO you never get the rep if you edit the post significantly enough that it generates additional rep. And you can't share rep gained for an answer with someeone who contributed good ideas for an answer via comments.