Friday, July 07, 2006

Star Calculations

Several people expressed interest in the star calculation method at PatternReview. I will give you what I think is an easier-to-understand explanation.

There are 3 components of your "score".

  1. Number of Reviews written. (This includes tips, and reviews in all categories; patterns, books, websites, stores, etc.) The breakpoints are 1, 26, 51, 76, 101, 126, 151, 175.
  2. Number of people who "favor" you. The breakpoints here are 1, 11, 21, 31, etc. up to 90.
  3. 25% of the average rating on all your reviews. This is the tricky one.

    As you know, you can rate other people's reviews. Here are the 4 ratings, and the points each represents.
  • Very Helpful +2
  • Helpful +1
  • Needs More Info +0
  • Off-Topic -1

You must make a chart or something (Excel is handy) with all your reviews and go through and write down how many helpfuls, very helpfuls, etc. you have on each review. Then add the number of points for each review individually, divide by the number of ratings on that particular review and that's the average for that review. Add up all the averages, divide by the total number of reviews, and then take 25% of that number, and that's the third part of the score.

Example:

  • 3Very Helpful, 2 Helpful on a review would give you a total of 8, divided by 5 raters = 1.6.
  • If you had another review with 15 Very Helpfuls and 1 Helpful, 31 divided by 16 = 1.9375.
  • One more with 2 Very Helpfuls only would be 4 divided by 2 = 2
  • Total averages: 5.5375 divided by 3 reviews = 1.84583
  • Multiply by .25 = 0.4614

You can see from this method that 2 is the highest average you can get on any particular review, and if anyone even gives you one Helpful rather than a Very Helpful, that drags you down quite a lot. I was really surprised to find out that Helpful is bad, apparently.

When you add the three parts of your score, you must then divide by 3, and that is your Star Rating Number. They're all recalculated every Monday morning, although I think that once you have a star, you keep it, no matter how many people unwittingly may deem you Helpful!

So, taking this example, your components would be:

  • 0.25 for a total of 3 reviews
  • 0.2 for 6 people who favor you
  • 0.4614 for ratings on your reviews
  • Divide by 3
  • Total is 0.3038 for 1 Star.

You must have at least 0.3 to get 1 Star, 0.6 for 2 Stars, 0.8 for 3 stars, 1.1 for 4 stars, and over 1.5 for 5 stars. The easiest thing to raise is number of reviews, as you control that. Next easiest is number of persons who favor you, and hardest is your review rating, as you're likely to get Helpfuls as well as Very Helpfuls on your reviews, and you may have reviews with no ratings at all, which of course will draw your average down as well. No matter how many Very Helpfuls you have though, 0.5 is the most that you can get in the rating category.

All this militates toward members writing many reviews, which of course is the main purpose of the site, so this formula gives the edge to quantity.

Thanks to Irina for her Very Helpful help!

2 comments:

Carolyn in AR said...

Liana -- I believe I spot an error in your calculation. You should have multiplied the average of the ratings by .25 (25%)rather than .025 (2.5%).

This person's score would then be .91/3 or .3, giving her a star.

Carolyn (Irina on PR)

Liana said...

You're right Irina! Actually, if you write 1 review, have 1 person favor you, and get 1 Very Helpful on your only review, with no Helpfuls, you should get 1 star for that. But, to move up to 2 stars, you would have to write at least 26 reviews and get over 20 people to favor you, while getting nothing but Very Helpfuls. It mainly seems to reward you for number of reviews and number of people who favor you. You're almost sure to have someone unknowingly give you a Helpful rating, and ruin your average there, so write lots of reviews. Since the site depends on having as many reviews as possible, this does work to further that outcome.

I'll change my post to fix the math. :)