News

This Week in devot:ee #24 - June 28, 2012

June 28, 2012
by Ryan Masuga

This week, a note on 1-star ratings and “censorship” of add-on reviews.

The Rise of the 1-Star Rating

Act I

It started with a 1-star rating and review for Crumbly. A customer buys the add-on, has trouble with a parameter, and claims the developer isn't available to help. To quote the review they left on devot:ee:

I’ve contacted the developer numerous times about couple things that I’ve run into using this addon and I’ve yet to get a a response.

Fair enough at the time it was written. However, shortly after the customer posted this review it became terribly obvious that Stephen Lewis, the developer at Experience Internet, was desperately trying to get in touch with the customer, which rendered the review false and misleading. Stephen posted an open letter to this customer on his blog, tweeted to the customer (and the ExpressionEngine community at large on the #eecms hashtag) to see if anyone knew this person. Stephen also emailed this person, and even called them, only to get hold music.

I contacted Stephen and we had a discussion about the matter and we agreed that we had reasonable cause to remove this review, which was now obviously false, by almost any definition. Stephen has always had a stellar support record and cares about his products and the people that use them.

After removing the rating, the customer finally connects with Stephen to get the help he needed. Than the customer emails me to say that devot:ee is engaging in censorship and that what we're doing is illegal ("I'll guarantee you that and put my money where my mouth is"). He also said that we "decide to remove ratings from software that you profit from especially if the add-ons make more sales, get the picture" (never mind he was responding to my questioning of his other 1-star review on NSM Override CSS, which we have never made a penny from, and which everyone else has given 4 or 5 stars). The customer also states, despite having left a review that was subsequently proved to be untrue, that devot:ee "doesn't give people looking to purchase something through devot:ee an honest reflection of what previous purchasers have wanted to display." He might consider that if his last statement is true, that he is part of the problem.

Act II

Yesterday, Issac Raway emailed be to ask about a 1-star rating (no review!) on his new Shortcode add-on which wasn't even available for purchase or download at the time the rating was placed. What to do? Does putting the developer in touch with the person leaving the rating help the situation (will that person be responsive and think about changing their rating? was it an accident?), or should we risk being accused again of "illegal" censorship because we feel that the rating is deserving of being removed because the user couldn't have possibly used the add-on?

Act III

Today I found a 1-star rating/review on Datagrab, which is a very popular add-on, and it's a proven fact that Andrew Weaver offers fantastic support for his products. The rating and "review" had absolutely nothing to do with the product, or the developer's level of support. In this case I left a note on the review with fair warning that it was going to be removed (and received an email from Andrew thanking me for being diligent about watching those reviews). Judge for yourself. Does the following sound like useful information to you as a potential Datagrab buyer?

(One Star) I tried to purchase this add on 2 days ago and it charged me twice and didn’t let me purchase it. Sadly I need it for a tight deadline but wont be able to buy it. Contacted DEVOTEE and no answer!

We subsequently learned you had likely entered the wrong address for your purchase. There is no need to punish the developer with a bad rating…that’s bad form. Giving you fair warning that I am going to remove this rating and review, because it has nothing to do with the product. - ryan

Do I really need to think twice about removing this? For better or worse, this is a review of devot:ee, not Datagrab. Will this customer come back and yell "censorship"?

The bottom line is this: Use your head when rating and reviewing add-ons. You're a member of the ExpressionEngine community, which says something. Please act like it.

And, you know, we have plans for the weekend. We don't want to be imprisoned indefinitely for removing ratings that have nothing to do with reality and can potentially hurt a developer.

This Week in Add-ons

  • RedirectURL EE2 Plugin (for EE2) by Rodrigo Passos
    Perform 301 Redirect to a given URL
  • Hijri Converter (for EE2) by Laisvunas
    Converts Gregorian date to Hijri and vice versa.
  • Interspire Email Marketer Subscriber (for EE2) by Laurence Cope
    The Interspire Email Marketer Subscriber plugin allows you to easily add user data, including custom fields, to an IEM contact list from an ExpressionEngine template.
  • Shortcode ($, for EE2) by Isaac Raway (Airways)
    Shortcode aims to allow for more dynamic use of content by authors and editors, allowing for injection of reusable bits of content or even whole pieces of functionality into any field in EE. Shortcode provides two types of codes for use in any content in EE: user-defined Macros specific to each author and each site, as well as a simple API for third-party plugins and modules to expose their tags through the shortcode Rich Text Editor dialog (or manual entry in any type of field) - complete with automatically rendered options forms.

9 Comments:

nevsie 06.28.12

nevsie

Hi… and i am completely behind you in this. Just because people are short of time, unable to get things to work, or do not receive instant support does not mean they have the right to review products with a inappropriate score. For example, if one parameter did’t work, yet 4 others worked perfectly, that would suggest at least a 4* review!!!

That said, i think the issue is that the ratings are being used as ransom against developers for the quicker support response. Would a potential solution to this be that people can only rate something 24-48 hours after purchase/download. This will give them time to work it out and resolve before reacting in whim???

Anyhow… change your membership terms. To include “we have the right to remove reviews and ratings where the Devot-ee Team feels they have been used inappropriately”... Or perhaps even a more blanket term!!! Then censorship smensorship - its down to logical opinion of team review.

Kristen Grote 06.28.12

Kristen Grote

I have actually been on the reviewer side of this argument before. Back when Aaron Waldon at Causing Effect first released CE Image, I hastily gave a 4-out-of-5 star review based on the fact that I a) did not contact support when I should have, and b) did not use the plugin correctly. Both issues that were obviously no fault of Aaron’s but spoiled his 5-star streak.

I wouldn’t have thought anything of it if Aaron hadn’t contacted me personally asking why I reserved a star and how he could make the product better. After chatting with him and realizing what an ass I’d been, I was able to get the review bumped up to 5 stars. That’s the kind of extreme level of customer service that Aaron and other add-on developers have come to be known for, which contributes to high reviews.

I’m glad Aaron took the time to contact me, because it made me re-evaluate how I review and think about what’s at stake when I do. If the reviewer isn’t willing to put her money where her mouth is in regard to her review, then she shouldn’t be surprised if it gets removed. These reviews equate to the livelihood of the developers who sell them, and they should be treated seriously by developer, merchant, and reviewer alike.

Chad Crowell 06.28.12

Chad Crowell

Hey Ryan - I know the feeling. Right now you are a bit scared of what someone could do to Devot-ee’s integrity with an internet campaign to defame you because you are trying to make everything fair.

I’ll tell you this, Heidi and I used to rein a web store where we sold some auto accessories. We did very very well with it. Inevitably, here and there someone would be pissed off about something and begin ranting in the forums about it. And we were scared about what *could* happen if that spark caught fire. But eventually, we learned to rely on our track record of great customer service and doing our best even in tough situations, and that the community at large already knew that we were good people trying to do good things. We learned to ignore the threats and they always died on the vine. I hope you can do the same, and the threats do the same.

You have the power of the community behind you. Many of us know you, and I know you quite well. Your reputation holds a lot of water and will stand up to anything along these lines.

JCDerrick 06.28.12

JCDerrick

I think nevsie is right, I think the terms simply need to be adjusted to fairly protect developers against false reviews.

I’ve seen this happen in reverse too however, where 5 stars were given to ask a support question or say a download wasn’t available at the time of the review. While this obviously isn’t as bad or inflaming as a negative review, it does mean it can happen in reverse as well.

I would like to see it where we can change our ratings and/or reviews. I think most negative reviews are generally misunderstandings and could potentially be corrected if the customer could in fact change or update their rating. This also works well so that if I love an add-on, give it 5 stars, and then need support but receive none, I can change my review to perhaps 4 or 3 stars by noting the lack of support. It provides more flexibility this way.

Another idea may be to request a comment/review for any rating, so that users can’t just flood an add-on with negative or positive reviews with no comment to back up the rating.

I think what everyone in the community wants to see is fair ratings given to add-ons and their developers. Some add-ons genuinely deserve one star, others genuinely deserve five. I definitely don’t think you’re censoring anyone, and I think a change in the terms would help protect the integrity of the community as a whole. Keep up the great work.

Euan 06.28.12

Euan

What about have different categories for rating (ie. functionality, ease of installation/use, customer support, etc). That way if someone had the situation of the add-on working as described but no customer support the could rate accordingly.

Natetronn 06.28.12

I don’t understand the reviews of add-ons which have no download or information yet.

I warned airways about this in fact and I’ve seen it before where devs are getting the add-on approved by devot:ee to later fill out the information which seems reasonable to me.

Then again, there could be the argument that add-ons shouldn’t go live if they aren’t ready. Maybe just ratings shouldn’t go live if the add-on isn’t ready?

Euan has very good idea in having areas to rate. That way people can see which area is lacking (or not) and devs can see this as well.

Requiring a comment to rate seems like a good idea as well though, you’ll have less people rating. I think I could live with that.

I remember you removed ratings for Template Variables when it was abandoned and then picked back up by a new dev who got it back in shape and this seemed like a good idea though, was that too censorship? Or erasing history?

Support via forums (or not) has always been an issue and I have to assume that in itself has played into some of the low ratings in the past. Has this been addressed yet?

Euan 06.28.12

Euan

Natetronn, I agree with your suggestion that adding that are not yet available (either free downloads or available purchase) are best kept off the live site.

There is a commercial addon that was added to devot-ee on 21 June but yet is neither available here to purchase Nor the developers website despite sending a tweet asking them how I could buy it.

I too wold like to see comments with ratings. There are some addons with lots of ratings but few if any reviews.

Nicolas Bottari - Zenbu Studio 06.28.12

Nicolas Bottari - Zenbu Studio

+1 on being able to edit ratings. There are times when I wish I could go back on a rating. Ratings and opinions also change across an add-on’s lifetime. I received a 2-star rating when I launched Zenbu, but maybe that opinion has changed since the time the vote was cast (well, I hope so ;)).

I think the Google Play store, for example, allows editing of star ratings and comments: for example, do a good job or add a killer feature, get some extra stars back (if you’ve lost a few). Of course, that shouldn’t be used to threaten the developers: it would be nice to avoid “Add-on doesn’t do (insert unique/obscure need here), 1 star”-type ratings when it’s not warranted.

Also, the ability to write more text in the review comment would be nice. The few times I commented I felt I had little space to fully explain my reasoning behind a rating/appreciation of the add-on.

Michael Rog (RogEE) 07.02.12

One thing I find helpful at larger sites (Amazon, for example) is the ability for the community to rate a review (thumbs-up/down, helpful/unhelpful). That way, when a newbie comes looking, they not only see individual opinions, but also have the context of at-large reactions to those opinions. This works best in situations where there are enough reviews of a particular product for a select few to rise out as “most helpful,” but it’s also relevant for identifying the particularly trollish reviews that the community has deemed to be illegitimate. (I’m not convinced Devot:ee is at a scale yet where such a feature would be warranted; just food for thought.)

You must be registered member to comment. If you're already a member, log in now, and if not go register (it's free and easy!).