Friday, October 30, 2009

Amazon Vine

Recently, Betsy Bird of the School Library Journal blog Fuse #8 posted a critique of the Amazon.com "Vine Program" of customer reviews. For those of you who are unaware of the program, here's how Amazon describes it:

Amazon Vine™ is a program that enables a select group of Amazon customers to post opinions about new and pre-release items to help their fellow customers make educated purchase decisions. Customers are invited to become Amazon Vine™ Voices based on the trust they have earned in the Amazon community for writing accurate and insightful reviews. Amazon provides Amazon Vine™ members with free copies of products that have been submitted to the program by vendors. Amazon does not influence the opinions of Amazon Vine™ members, nor do we modify or edit their reviews.

Hardly seems problematic on its face, and I believe that the majority of Vine reviewers are indeed probably "accurate and insightful." But I appreciated Betsy's critique, particularly this quote. Parenthetical notation is mine:

Guess Again! (my book with Mac Barnett) is the straight picture book equivalent of a fractured fairytale, upsetting a reader's expectations, making it hee-larious to kids around the 5-8 year-old age range (I agree with Betsy's age recommendation here, though in the interest of honesty I should point out that the publisher markets it as 4-8). But the Vine program sent the books out to folks with small children who were then shocked SHOCKED when they discovered it wasn't for their tiniest of tiny tots. That would be fine, but then they felt obligated to leave tepid reviews in spite of the fact that they knew perfectly well that their kids weren't the intended audience.

Betsy's post has generated about ten times the comments she could normally expect, and these are mostly from Vine reviewers who feel she's being unfair. I originally thought I should stay out of it, since my book is one of the two examples she uses to explain her position. But that's cowardly.

Amazon reviews (and, presumably, customer reviews in general) have always had a problem with people buying books under false pretenses and then trashing those books for not meeting their ill-informed expectations. The trouble with Vine is that it seems to be institutionalizing these mistakes.

Literally all of the 19 middling-to-poor (3 stars or less) reviews of Guess Again are by Vine reviewers. Of these, a full seven, by my count, base their low rating on the fact that the book failed to appeal to a person for whom it was never intended. In most cases the reviewer was a parent of a 2 or 3-year-old. In one baffling case, the reviewer acknowledged that Guess Again! is a kids' book but explained, "...I am adult and not a 5 year old."

These people are essentially assuming that all clothing is one-size-fits-all, and then grumbling when a grade school uniform is too large for their toddler and too small for their husband.

I hope readers will understand that I do not mind poor reviews if they're thoughtfully written. Kirkus called my novel The True Meaning of Smekday "Inspired but problematic." They felt I'd missed my mark but at least acknowledged that I'd aimed high, and I preferred that review to some which were quite positive but described the book as nothing but a zany romp.

Anyway, the point is probably moot for a number of reasons. Poorly considered customer reviews are unlikely to go away, and they probably have less of an impact on sales than I think–I'm told, despite everything, that Guess Again! is doing all right.

19 comments:

Anonymous said...
This comment has been removed by the author.
Anonymous said...

What meant to say was zany romp and wacky are my two least favorite descriptions in any context...but I'd typed "rump" by mistake. I'd have no problem with that word describing my work if used properly so I removed the comment.

Fuse #8 said...

I dunno. I would definitely read a book if it included "a zany rump".

Jennifer said...

Well, for what it's worth...I read Guess Again (and Billy Twitters) to my 2nd grade school visits (and some of the 1st graders too) a few weeks ago. That's over 100 kids - and they all went NUTS over it. They were laughing, yelling out suggestions, and groaning in unison. But do I read these to my preschool groups? Uh, no. Not unless I happen to have some older kids attending. They'll enjoy them in a few years, when they understand the humor and subleties.

Anonymous said...

GUESS AGAIN is a delight. How can a review be legit if it's considered for the wrong age group? Many wonderful picture books are not for the under 4 crowd. I think that is irresponsible of Amazon.
Angela

MotherReader said...

It's funny, I was annoyed by a review of another of your books, Frankenstein Takes the Cake, where the reviewer noted that her 2yr loved the first book, but now at 4 yrs didn't enjoy the second. So people shouldn't buy the book.

Yeah, I'm sure your 2yr old was enjoying the play on Spy-vs-Spy in the first book, but the Poe Raven reference in the second was too on the nose for him.

Yeesh.

Rooie said...

Problematic??!! Problematic!!!??? I'm sorry but what was problematic about Smekday? I love, love, love that book. I push it on lots of people. I think it's a great book.

Sorry, I know....that's not what we're talking about but on behalf of J. Lo I just had to vent.

Bellatrix said...

Adam,
Your point is well-reasoned. Not all Vine reviews are perfect, and common sense doesn't always prevail. I still support the overall program.

I've tried to be reasonable and factual in this discussion, only to be insulted on Twitter, and treated like a moron or disingenuous for coming from a different place. I think many people just like to fight these days, rather than debate and shake hands at the end of it. Most people have no interest in being persuaded, and so any efforts to do so will be dismissed out of hand and met with derision.

FWIW, I bought your book. I won't review it though, because all my kids have fur.

Best Wishes!

Adam Rex said...

Thanks, everyone, for chiming in.

Bellatrix, I'm not on Twitter, so I probably haven't been getting all sides of this argument. I've mostly just seen the comments on Betsy's site, which are largely pro-Vine and seem to include a depressing number of ad hominem attacks against Betsy herself.
Your name did stand out as someone who was interested in a legitimate conversation. Sorry to hear you've been getting pushed around as well.

Adam Rex said...

And I think I just figured out what you meant by "all my kids have fur."

It's not my position that one needs to have a kid in the requisite age group in order to review a kids' book. Quite the opposite–I think all conscientious reviews should simply consider the intended audience. If you think you have a good grasp on what 6-year-olds like, by all means–review a picture book.
Instead, many reviewers seem to be panning books precisely BECAUSE they don't have a child in the intended age group. Which is just weird.

Anonymous said...

I think that if you read the ad copy for a book and ordered it--thinking from the description that it WAS for your pre-schooler, you have a legitimate complaint to put in your book review. It's exactly what other buyers of Amazon need to know before they buy your book. And you put it in as a two star review because you want others like yourself to see it and be warned.

I think "My two year old hated it, don't buy it for yours," is a really helpful review. It doesn't tell you who SHOULD buy the book, but you can probably find that by scrolling down to the next opinion. That's what I originally liked about Amazon reviews. They were an agglomeration of people's personal opinions NOT an attempt to objectively weigh a book's merits.

Now they have turned into business. Bellatrix, I am sorry people have been rude. I read what you had to say on Betsy's blog and you certainly appear even-handed and reasonable. This is my first comment and I have tried to be polite, but it seems to me that you are being used by Amazon, and that you are defending your own exploitation. That makes me uncomfortable.

Bellatrix said...

I like children's books. I actually tracked down my favorite books from childhood.

I didn't personally catch heck on Twitter, just saw comments to the effect that no one had addressed the issues, which I thought I had, and that we were shallow. It's disheartening to be dismissed when I felt that I, and others, had responded in good faith. For the most part, it just felt like a shame since the people making the comments seemed lovely, admirable, and like people I'd like to know under different circumstances.

One thing I did mention in the thread is that some items are marketed wrong. Sometimes the information we're given in order to select is misleading.

jojoleb said...

I am a Vine reviewer, but never reviewed one of your books.

There are certainly times where the 'directed' monthly list was misdirected and where reviewers get it wrong.

I also commented on Elizabeth Bird's site. I think she made some assumptions that weren't true, but had some quite valid points about the differences between a professional review and a Vine (or layperson's) review.

When you realize that Vine is really more of a marketing tool for Amazon and publishers than a search for a readers opinion it all becomes more clear.

All reviewers on Vine pick selections from a directed list. Most reviewers tend to pick things that they think they will like.

The overwhelming majority of my Vine reviews are four and five stars. This isn't 'grade inflation.' Rather, it is a self-fulfilling prophesy. My list is directed towards me. I try to pick things that I like. Most of the time I'm satisfied, so most of the items get high ratings.

On the whole this is good for Amazon and the publishers, who get a good launching pad for Amazon reviews. Shoppers come to Amazon in part to buy, but also because it is the best site to read about a book or item from people a lot like you who have used an item. No other bookseller on the web has so many reviews.

This is why publishers pay to have their books put on the Vine lists. They want these positive reviews.

The system, however, can backfire for many reasons. Some of them include: if the directed list goes to the wrong people; the book description or age groups listed are incorrect; or the product doesn't live up to expectations.

The first two are remediable. The last one is a matter of product quality and the reviewers' opinions.

It sounds as though your book suffered from the wrong target audience and the wrong expectations from that audience. (Note that the blurbs about the book for the Vine program are different than the final blurb on the product's final page on Amazon. The description may be right on the product page, but wrong on the Vine product page.)

As with any review, you hope that the readers read them carefully and review them fairly. I can only speak for myself in this regard. Still, Bird is quite right that a lay reviewer is probably more likely to miss some salient points about the product, as likely happened in your case.

One of the main problems with Vine, is that Amazon keeps an air of secrecy about the program. Even those of us who are in the program have no specific idea as to why we were chosen and at times it isn't really clear why certain items showed up on our targeted list.

For my part, I try to review the products honestly and with integrity. I read all the Vine books I review cover-to-cover, even if I get a book that isn't so consonant with my tastes.

Adam Rex said...

Thanks, jojoleb. That's an interesting perspective.

Comments on this and other posts have at least taught me that different reviewers are using the Amazon review system in very different ways. A week ago I never would have guessed I'd see someone praise Amazon reviews for being subjective and non-merit based, as Anonymous appears to have done above.

It's interesting that you suggest that the Vine newsletter may include errors that affect targeting, even if those errors are not reflected on the book pages themselves. This shouldn't come as a surprise to me–Amazon used to list me as an author on the GOOD DOG, CARL books for no obvious reason. I pointed out the error, but that situation wasn't remedied for weeks.

Anonymous said...

Wait, wait. They credited you for Good Dog, Carl and you told them to CHANGE it?

I'm your anon at 7:12 and also an author with a number of negative reviews on amazon (the word "boring" appears a number of times). I am surprised that you are surprised that I praised the subjective review. I'll have to give it some more thought as I can see that Colleen Mondor is all about objectively weighing a book's merits as well. It's just . . . I like knowing what a lot of people think about a book. There's no "wrong" when you are giving your own thoughts and describing your own reaction. For example, let me tell you how much I hate "The Pearl" by Steinbeck. My feelings have nothing to do with the quality of the literature, and all about my taste. Warn me, please, amazon readers, if I am looking at a book just like The Pearl.

These "professional" reviews that Collen talks about. I certainly appreciate them, too. But they have downsides-- the biggest is that they *can* be wrong. The people who are being "objective" aren't really that objective at all. They don't know as much as they think they do about child development of children's literature. They are trying to tell me what they think I will think about a book. Dude, I'd rather have you tell me what YOU think of it, and I'll work from there.

Sorry about the long comment. Stopping now.

Kristen McLean said...

I think this discussion has some larger implications for the industry, and that's why it's getting so much play. Here's what I find interesting:

1) Lack of transparency at Amazon

Amazon holds a very influential position in terms of consumer behavior at the moment, and it's not at all clear, even among Vine Reviewers, how they were picked and how their targeted lists are generated.

Publishers are similarly in the dark. I spoke to the head of marketing at one of the larger publishers yesterday who has not yet participated in the Vine Program because her department is unclear on how it works. They have the same questions we do.

I hope this discussion sheds some light on the issue, because I don't think it's a great practice to start a program that gives individuals an influential voice without being clear about who they are and how it works. It does those chosen individuals a disservice, as well as the authors they are reviewing, and it taints the reviews with the air of mistrust. Probably the opposite of Amazon's intent.

I'm sure there are some very thoughtful people in the Amazon Vine program--too bad they are all being lumped together. Because Amazon takes a strictly hands-off approach, it seems like there is no baseline being set for how to write a thoughtful review that tells the readers what they need to know to decide if this book is for THEM. Just saying whether you liked it or not isn't the same thing.

It has ALWAYS been a problem that Amazon reviews can't be modified in any way, even if the publisher or author feels they are hurtful or wildly inaccurate. The fact that these reviewers are working from advances just exacerbates the problem, because they can get their (sometimes unhelpful)reviews out early, and that can dominate the consensus as it did recently for Adam's book or Tony DiTerlizzi's Meno books.

I noticed today that the reviews on the first Meno book are balancing out (to a 3) now that people are posting some more positive reviews, but we live in a blockbuster environment. Early reviews matter, and I for one want them to be as thoughtful and trustworthy as possible.

2) This opens up the larger discussion about the difference between a crowd-sourced model of review information as opposed to an "establishment" model.

What is different about a review from someone who does it for a living versus someone who does not? Is one better than the other? Is one fairer than another? Is there a way to use a crowd-source model that doesn't reduce all ratings to 3 stars over time? What about special books that don't appeal to all readers, but are for a particular audience? Can I still find them in a crowd-sourced review environment? Will publishers put them there?

As the professional sources for mainstream reviews are dwindling, is this the only way to do it? I'm not so sure. I'd like a new model that puts a trusted POV back into the equation, and I'd like not to have to hunt and peck across a thousand blogs to find it. I think it's interesting what the crowd thinks, but that doesn't necessarily mean I trust it to align with my own tastes.

continued below....

Kristen McLean said...

3) How important it is to get the information about audience and content right.

I'm not sure the good folks at Simon & Schuster thought about the possible implications of putting these kinds of offbeat books into the Vine Program. I guarantee they will now.

I think this is particularly true about DiTerlizzi's Meno books which are a BIG departure from his previous blockbusters for middle grade readers.

So often in the marketing process books are promoted on the basis of the author's previous record. The 10 second handle is 'The newest book from best-selling author XX.' This works great for series and genre books, but is an obvious failure for new books that seriously deviate from the author's previous work. And the truth is, when a publisher/marketer/sales team is dealing with a list of 100 or more books a season, some of the nuance is lost in the presentation.

If it is true that publishers are paying for the Vine program--I'm still unclear about this--I'm sure S&S isn't happy to pay for the privilege of having it's books trashed in early reviews.

Some of the blame lies in errors in the information process at the publishers. Marketing departments are crafting materials as the books are being published, and sometimes that info needs to change with the finished book, but can't once it gets out into the world.

Sometimes publishers default to set categories: picture books are usually labeled 4-8 as a default, even when they should be 5-8 like Adam's book.

Once the bibliographic information is released by the publishers (way before the book it finished) it's like letting the genie out of the bottle. What is done can't be undone, and then anything that draws from that info (like Amazon Vine target lists) is corrupted.

***

I'm sure this discussion will continue to resonate in many different conversations across the industry. In the end authors will need to take an active role in overseeing and commenting on these issues, because that is the place where their voice can be heard.

They aren't the ones sitting in a cubicle deep in the bowels of a publisher, plugging the bibliographic data into a computer before upload to the web. "4 or 5 to 8? It's not that big a deal, right?" Wrong, but the ship has long sailed.

Authors, your readers do care about what you have to say. Speak out, and people will listen. It can be a game changer, and perhaps people will be a little more thoughtful all around the next time they deal with a book in the pipeline.

-Kristen
The ABC

Adam Rex said...

Hi Anonymous and Kristen.

I'm certainly interested in conversation about some of the broader issues surrounding the Vine program, and I hope this conversation will continue. You've given me a lot to chew on, Kristen.

But I feel I ought to reassert that my post was a fairly specific platform in which to vent my frustration over people reviewing books under the pretense that they should appeal to readers for whom they were never intended.

(Badly considered or misinformed reviews are hardly just the purview of customer sites like Amazon, of course. I was talking to an author recently about his book–he told me a NYT reviewer criticized it for failing to put kids to sleep. As though all picture books are, first and foremost, sedatives.)

I have nothing against subjective reviews per se, and of course I understand that "objective" reviews can be mistaken. Anonymous, your example of THE PEARL is a valid customer review because you're the intended audience. If you explain WHY you didn't like the book, rather than just typing BORING and leaving two stars, then your review might even be helpful to others. But if you bought Steinbeck's novella for your five-year-old who loves pearls and then trash it because SHE didn't like it? Not so much, in my opinion.

Maybe the real crux for me is the star rating system. A grossly misinformed review wouldn't sting much if I didn't know it was dragging down an overall grade. A grade which most amazon customers will interpret as a "wisdom of crowds" verdict on the book's quality and value to its intended audience.

There are other reviews on Amazon that frustrate me, but I can't really object to them on any legitimate grounds. In at least one review of GUESS AGAIN! the reviewer makes it clear he or she simply didn't understand the premise of the book. It's entirely subjective, but I can't say it's not valid or useful.

Anyway, this should be the last I write on this subject. I wrote the original post largely because I didn't like the amount of heat Betsy Bird was taking on my behalf. I've tried to be clear that I don't mind negative reviews simply because they're negative, but if I keep trying to explain and re-explain my position there's no way I don't start coming off as petty, if I haven't crossed that line already.

Bellatrix said...

I just wanted to let you know that I received Guess Again and liked it quite a lot. :)

At some point in the future, I might review it, when the arguments have died down more, and when I think some of the vindictiveness against people of my evil ilk -- Vine members -- might have given way to cooler heads and a reminder that most of us have a love of books in common.