tag:blogger.com,1999:blog-5439941893980599296.post2445988444362457539..comments2023-09-24T07:49:19.084-04:00Comments on Games By Design Has Moved!: Reviewing A Bad ReviewChristopher M. Parkhttp://www.blogger.com/profile/16719365007524426389noreply@blogger.comBlogger14125tag:blogger.com,1999:blog-5439941893980599296.post-37976716551368089092009-10-28T08:21:49.266-04:002009-10-28T08:21:49.266-04:00Christopher,
I have just spent 40 minutes install...Christopher,<br /><br />I have just spent 40 minutes installing the pre-requisites for AI-War demo and so I am writing this before I have to do the mandatory re-start of Windows.<br /><br />I am a long time .NET developer and I understand completely that this is not something you have any control over. Apart from the (understandable) choice to use .NET 3.5 technologies in AI-wars, that is.<br /><br />The major problem for me is that the .NET 3.5 SP1 installer is so awful. The progress bar jumps all over the place, eg. it gets to 6 MB of 56MB and then goes back to 0MB. I cannot believe a company like Microsoft can release such utter crap. <br /><br />The big problem for you as the vendor, is that in the eyes of most customers you created and own the entire install and you are making them go through it. It's even worse when all they want to do is play your demo just to see if they want to purchase it.<br /><br />You are not the only company to have exactly this problem with .NET installation. See Paint.NET at: http://blog.getpaint.net/2008/08/24/the-paintnet-install-experience-part-1-version-3xx/<br /><br />Also see GiniPic at: http://ginipic.com/blog/?p=24#comments<br /><br />On some further research, I believe the SP1 install could be using the Client Profile which may explain the unacceptable progress bar behaviour.<br /><br />The reason I felt the need to comment is that I currently stick to .NET 2.0 in my applications for exactly this reason. Until XP goes away and everyone gets .NET 3.5 + by default, the level of frustration and inconvenience to customers/users is just stupid.<br /><br />If your customers are technically advanced / developers, they can probably handle this with out major issues. If they are non-technical "normal people", I can see the majority thinking the effort to install is just not worth it.<br /><br />Once again I am a developer myself and know exactly where you are coming from. I think your negative review probably greatly stems from from the negative install issues. <br /><br />From a selfish viewpoint, it is also incredibly instructive to see real-world issues regarding deploying .NET to customers, and how to overcome them, so thanks.<br /><br />Anyway, best of luck with AI-wars, I'm looking forward to trying the demo , after I restart of course ;)<br /><br />AshleyAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-5439941893980599296.post-56374526450901947682009-08-24T08:00:38.555-04:002009-08-24T08:00:38.555-04:00Lars-
I think you are echoing a lot of what Chris...Lars-<br /><br />I think you are echoing a lot of what Chris and I discussed. The scoring system certainly has been brought to our attention and will be addressed. I think the problem in general is that when the system was originally conceived it was presented to me as a tool for developers to gauge their demo (not their game). In reality, the reviews need to be for the gamers and readers.<br /><br />I disagree that "most" of the reviewers are negative. I think there are 1 or 2 who are problematic or overly negative.<br /><br />Finally, having a .zip file for your demo is totally unprofessional for an indie developer. I agree that this shouldn't be weighed the same as gameplay, but don't defend someone distributing a game as a zip file. Commercial AAA titles don't ask customers to unzip a folder to play their game and neither should indies.Mikehttps://www.blogger.com/profile/06764539910300982854noreply@blogger.comtag:blogger.com,1999:blog-5439941893980599296.post-33188646152373108082009-08-23T21:48:20.795-04:002009-08-23T21:48:20.795-04:00Mike: I found out about the 10 point criteria afte...Mike: I found out about the 10 point criteria after reading several demo reviews and was completely surprised to see it was so strictly laid out, in no small part because the reviewers remove points for whatever they feel like (which is crazy. You get one point for good gameplay, but can lose one because the game comes in a zip file instead of using an installer?). I accept that there are differences in opinion, but the review scores vary for reasons beyond what I consider to be taste unless writers are reviewing games in genres they dislike, leaving them unlikely to enjoy even the best examples of that sort of game. One set of reviews I looked at gave two 1's, a 5, and an 8. I don't believe that the criteria can be so differently interpreted in any responsible way that scores can range from 'downright terrible' to 'very good'.<br /><br />The reviewers in general were very negative and sometimes seem to parade around their ignorance of the very game they're reviewing, commenting on how little they bothered to play it before writing an insulting review that will be telling readers to stay away from the game. Reviewers don't exist to throw insults around at games and the people who made them. They exist to inform their readers, the potential players of the game who are interested in it enough to read the review. I feel that some strong thought needs to be put into this section of the publication so that it better benefits its readers and doesn't a disservice to them or the very games that allow the magazine to exist in the first place. A bad review for a bad game is fine, but please try to be informed about them and less superficial, even if it is only for a demo; it still carries a lot of weight. The reviewers supposedly have enough appreciation of indie games to be reviewing them for a magazine, so more effort to stay true to that would be a really great step.Larsnoreply@blogger.comtag:blogger.com,1999:blog-5439941893980599296.post-6253409361815524272009-08-20T12:41:51.963-04:002009-08-20T12:41:51.963-04:00Mike -
Fair enough about Cas, he's his own pe...Mike -<br /><br />Fair enough about Cas, he's his own person. Pass on my comments if you wish, but I'm not sure I care to have any discussion with him on the topic. He does not strike me as a reasonable person, and I don't care to get into a shouting match with him.<br /><br />You're welcome to use my comments in a future version of the magazine if you wish. You guys have amazing production values and what seems to be a very great setup for the site, distribution, etc. You are ideally placed. However, I just feel like the tone of a lot of the reviews can be overly negative in general for what is, after all, a magazine devoted to indie games -- many of which are spare time projects, zero budget projects, or at other serious disadvantages to AAA games.<br /><br />I have often complained that the mainstream gaming press can be too positive and too caught up in hype for games, so I'm not suggesting you go all rainbows and puppies and give everything a similarly high, meaningless score. I'm also not suggesting that you only rate most games in the 7-10 score range -- some indie games are indeed complete dreck. However, for even the terrible ones that a twelve year old made in his spare time or whatever, there is a nicer way to say that. The person who makes those terrible games might later turn around to make something better, if you don't discourage him out of it first.<br /><br />That part has nothing to do with me or how you reviewed me, since AI War is very far from dreck despite the graphics, but hopefully you see my point. That's what I mean by "nurturing" the industry you are talking about. The caustic put-downs that were prevalent in almost all of the reviews (especially that Blueberry Garden "would have been better as a free flash game" -- that one really bugs me a lot, even though I have yet to play that game) are simply not helping anybody that I can see.<br /><br />I'm very surprised that a large majority of people said they could not get AI War to install. I have not heard that from anyone else, ever, except when the installer version was corrupted at one point (Filefront corrupted an upload and then that got propagated elsewhere). At this point AI War is one of the best-selling non-darling indie games of the year, it's a few months away from providing a fulltime income for myself and a few others, so it doesn't appear that many people are wrestling with the installer overly much.<br /><br />But, point taken that there is a lot of stuff upfront for the game. Unless you are up to date with windows updates, in which case you have the one little 12MB SlimDX install, which does not require a reboot or anything. That's just the curse of the .NET programmer, I guess. It's better than having to install .NET and the giant XNA runtime, though. You're going to start seeing a lot more indie games with that as a requirement, assuming all of them don't just go for the 360.<br /><br />Anyway, your willingness to discuss this is appreciated and unexpected. Thanks again for stopping by.Christopher M. Parkhttps://www.blogger.com/profile/16719365007524426389noreply@blogger.comtag:blogger.com,1999:blog-5439941893980599296.post-40914262028007609292009-08-20T12:22:27.616-04:002009-08-20T12:22:27.616-04:00Chris-
I'm not going to defend Cas or his com...Chris-<br /><br />I'm not going to defend Cas or his comments, but I will pass on your comments if you wish and keep an eye on him.<br /><br />I do agree that our demo review scale should change to a more traditional 10 is perfect, 1 is treadful, 5 is average type thing so it's easier for a casual reader to understand.<br /><br />I definitely hear what you're saying and don't want snap judgements to proliferate reviews. In fact, I would be completely open to including some of what you said/posted in your article in the next issue of the magazine (I'll contact you in the future about that).<br /><br />Really sucks that your installer and the tech you used required all that. A large majority of people could not get the game running (by the time they sent in their reviews and told me, I was up against a deadline). I definitely understand your criticisms of my review and I was somewhat tied to the review system: The game didn't install painlessly and the whole process seemed long for retro style graphics/sounds. I tried to indicate in my review that if strategy style games are not your thing, you should stay away from the frustration of this demo. You're right though, if it was a general 1-10 scale though, your game would've fared better (from me) since games are more than the sum of their parts.<br /><br />Our full reviews also have a lasting appeal category which games such as yours do extremely well in.<br /><br />Thanks for the input. The Demo Reviews are a newer feature and need some adjusting.Mikehttps://www.blogger.com/profile/06764539910300982854noreply@blogger.comtag:blogger.com,1999:blog-5439941893980599296.post-14770314577544500522009-08-20T12:00:33.739-04:002009-08-20T12:00:33.739-04:00Mike,
Thanks for commenting. It's more than ...Mike,<br /><br />Thanks for commenting. It's more than that I take offense at my review score, I think there are a lot of problems with this whole on trial review system.<br /><br />1. Reviewers can deduct random points from your "strict scale."<br /><br />2. Reviewers did not spend much time at all with the game in several cases before dismissing the game out of the hat -- Cas clearly did not run the game more than once before deciding it was a run of the mill game.<br /><br />3. Reviewers are overly caustic in general.<br /><br />4. Reviewers give up immediately and post an N/A when they can't get the game to run for whatever reason.<br /><br />Cas's review was arrogant, largely off topic (half of it is about the installer), and filled with factual inaccuracies (the game is "pretty much the same fare as any other RTS," "the tutorial stages cannot even be skipped," plus a number of other things that are technically worded so that they are a matter of opinion, but which are also in stark, completely opposing contrast to every other review we have ever had -- regarding the interface, the tutorials, and the amount of feedback we get from players/testers). <br /><br />It's irresponsible reviewing, period. I don't care if someone doesn't like the game, not everyone will. But these were hardly reviews at all except in the case of Sam -- there were criticisms that Sam made that I disagree with, but I don't feel like his was an irresponsible review next to the others.<br /><br />I would seriously question the validity of having reviewers making "snap judgments" on games in the first place. Anyone can do that, and people tend to from screenshots or other tidbits of info, anyway. Reviewers are supposed to be providing information and perspective to help readers make informed choices, right? Snap judgements that are called reviews are not helpful.<br /><br />Regarding the install process, <b>this is not something I can control</b>. It has the components it needs to install, if you don't already have them, but that's it. I can't change the fact that I am a C# programmer. I can't change the fact that SlimDX doesn't allow for a silent install. There have been a few grumblings elsewhere about the installer, but there have been other people who have commented on how much they liked it. I personally think it is nonideal, mostly if you are not up to date on your Windows Updates, but as good as is possible given current installer technology available.<br /><br />Thanks for stopping by and posting, but I hope you don't let the fact that I am angry dilute the message of what I am trying to say. You guys should really take a long hard look at what you are doing with the "on trial" section. There's also a big difference between "humorously sarcastic and cynical" (see Out of Eight), and "caustic jerk" (see Cas).Christopher M. Parkhttps://www.blogger.com/profile/16719365007524426389noreply@blogger.comtag:blogger.com,1999:blog-5439941893980599296.post-30791303323645817262009-08-20T11:42:07.612-04:002009-08-20T11:42:07.612-04:00Chris-
First off, I am very sorry that you took s...Chris-<br /><br />First off, I am very sorry that you took such great offense to your score. I would like to note that Demo Reviews are reviewed much harsher than a full review for a few reasons:<br /><br />1. The 10pt Criteria is Strictly Laid Out i.e. if someone thinks your gameplay is fantastic you get only 1 pt. This is rather different than a 10 pt. scale where 3 is significantly below average. Perhaps Cas's scale is too strict and we should adjust to a more normalized 10pt scale.<br /><br />2. They are demo reviews and typically take the perspective of a potential customer who is downloading a free game demo. A lot of times, these customers don't give a game much chance.<br /><br />I think ultimately, AI War Fleet Command was at a huge disadvantage from the beginning and it was probably a poor decision on my part for allowing an intense AI RTS such as yours into the demo review roundup since demo reviews are more snap judgments.<br /><br />I do think that the install process is overly complex and that you could probably grab some more sales by making it simpler and quicker.Mikehttps://www.blogger.com/profile/06764539910300982854noreply@blogger.comtag:blogger.com,1999:blog-5439941893980599296.post-11377756170829645662009-08-19T10:23:28.536-04:002009-08-19T10:23:28.536-04:00Christopher, as a fellow coder and gamer, I have e...Christopher, as a fellow coder and gamer, I have enjoyed reading your articles on the AI design. I have never really played an RTS, so I downloaded yours last night and played the trial to check out and thus far I have been nothing but pleased. I think you did a great job on the game and once I get the $20, I am sure I will purchase a key to unlock the full game.<br /><br />And for the installer, I installed on Windows 7 x64 and had no problems, it only stated I needed the SlimDX distributable. And that downloaded and installed without a problem as well.<br /><br />Anyways, just wanted to say great job and don't let those bloats get you worked up. ha ha.brogdogghttps://www.blogger.com/profile/00241363756866906761noreply@blogger.comtag:blogger.com,1999:blog-5439941893980599296.post-42371741573772113512009-08-19T09:54:34.274-04:002009-08-19T09:54:34.274-04:00Yeah, that or regular magazines/sites, which have ...Yeah, that or regular magazines/sites, which have thankfully been the ones that like the game. Ah well.Christopher M. Parkhttps://www.blogger.com/profile/16719365007524426389noreply@blogger.comtag:blogger.com,1999:blog-5439941893980599296.post-21396969574022316242009-08-19T09:53:33.246-04:002009-08-19T09:53:33.246-04:00Luckily for you only Indie game developers read th...Luckily for you only Indie game developers read that crap. True gamers are too busy playing, and they get their updates while at work searching their favorite forums.Ryan P Gileshttps://www.blogger.com/profile/17676390442718760077noreply@blogger.comtag:blogger.com,1999:blog-5439941893980599296.post-80533744653949815262009-08-18T10:37:34.884-04:002009-08-18T10:37:34.884-04:00I agree completely. I had not seen how they pick ...I agree completely. I had not seen how they pick their reviewers, man that is pretty crazy. What an unprofessional setup in general they have there. It's a real shame their site gives off such an air of professionalism until you look into it more.Christopher M. Parkhttps://www.blogger.com/profile/16719365007524426389noreply@blogger.comtag:blogger.com,1999:blog-5439941893980599296.post-11665512487302667362009-08-18T10:23:53.254-04:002009-08-18T10:23:53.254-04:00It looks like IGM grabs people off the street to w...It looks like IGM <a href="http://www.indiegamemag.com/forums/topic.php?id=19#post-39" rel="nofollow">grabs people off the street to write for them</a>? And there <a href="http://www.indiegamemag.com/guides/AGuidetoWriting.pdf" rel="nofollow">"Guide to Writing"</a> is pretty bland and indicative of what to expect from their reviews. No where does it talk about how indie shops should be treated a bit differently than EA or Blizzard or Valve.<br /><br />IGM is just proving anyone with a Wordpress site can voice an opinion in today's society. Don't feed their trolls! :)Greghttps://www.blogger.com/profile/16270865687621486990noreply@blogger.comtag:blogger.com,1999:blog-5439941893980599296.post-86818882694626401012009-08-18T09:30:06.737-04:002009-08-18T09:30:06.737-04:00It seems that game reviewers have (in some cases) ...It seems that game reviewers have (in some cases) an air of self-importance about them. <br /><br />The "n/a" score due to installer failures or other complications further illustrates this point. I think a better way for IGF to handle it would be "would a reasonably intelligent PC user be able to troubleshoot and get it working without too much trouble?"<br /><br />To be quite honest, I can't think of ANY PC game that has ever been 100% smooth to install, upgrade, patch, and play with absolutely no bugs or issues.<br /><br />Examples: I installed the Demigod demo 2 days ago. I couldn't even get it online due to some network code bug. I spent an hour scouring forums and troubleshooting to try and get it working. In the end I couldn't. (maybe demigod should get an n/a?) <br /><br />Of course a AAA title will never get a score of N/A though because it is too difficult for reviewers to bash on their primary stream of revenue. (ads from AAA publishers).sox3502ushttps://www.blogger.com/profile/17708079776110396997noreply@blogger.comtag:blogger.com,1999:blog-5439941893980599296.post-63867682853982222602009-08-17T19:06:36.016-04:002009-08-17T19:06:36.016-04:00There is a huge amount of further discussion about...There is a huge amount of further discussion about this in the Arcen Games forum <a href="http://arcengames.com/forums/index.php/topic,769.0.html" rel="nofollow">here</a>Christopher M. Parkhttps://www.blogger.com/profile/16719365007524426389noreply@blogger.com