Making a comparative analysis

Unless you are inventing a totally new genre, when you're designing a feature, chances are that someone has already done something that at least remotely looks like what you're trying to achieve. Incidentally, if you are making a game of a crowded genre, it is wise to look at what the competition has done, in order to not make the same mistakes and/or nick the interesting ideas. That's why you usually do comparative analysis (that's the part where you can say that your job as a designer consists in "playing" video games).

We all have our references when it comes to game mechanics, depending on our experience as a player. However, bad designers (or inexperienced ones) will often cling to one or two games and say things like "I want a COD-style aiming system". Video game culture is an important tool for a designer when it comes to finding references (that's why it's important to play all sorts of games).

Now, when you're doing that kind of research, you are certainly not comparing games as a whole. When you do a comparative analysis, you should do it on a per-feature basis. Focusing on one thing makes it easier to notice when you test the games. As features can be broken down into several elements, you also need to define what are the distinct elements you are doing your research on (obviously, these would be the elements you want implemented in your game). Once again, it's easier to find stuff when you know what you are looking for.

Failing to do that might result in you losing focus and ending up playing the games rather than testing them (I've been there).

You also need to choose which games to test. Obviously, these would be the ones that contain the feature you are doing your research on (even if those games seem quite different from yours, as long as the feature is present).

Now we know what to look for and where. We can lay out a simple spreadsheet in Which we're going to put the notes taken from the test session.

Example of test sheet for combat AI in games with brawler-type combat (I've replaced the actual notes). The number next to the game's name is the number of platforms it has been released on.

As you can see, there's some additional information we can put on that spreadsheet which can be used at the time of analysis: the game's sales (from VGChartz, for instance) and its Metacritic score. 

Now don't get me wrong, I'm not saying these figures should be considered as gospel (especially the VGChartz ones, which don't take digital distribution into account) BUT it's a rough indication that can give some insight on your research.

For instance if you find a game that is critically and commercially successful but seem to approach the feature in an opposite way to what you had planned for your game, it might be wise to ask yourself questions. That doesn't mean you should copy that game's approach, but keep it in mind if your way doesn't work.

A side note on "taking inspiration" from other titles. I know many people would see that as idea stealing, or some would be annoyed to hear people "hey, they've done the same as X". My opinion on the matter is that I don't see what's wrong with reusing good ideas. I'd even say that rejecting a proven solution to a problem because someone else has done it is stupid. But then, if you really don't want to be compared to a given game because of that, it's fair enough.

Now you're ready for your testing session. Nothing special here. Play the relevant sections of the games, observe the elements you've listed, take notes.

Once the testing session is over, it's time to take a look at your notes. I can't really tell you what to do here, it's entirely up to you. You've done this research to know if your design is going in the right direction. It might validate or invalidate your ideas. It might even spark new ideas. What matters in the end is that you know why you do or don't do the same as the competition.

I'll conclude with this: if one of your game's features has never been done before, keep in mind that it might be for a good reason... ;)