I have been complaining for a while that some of our test suites are out of date, but as the days pass, they never seem to actually reach the point where we can switch over. Well, where our GPU test suite is concerned, at least, much time has been dedicated to it over the course of the past week, and I believe we're quite close to purging all of our currently-used games in favor of much better, and newer, ones.
The major problem with most of the games we've been testing with for the past while is the fact that they simply don't push current graphics hardware enough. The exception is Crysis Warhead, but let's be real here... that's probably due more to an inefficient game engine than anything, given today's games have far, far better graphics and tend to run better (I'd be willing to say even the console-port Modern Warfare 2 looks better).
In the search for replacement games, it became clear that the market is chock-full of titles that look good, or don't, and run at 100 FPS+ at 1920x1200 on a decent card. Finding a game that isn't just good for benchmarking now, but for the next ~6 - 8 months was a little tough, but I do think we have picked six good titles to head up that task, while still managing to keep an even spread across multiple genres.
Aliens vs. Predator
I admit... this game is still up in the air, and I might drop it and just stick to testing with five games for the time-being until something better comes along. I first gave this game a good test for our Eyefinity article, and quickly found myself bored. I mean real bored. Testing it again this past week reaffirmed by belief that yes, this is indeed a boring game, so I'm stuck.
The game might be boring (to me), but it does happen to have good graphics and the capabilities to push our entire range of graphics cards just fine, so it makes for a good benchmark.
So the question remains... should we benchmark with a game we don't even like, just because it's good for benchmarking? Or is it still a useful enough metric to include, so it should be included?
Blur
For each GPU suite revision, I like to choose at least one racing title to benchmark with, and this time around, I'd say that would have to be Blur. The graphics are not quite on par with, say, Metro 2033, but as far as racers go, they're quite good, and the entire game packs in a ton of eye candy. Plus, as you could tell by my review, I totally dig it, and would rather be playing it right now than writing this post!
Just Cause 2
I've tested this game a fair bit in the past, but due to time, I never made it past the first level. With Blur, I fell into a trap where I thought I'd use it just for benchmarking, but ended up actually playing it because it was so much fun. The same can be said for Just Cause 2... so far, the game is FUN. I'm only a few missions in, but all I can say is... check this one out if you can.
The game isn't just fun, though, it has stellar graphics, too. Like Blur, it has a ton of eye candy, but not in the flashy "ooh pretty colors" kind of way. The game takes place on what appears to be a massive array of islands you can traverse across, so it's quite realistic in that regard.
Metro 2033
The inclusion of this game should be a surprise to no one. Like Crysis, Metro 2033 pushes the boundaries of PC graphics and runs horrible at max detail at high resolutions. That's just how we like it. It's just unfortunate that so much of the game looks the exact same... bleh.
Napoleon: Total War
For our lone RTS game, there was just one choice... Napoleon: Total War. The game, released this past February, sets a benchmark for what a system-intensive RTS game can be. The character models themselves are not the most impressive out there, but when there's hundreds of them on a battlefield, does it even matter? The game isn't just good for GPU benchmarking, but likely CPU benchmarking as well, so it's one we might include in our CPU suite revision as well.
Tom Clancy's Splinter Cell: Conviction
It's rare for us to include a third-person adventure game in our suite, but Conviction offers enough to break that rarity. The game has great graphics, scales well, and most importantly, superb gameplay.
In addition to these games, we're going to continue using 3DMark Vantage, and also introduce Unigine's Heaven Benchmark 2.0, since it seems to be gaining some traction, at least where DX 11 is concerned.
We're also looking at the possibility of including a GPGPU-type benchmark, but that's something unlikely to actually happen, despite how nice it would be to include. As far as I'm aware, there isn't a GPGPU test that's fair to both AMD and NVIDIA, and even if there was, it still might not give us an accurate interpretation to which card is better (for example, a password cracker might use the GPU in a much different manner than a video clean-up tool).
As always, we welcome critique and other ideas to help make our test suite as good as can be. So, please don't be afraid to speak up. Alongside our suite overhaul, we're also going to upgrade our GPU testing machine just a bit. We're still going to stick with Gigabyte's EX58-EXTREME motherboard, as it's still sufficient for our needs (and supports tri-GPU for both AMD and NVIDIA), but we're going to boost our RAM from 6GB to 12GB.
I was contemplating upgrading that machine to the Core i7-980X, but at this point in time, that seems like overkill given none of the common games out there can even use a quad-core to its full potential. The Core i7-975 is still a great fit for our needs here, though we might overclock it to 3.60GHz for fun. I'm also going to toss a Thermaltake FRIO CPU cooler in there as well. There's nothing wrong with Thermalright's Ultra-120, but I've been looking at the same CPU cooler for far too long and need an aesthetics change! Bill's review of the FRIO put the cooler in a great light, so I can't wait to get it installed.
Thanks guys!
The major problem with most of the games we've been testing with for the past while is the fact that they simply don't push current graphics hardware enough. The exception is Crysis Warhead, but let's be real here... that's probably due more to an inefficient game engine than anything, given today's games have far, far better graphics and tend to run better (I'd be willing to say even the console-port Modern Warfare 2 looks better).
In the search for replacement games, it became clear that the market is chock-full of titles that look good, or don't, and run at 100 FPS+ at 1920x1200 on a decent card. Finding a game that isn't just good for benchmarking now, but for the next ~6 - 8 months was a little tough, but I do think we have picked six good titles to head up that task, while still managing to keep an even spread across multiple genres.
Aliens vs. Predator
I admit... this game is still up in the air, and I might drop it and just stick to testing with five games for the time-being until something better comes along. I first gave this game a good test for our Eyefinity article, and quickly found myself bored. I mean real bored. Testing it again this past week reaffirmed by belief that yes, this is indeed a boring game, so I'm stuck.
The game might be boring (to me), but it does happen to have good graphics and the capabilities to push our entire range of graphics cards just fine, so it makes for a good benchmark.
So the question remains... should we benchmark with a game we don't even like, just because it's good for benchmarking? Or is it still a useful enough metric to include, so it should be included?
Blur
For each GPU suite revision, I like to choose at least one racing title to benchmark with, and this time around, I'd say that would have to be Blur. The graphics are not quite on par with, say, Metro 2033, but as far as racers go, they're quite good, and the entire game packs in a ton of eye candy. Plus, as you could tell by my review, I totally dig it, and would rather be playing it right now than writing this post!
Just Cause 2
I've tested this game a fair bit in the past, but due to time, I never made it past the first level. With Blur, I fell into a trap where I thought I'd use it just for benchmarking, but ended up actually playing it because it was so much fun. The same can be said for Just Cause 2... so far, the game is FUN. I'm only a few missions in, but all I can say is... check this one out if you can.
The game isn't just fun, though, it has stellar graphics, too. Like Blur, it has a ton of eye candy, but not in the flashy "ooh pretty colors" kind of way. The game takes place on what appears to be a massive array of islands you can traverse across, so it's quite realistic in that regard.
Metro 2033
The inclusion of this game should be a surprise to no one. Like Crysis, Metro 2033 pushes the boundaries of PC graphics and runs horrible at max detail at high resolutions. That's just how we like it. It's just unfortunate that so much of the game looks the exact same... bleh.
Napoleon: Total War
For our lone RTS game, there was just one choice... Napoleon: Total War. The game, released this past February, sets a benchmark for what a system-intensive RTS game can be. The character models themselves are not the most impressive out there, but when there's hundreds of them on a battlefield, does it even matter? The game isn't just good for GPU benchmarking, but likely CPU benchmarking as well, so it's one we might include in our CPU suite revision as well.
Tom Clancy's Splinter Cell: Conviction
It's rare for us to include a third-person adventure game in our suite, but Conviction offers enough to break that rarity. The game has great graphics, scales well, and most importantly, superb gameplay.
In addition to these games, we're going to continue using 3DMark Vantage, and also introduce Unigine's Heaven Benchmark 2.0, since it seems to be gaining some traction, at least where DX 11 is concerned.
We're also looking at the possibility of including a GPGPU-type benchmark, but that's something unlikely to actually happen, despite how nice it would be to include. As far as I'm aware, there isn't a GPGPU test that's fair to both AMD and NVIDIA, and even if there was, it still might not give us an accurate interpretation to which card is better (for example, a password cracker might use the GPU in a much different manner than a video clean-up tool).
As always, we welcome critique and other ideas to help make our test suite as good as can be. So, please don't be afraid to speak up. Alongside our suite overhaul, we're also going to upgrade our GPU testing machine just a bit. We're still going to stick with Gigabyte's EX58-EXTREME motherboard, as it's still sufficient for our needs (and supports tri-GPU for both AMD and NVIDIA), but we're going to boost our RAM from 6GB to 12GB.
I was contemplating upgrading that machine to the Core i7-980X, but at this point in time, that seems like overkill given none of the common games out there can even use a quad-core to its full potential. The Core i7-975 is still a great fit for our needs here, though we might overclock it to 3.60GHz for fun. I'm also going to toss a Thermaltake FRIO CPU cooler in there as well. There's nothing wrong with Thermalright's Ultra-120, but I've been looking at the same CPU cooler for far too long and need an aesthetics change! Bill's review of the FRIO put the cooler in a great light, so I can't wait to get it installed.
Thanks guys!