July 9, 2010
A long, long time ago on a website far, far away there appeared a post which argued that feral druid bear tanks were in short supply. The article also made the perfectly reasonable point that armoury datamining sites had no data on the popularity of the various druid forms.
Now, being the kind of nerd who likes a challenge (is there any other kind?) I couldn’t let that one go past. But how to solve the problem? Druids get some of their forms through talents; easy enough to get a count of the toons invested in those talents. But cats and bears were not so straightforward.
As an old SQL hacker of the in Codd we trust school, my first cunning plan to get the numbers on feral druid bear tanks basically went like this:
- Mine data
- SELECT level 80 feral druids FROM toons
- GROUP BY ???
Unfortunately, that didn’t work so well, for a number of reasons. I now understand one very interesting reason why it didn’t work: the feral talents that we expected to use to identify cats and bears are not actually distributed that way. But more on that later.
The solution involved spending some time getting up to speed with more sophisticated datamining algorithms. These algorithms are also based on a sort of GROUP BY principle, but are capable of grouping (or “clustering”, as the datamining jargon has it) across multiple data dimensions. They can easily handle the 85 talents in the three druid trees and, in essence, can group samples of druids into clusters of toons in an 85-dimensional space. Alternatively, we can cluster on character stats – health, mana, strength, agi etc – or on any combination of talents, stats and playstyle numbers that interest us.
They also use calculus techniques to find the borders of each cluster in a way that can tolerate outliers. This is important in real life data, but also important in WoW data where there is always a small but significant number of players who insist on being… individuals…
Up until a few days ago, I was expecting to have to put off working on the druid forms question until I had a really good understanding of these algorithms. But that is not necessary, for the simple reason that the data we are dealing with is not all that complex. Consider the following graph:
Here we have selected for toons that have a history of running instances and raid dungeons, and have filtered out the toons that are serious PvPers. What we have are two groups that in essence are clustering themselves – no real datamining required.
The bottom, horizontal, group is emphasizing health over mana. This group is stacking stamina, agility, armour and dodge (I’ll prove all those things in the next post) and selecting some of the talents we’d expect for bears. In other words, Tanking 101. The top group is selecting for mana over health and is stacking intellect and spirit. This group has taken some of the cat talents too, although as we’ll see in the next post, talents do not seem to be a great predictor of role.
That graph was generated from a 500-toon data set. So we’re ready to cluster and count the full sample. And voilà:
You can see there are some outliers, but the bulk of the sample falls neatly into the two clusters. What you can’t see properly from the chart is that the blue tank cluster is in fact more populous than the red DPSers. It’s just that their health and mana stats don’t vary much, so the cluster is more dense. That’s where a clustering algorithm is needed to get a count of the population of each blob.
And the answer? There are 13, 187 level 80 feral druids in the sample who have done more than 75 instances and raids and have done no arenas. That’s my (s0mewhat generous) working definition of a PvE raider. It’s also a problematic definition because the arena stats are historical – they don’t prove that the toon was not geared up for raiding when the armoury snapshot was taken.
Of those raiders, 60% are in the blue tank cluster and 40% are in the red DPS cluster. So that’s one useful piece of information: feral druid raiders do seem to prefer to tank rather than DPS by a narrow majority.
But the PvE raiders are less than 1/2 of the total sample. So, the worst case scenario is that on any given day, only 30% of level 80 feral druids are set up for tanking (although, to repeat, it is not likely that every arena player is geared for PvP all the time).
The data is from patch 3.3.3.
Then there is the question of effective tanks. Some of those blue crosses in the bottom left hand corner of the chart are probably not seriously gearing up for very much at all. That will be the subject of the next post, when we will use some of the wonderful data visualization tools in these datamining packages to look much more closely into the dark heart of that big blue blob.
Meanwhile, if you’d like to play around with the data for yourself, here are the data sets I’m using. I’ve got a small set of feral talent builds, a larger set of builds and a large set of character stats. Each data set contains counts of instances and raids, battlegrounds and arenas played so you can filter on the raiders.
NB these data sets have been corrected and updated on 12 July. If you downloaded them before that, apologies for the error, and please download them again:
July 5, 2010
Things may look quiet here but behind the scenes I’ve been working on my project to get up to speed with modern datamining algorithms. The first step has been to assemble some sources of information and some tools for the job.
For a textbook, I’ve chosen Introduction To Data Mining by Tan, Steinbach and Kumar. It provides a good overview of the key algorithms, along with important issues like data quality and consistency. It also introduces the maths in a reasonably gentle way.
Fortunately, while it is important to understand how the algorithms work, it is not necessary to work the maths by hand. There are some first class freeware datamining programs available that do all the heavy lifting, so long as you know how to prepare the data and how to set the parameters of the algorithms so they produce valid results.
Three datamining packages in particular are worth noting:
Weka and RapidMiner are GUI-driven toolsets where R is more command-line oriented. You can download all of these and play around with them at home. They’re not toys, so you need to have some confidence about plowing through the user guides and technical manuals, but they are easy enough to get up and running.
The choice between Weka and RapidMiner is a difficult one. At the moment I’m working with Weka but that is mainly because it was the first one I started experimenting with.
The other crucial thing to have at hand is some test data. Of course you may want to remind me that I have several GB of WoW-related data right here. But that is not the place to start. The first step is to learn how the tools work and how to use them. For that we need data where we know the expected results – so that when the tool doesn’t produce the right result we know to look again at how we have applied the algorithm.
The data has to be challenging enough to put the algorithms to a test, but not so challenging that we are left wondering whether the wrong answer is due to the complexity of the data or just due to some dumb mistake.
Over at the Expressive Intelligence Studio blog, Chris Lewis posted an interesting report about using a toon’s gear to predict its class. That’s exactly the sort of place we want to start since all sorts of classifying and clustering algorithms could be tested on a data set like that. The other idea that occurred to me is to use talent builds to predict the spec of the toon. Of course you can do that in a very simple way by just adding up the points spent in each of the three trees: a paladin that has most points in the holy tree is a holy paladin.
Where the problem becomes interesting is with those classes where there is a tendency to spread talent points across more than one tree. I’m thinking mainly of mages and warlocks but any class where the three trees don’t map straight onto the tank/healz/dps holy trinity should see some points spread across multiple trees. Can datamining algorithms handle “fuzzy” data like that?
To make this discussion more concrete, let’s have a quick look at that very question. We can fire up Weka and feed in a sample of level 80 paladin talent builds. To keep it simple, I’m using a toy data set of only 150 paladins with 50 from each of the 3 trees. We can run a basic k-means clustering algorithm over the data, which we hope should produce 3 clusters: one each for the holy, protection and retribution trees. And voilà…
That works because holy paladins don’t spend many points in protection or retribution talents. But for mages, where there is a significant tendency to spend points in more than one tree we get this:
Now the algorithm is flummoxed – putting arcane and frost mages in the same cluster and splitting fire mages into two clusters. So we have a simple data set that is also challenging enough to put these tools to a bit of a test.
I’ve also made a third data set using priest builds. Priests have more talent points invested across the trees than paladins but fewer than mages. Clustering this data set is left as an exercise for the reader…
No, seriously… Anybody who’d like to experiment with these data sets can download them from the links here. They’re in a standard .arff format (really just an annotated csv file) that Weka and RapidMiner both know how to load. Note that I’ve used a “.pdf” extension since WordPress will not allow me to upload arbitrary file types. But if you open them in a text editor you’ll see they are just csv data. Rename the extension to “.arff” and they’re ready to go.
I’ll have a lot more to say about these little data sets in the next few posts.
May 17, 2010
So, the second week of my holidays and another pleasant distraction. This time I’m working on joining the smartphone revolution. I’m still carrying an old Nokia 5-series, since none of the 3G phones have really done much for me. The basic problem is screen realestate – Google maps or GPS navigation on a postage-stamp sized display never seemed a lot of use because you couldn’t see enough context with your location. And the killer app for me is mobile web browsing; not being a chatty teenager I don’t actually make that many outgoing calls.
Plus I’m becoming increasingly annoyed about gadget overload. A phone, a camera, a GPS, an MP3 player… They’re all just microcomputers with displays and sensors. Where is Sauron to forge the one gadget to rule them all?
Well in fact he’s in Cupertino, California. And so you may well ask “but Zardoz, why not just buy an iProduct?” This is why not. Apple have produced the goods, sure, and they have the first-mover advantage, but we’ve seen this movie before. The original Mac was the be-all-and-end-all, but most people went with DOS and the odd how-to book from the local techie bookshop and got by just fine.
And inspiration has struck. I’ve decided on this one – the HTC Desire.
There really should be some sort of consumer protection law that stops companies naming sleek and sexy gadgets desire. Or, just call it lust and be done with it. Just writing this post is making me think of taking a cold shower.
However, as is the way with these things, the path to the fulfilment of my desire is narrow and winding. From another point of view, we know lust is one of the seven deadly sins. And sins do seem to travel in packs.
In this case, the other one in the mix is avarice.
The Desire is on sale in Australia, but through Telstra, Australia’s national “phone company”. Although it still majority owned by the guv’mint has the government as a shareholder, Telstra is the sort of rapacious dollar-per-bit carrier that my US readers probably know only too well. It makes ol’ Ma Bell look like your favourite grandmother. No thanks.
But there is another solution, which I discovered thanks to Whirlpool, that wonderful online resource for Australian internet and gadget lovers. Order the phone from the UK and ship it out, FedEx, for £30 and a seven-day-more-or-less-delivery (barring the odd Icelandic volcano). The British phones are on the correct UMTS frequencies for the Australian networks that compete with Telstra. And over a hundred Oz dollars cheaper. The smart money seems to be ordering them from here or here. I think I’ll be joining them in a few days.
Hopefully I’ll be able to make my first ever mobile blog post in the next couple of weeks. Oh I love living in the future!
May 12, 2010
My Google Appengine site is now updated for patch 3.3.3. I’m also on holidays at the moment. Thanks to some historic victories of the working class (no, seriously…) we get four weeks annual leave in Oz. Plus I’m taking an extra week for good luck.
After doing some intense lazing about, I should be ready for a bit of intellectual stimulation. I might finally be able to make some progress on my datamining project. That’s the plan anyway. We’ll see how it goes.
April 19, 2010
Just a brief note to say I’m updating my database for patch 3.3.3 now. I should have all my reports refreshed by the end of the week. Not much progress on anything else but; distractions are coming from every direction at the moment.
April 10, 2010
A commenter by the name of Tankadin asked me to look at some problems with my Protection Paladin stats page. First off, apologies for the time it has taken to respond to this; I’m always happy to check for errors.
Tankadin asked whether 1.2 was a reasonable minimum mainhand attack speed for prot pallies. That is what the data shows. Level 80 prot pallies come equipped with a variety of weapons in the mainhand slot. In my sample we have this for mainhand speed:
The other question was: why do they show any offhand dps, when they should be equipped with shields. The answer to this one is a bit more disturbing. The stats shown in the armoury are all calculated on the server side (presumably to prevent people reverse-engineering the equations involved) and there is no direct connection in the XML between the stats and the items that the character is equipped with.
No doubt, somewhere in the bowels of Blizz’s servers, the stats are being computed by taking into account the bonuses granted by each item, gem, enchant et cetera, but when the data gets to your browser it is just a set of disconnected numbers.
What is happening is that a paladin who has nothing equipped in the offhand slot gets reported as having a non-zero offhand dps. It is a purely theoretical number since there is no weapon in the slot that could attack anybody using that dps. But the number is there in the XML.
So, regrettably, that table is… misunderestimated… There may be other examples of this in some of the stats reports and I’ll try to set aside some time to see if some better sanity checks can be put in place. Unfortunately there are limits to what can be done since I use a lot of pretty generic SQL to do the work.
Tankadin’s final questions were about whether the stats were being influenced by outliers. Well, outliers there certainly are. I found one one level 80 prot pally wielding this in the mainhand slot. If you’re ever tempted to equip it for the next guild raid, you’ll be pleased to know that it has an attack speed of 2.0!
The reason why that doesn’t bother me too much is that the outliers form a relatively small proportion of the sample set. Most characters adopt one or another of a very small subset of items. We might call them the “popular” group. We’ve remarked on this interesting distribution before, but its worth looking at a chart again. Here is a plot of the number of level 80 protection paladins equipped with each distinct item in the mainhand slot:
You can see the “long tail” of outliers which stretches to the right hand axis of the chart, but the reality is that there are about 1800 toons in the long tail, compared to 22,000 toons in the popular camp.