twilight zone

July 9, 2009

Anybody out there happen to know why the armoury is currently returning talent strings 100 characters long for Druids? They still have only 85 talents AFAIK.

I can see that the string is just right-padded with zeros so it’s no biggie. But hey, we’re trying to maintain a quality service here. Six sigmas and all that stuff.

pet sounds

July 3, 2009

Here’s news for Hunters:  Blizz has added your pets and their talents to your character talent page in the armoury.

That will give us a lot of new insight into the types of creatures that make the most popular pets, and how players spend pet talent points. The XML is straightforward and contains no quirks and so I’m cranking up the SQL editor as we speak.

I’ll try and have the new reports ready before the next refresh of my data. As to when that will be, well, about a week after Patch 3.2 drops. As to when that will be… well…

blog status

June 24, 2009

Things may look quiet here but that’s just because I’ve been busy setting up my Google App Engine site. I make no claim to being a level 80 elite Tauren web designer, so the process has been a bit less smooth than I might have liked, but it’s nearly all done.

If you have a few spare moments, you could hop over there and see if you can find anything wrong with the Death Knight reports. That’s the first dropdown on the main menu bar, where all the submenu links should now be working.  Don’t bother with any other class – DKs are the only class where all the reports are up, for now.

But all the machinery is in place to generate reports for the other classes. Just a matter of cranking the handle.

Incidentally, if you’re ever faced with the problem of creating HTML and CSS using only free tools, let me sing the praises of Firebug as a layout debugger. In particular, its ability to debug the CSS cascade is invaluable. When your H3 tags, which should be 2.7 ems and dark grey, are all coming out as 1.6 ems and a funky sky blue, who you gonna call? Well, somebody who can analyse the cascade of style rules, so you can see which rule is clobbering the rule you think should apply. Firebug does this perfectly. And you can edit the rules and the HTML on-the-fly in the Firebug window and immediately see the change rendered in the browser – invaluable for those final tweaks to the layout.

And just to get it all out of my system, let me say I HATE CSS! Phew, that’s better. Unfortunately there’s no doubt that it is the Right Thing to use, but couldn’t somebody have made it a bit less counterintuitive? And as software engineers know, all code evolves towards the big ball of mud. But nothing gets there faster than a CSS stylesheet in the hands of an enthusiastic amateur.

freaky Wednesday

June 18, 2009

I had an email exchange yesterday with the developer of the WowPopular site. This was Talent Chic but has now become a comprehensive armoury site that presents a different view of the data to what you get here. He’s certainly been more diligent than me about partitioning the data into PvE/PvP categories – something that is still languishing on my to-do list.

Anyway, it turns out that not only are we both in Australia, but we’re in the same Oz city – Melbourne. Now that is freaky!

Calculating the probability that any two armoury dataminers would come from basically the same spot on the planet is left as an exercise for the reader.

But it did occur to me that with odds like that, we’re wasting our time on something as trivial as MMOs. We should be out developing the infinite improbability drive instead. I think we’d have no trouble getting it to work.

I see I’ve got a lot of visitors over the last day or so from here. The author of that post has come up with a simple but interesting datamining idea – producing a consolidated list of the most popular gems to aid people playing the AH.

Unfortunately it isn’t really correct to add the percentage values in my tables since they are relative measures designed to show what is popular by spec. But the population in each class/spec combination varies wildly which make it impossible to go from that to a general count.

However, your humble dataminer stands ready to help with questions like this. I ran a query on gems across the overall sample and came up with the following list. The numbers are the total count of each gem and for comparison’s sake there are 171,148 characters in this sample who have at least one gem. I’ve chopped the table off at 5000, but the list goes all the way down to various cuts of shattered and opaque Dark Jade which are worn by one toon only in the sample.

Be aware that this is not a random sample – it is biased with a lot of higher level twinks, who could be expected to be gemmed up to the gills. But since twinking players are ones most happy to splash the cash, maybe that is no bad thing.

Runed Scarlet Ruby 67720
Solid Sky Sapphire 50331
Bold Scarlet Ruby 39622
Rigid Autumn’s Glow 27453
Balanced Twilight Opal 26209
Delicate Scarlet Ruby 20147
Smooth Autumn’s Glow 18689
Chaotic Skyflare Diamond 17878
Bright Scarlet Ruby 16895
Brilliant Autumn’s Glow 14784
Runed Bloodstone 13025
Potent Monarch Topaz 11621
Bold Bloodstone 11099
Sovereign Twilight Opal 10407
Pristine Monarch Topaz 10371
Luminous Monarch Topaz 10257
Sparkling Sky Sapphire 9610
Glowing Twilight Opal 9414
Purified Twilight Opal 8304
Relentless Earthsiege Diamond 8057
Enduring Forest Emerald 8049
Veiled Monarch Topaz 7508
Deadly Monarch Topaz 7491
Vivid Forest Emerald 7483
Runed Dragon’s Eye 7376
Royal Twilight Opal 7337
Thick Autumn’s Glow 7069
Austere Earthsiege Diamond 7066
Solid Chalcedony 6857
Ember Skyflare Diamond 6604
Dazzling Forest Emerald 6482
Glowing Nightseye 6344
Bold Dragon’s Eye 6100
Steady Forest Emerald 6072
Smooth Sun Crystal 5793
Inscribed Monarch Topaz 5724
Jagged Forest Emerald 5701
Perfect Runed Bloodstone 5658
Regal Twilight Opal 5332
Reckless Monarch Topaz 5136
Kharmaa’s Grace 5116

Let me say I’m right with you on questions about the WoW economy. The AH is one of the more inspired minigames in WoW. It’s a sort of non-zero-sum PvP where skill actually counts for something and you can’t twink your way to stardom. Even with Auctioneer, you still have to think about what the data is telling you. So, I’ve added consolidated reports to my to-do list. Anybody with other suggestions for reports that have an economic focus is welcome to leave a comment.

Over at Wowenomics, Gevlon from the Greedy Goblin left a comment about datamining that is worth replying to. Basically, his point is that sites like this one report on what the average player is doing, but that is not much use because the average player is only making average choices. (…or at least that’s the polite paraphrase of his point.)

In fact the data shows that this sort of argument is not true. Let’s start from what the data actually looks like. Here, I’ve taken an example picked at random: all choices made by 69 DKs for the chest slot:

Item Count
Mightstone Breastplate 2421
Battlemaster’s Breastplate 1010
Scavenged Tirasian Plate 996
Adamantite Breastplate 898
Murkblood Avenger’s Chestplate 655
Gorge’s Breastplate of Bloodrage 309
Battle Leader’s Breastplate 208
Saronite War Plate 197
Fel Iron Breastplate 141
Unscarred Breastplate 136
Westguard Armor 78
Coldrock Breastplate 75
Azure Chain Hauberk 71
Durotan’s Battle Harness 70
Baleheim Armor 68
Light-Touched Breastplate 63
Breastplate of the Warbringer 48
Andrethan’s Masterwork 46
Bone-Threaded Harness 44
Segmented Breastplate 32
Conqueror’s Breastplate 30
Blacksoul Protector’s Hauberk 26
Bloodfist Breastplate 23
Chestguard of Illumination 21
Nether Protector’s Chest 21
Vest of Vengeance 21
Soul Saver’s Chest Plate 21
Scavenged Breastplate 18
Chestguard of Salved Wounds 18
Breastplate of Blade Turning 18
Light-Bound Chestguard 17
Warmaul Breastplate 17
Breastplate of Retribution 15
Blackened Chestplate 15
Boulderfist Armor 14
Heavy Earthforged Breastplate 14
Shattered Hand Breastplate 12
Talonguard Armor 12
Reaver Armor 11
The Exarch’s Protector 9
Lost Chestplate of the Reverent 8
Bloodscale Breastplate 8
Gilded Crimson Chestplate 7
Chestplate of A’dal 7
Jerkin of the Untamed Spirit 7
Khan’aish Breastplate 7
Redeemer’s Plate 7
Torn-heart Family Tunic 7
Protectorate Breastplate 6
Marshwalker Chestpiece 4
Bogslayer Breastplate 4
Demon-Forged Chestguard 4
Warden’s Hauberk 4
Shamblehide Chestguard 3
Garmaul Chestpiece 3
Elegant Dress 2
Crimson Mail Hauberk 2
Cenarion Thicket Jerkin 2
Ango’rosh Breastplate 2
Azure Silk Vest 1
Acherus Knight’s Tunic 1
Black Mageweave Vest 1
Bonechewer Berserker’s Vest 1
Corsair’s Overshirt 1
Darkcrest Breastplate 1
Demon-Forged Hauberk 1
Drakescale Breastplate 1
Breastplate of Many Graces 1
Chestguard of the Dark Stalker 1
Chestguard of the Stormspire 1
Chestguard of the Talon 1
Farshire Robe 1
Flimsy Chain Vest 1
Lovely Black Dress 1
Lovely Blue Dress 1
Simple Black Dress 1
Skom Chain Vest 1
Scale Brand Breastplate 1
Refuge Armor 1
Runecloth Robe 1
Nexus-Strider Breastplate 1
Spring Robes 1
Tuxedo Jacket 1
Twilight Cultist Robe 1
Warrior’s Embrace 1
Worgblood Berserker’s Hauberk 1
Wrathfin Armor 1

It’s worth charting this distribution too, since the shape of the distribution curve is important:

69 DK chest item choice distribution.

69 DK chest items - distribution.

Log plot 69 DK chest item choice.

69 DK item distribution - log scale

The error in Gevlon’s argument stems from our common-sense understanding of average. Most often we think in terms of a Gaussian distribution – so often that it is actually called a normal distribution. When events or things are distributed normally, then the average outcome is, well, average. But, as various people have observed, when networking effects come into the picture, the typical distribution is not Gaussian but power-law-like. The majority cluster around a very few choices, with a rapid fall-off into a long tail of more funky choices but where each choice in the long tail made by only a few individuals.

Now WTF does all that mean in plain English? Simple. If WoW gear, gems or enchant choices were normally distributed, a few people would make the best choice,  a few people would make the worst choice and most would make a so-so choice. We would expect to see a few 69 DKs with the Uber Breastplate of Pwnage, a few with the Scruffy Tunic of Suckage, but most would be wearing the Mediocre Breastplate of, um…, Mediocrity. And that’s what my report would find for you.

But you can see from the charts that the data looks nothing like a normal distribution. Most players have in fact made the same few choices – which generally represent a trade-off between how powerful the item is and how easy it is to get hold of.  Those people who don’t follow the crowd are out in the long tail – here the picture is murky because we don’t know whether they are there because of ignorance or whether they have hit on some effective but as-yet unknown (or difficult to obtain) solution to the problem. (And some are out there because the data is capturing multiple playstyles – no doubt those toons wearing tuxedo jackets and lovely blue dresses are being played by people who know exactly what they are doing.)

But for our purposes, averages are just what we want – they show the consenus view across the player base on what are the reasonably sensible and effective choices.

To me, the interesting question is how this consensus forms. Undoubtedly Gevlon’s point has an element of truth – the average WoW player is no theorycrafter. But there are feedback mechanisms that shape their choices. They have the game itself. And they have instant access to the collective wisdom of the player-borg’s vast hive mind thanks to all the commentary and guides here in cyberspace. It is these network effects that make the distribution take the shape that it has.

the g-spot

May 26, 2009

The site should now be fully updated for patch 3.1.1. Just in time for patch 3.1.2, yah!

That does raise a serious issue – this blog-style site is becoming harder to maintain as the number of data pages grows. For example, given the blog management tools available, I can’t see any simple way to even be sure that I have updated every page. If you see one I missed, let me know.

And the number of pages is going to continue to grow, making navigation more problematic, given the simple set of link-y widgets available. The next big thing on my to-do list is to divide the level 80 reports up into PvP and raiding groups. Reports are going to multiply faster than the proverbial pair of breeding rabbits.

Over at Armory Musings, Okoloth, one of the original WoW dataminers, talks about why he moved his main reports to Google’s app engine. When I read that a while back, I had a bit of a d’oh moment – I’d just finished the reporting tool chain for this site and didn’t want to scrap the whole thing and start again. But his logic can’t be faulted – blog sites just don’t have the flexibility needed to manage complex sets of pages, especially when each page has to be updated regularly.

Google’s app engine has all the good stuff  – fast, free and flexible. Can’t ask for more than that!

So I’ve been busy playing with Google’s development environment. You’re welcome to have a stickybeak at what I’ve been up to here. There is no data there yet, and there won’t be for a while yet as I’m still thinking about how to structure the site.

But you can see from the menu bar over there where I’m trying to go: a navigation system that will take you directly to the reports you are looking for and with room to expand through several levels, as the reports themselves become more selective.

And at last we should get those nice item tooltips that every other WoW site manages to have.

minority report

May 11, 2009

Once again it’s time for our check on where the latest patches have left class balance. This is the state of play at level 80, post 3.1.1:

Class Popularity
Death Knights 15.0%
Paladin 14.2%
Druid 10.3%
Warrior 9.7%
Mage 9.6%
Priest 9.5%
Hunter 9.3%
Shaman 7.6%
Rogue 7.3%
Warlock 7.3%

As you can see, the trends we noted last time have not been reversed. The only new thing here is that another class – the rogue – has joined warlocks on the Azeroth endangered species list. This is offset by the rise of the druid as the hybrid class of choice a popular hybrid class and the continued enthusiasm for paladins and DKs.

moar content plz?

May 8, 2009

Things are progressing nicely here. My scanner is past the 100K character mark – just a bit behind where I’d hoped to be by now. All the new glyph and enchant pages are up. I’ll refresh them at the end of the scan when I have a bigger data set.

And as always, the raw data has interesting things to say. What struck me this time is how many characters are piled up at level 80. Here’s the chart:

Character distribution by level

You can compare that with the distribution at the beginning of the year. The logjam of level 70s has been cleared, which means people have taken most of their alts to 80. Of course that begs the question – what next when you’ve got all your toons to 8o?

I must admit that this raises a design issue that I have never understood. Why does Blizz downplay the levelling experience so much? Some of my fondest memories in the game are instance runs through some of the mid level content like Blackfathom Deeps. A bit challenging for a leveling character and with plenty of lively, fun fights.  There is so much content in old Azeroth that has been devalued by the changes since BC.

True, a  lot of it is just kill-10-rats quests, but surely it would be easier to beef up that content than create complete new zones. A few creative writers to add new questlines with a bit of variety and challenge must be gazillions of dollars cheaper than the army of artists needed to work up a whole new continent.

Update:

It’s worth saying that this chart puts the dual specs data in a different light too. In my post on that subject, I said that the take-up of dual specs was lower than I was expecting. A commenter pointed out that I didn’t seem to have taken alts into account. Well, I had, sorta… I thought that most alts were still back in the 70-79 range and weren’t being counted in my sample.

LOL! I can see that that assumption was, like, so totally February 2009… It’s clear now that the commenter was quite right. If about 50% of all level 80s have dual specs, as they do, then everybody’s main has them, statistically speaking at least. And a good few alts as well.

It looks like the patching frenzy has subsided so I’m building my next database now. Shouldn’t be too much longer till I can refresh the site. Also, the data in my test database is good enough to start generating the new glyph and enchant reports that I’m planning, so I’ll get them up in the next few days and refresh them when the current scan completes.

In terms of reports for glyphs and enchant, I’m just planning simple popularity tables of the type you see on the gear and gems pages. There doesn’t seem to be much more required – but, as always, suggestions are welcome.