a glyph in the matrix

November 27, 2008

I’ve got some good news and some bad news…

The US armoury has just received a substantial makeover, with two new pages of data available per character. That’s the good news. Unfortunately, one bit of new information that would definitely be worth datamining – the use of glyphs – is not included. To me, that is really bad news, since glyphs actually contribute to the power of our characters; the main thing we care about, surely.

Since the glyph system is so new, there must be a real demand out there to know which glyphs and glyph combinations are popular. Unfortunately the armoury can’t tell us that yet.

What we get instead are achievements… yawn… sorry… achievements. Frankly I see most of the achievement stuff as just grease for the hamster wheel that we’re all furiously running ’round. However, to be fair, the armoury now pumps out a lot of new data, and some of it can be used to advantage.

From a quick look at the XML, I see we can now get some interesting information on:

  • a character’s cash flow:
    1. the amount of gold held
    2. a breakdown of where the money is coming from – the AH, quests, vendors etc
    3. the largest amount of gold that the character has had
  • the use of potions, elixirs and foods
  • a very detailed breakdown of where the character has been questing, raiding and PvPing.

That last point could be very useful in providing further categories for breaking down character stats. It has always been possible to guess a character’s main “playstyle” – questing, raiding, PvPing – based on their gear. The XML that describes items of gear provides an “itemsource” description; the item sources being things like “quest reward”, “pvp reward”, “creature drop” etc. But some amount of computation has always been necessary to jump from this gear data to the playstyle.

Aside from anything else, it is a statistical question since many characters carry gear from a variety of sources, especially in the case of crafters who can make some of their own gear. The playstyle has to be inferred from the majority gear source.

Now there is no need to use gear at all; a character’s activity in battlegrounds, in world PvP, in normal and heroic instances and in raid dungeons is all there for the taking.

That should make it easy to create separate sub-reports in each class for some of the main playstyles.  Battleground PvPers can be split out from characters who just happen to be on PvP servers. It should be much easier to spot characters twinked for BGs. Levelling players who make a habit of running instances can be split out from players who just quest or grind for XP.

So OK maybe the achievement data is a bit more than hamster food…


5 Responses to “a glyph in the matrix”

  1. Ralph Says:

    Have you figured out how to get to the xml for the new data then? What are the urls to get the raw xml?

  2. zardoz Says:

    I’ll update this post tonight with more details on how to get the XML. Basically there is an async ajax-type call back to the server for each category of achievements or stats on the armoury page.

  3. Ralph Says:

    Yeah, I have started to figure it out some myself now, c10 being the character statistics and such…..

    Going to start my crawler in a day or so, don’t know if you want to share data or not so we each can crawl less of the armory?

  4. zardoz Says:

    Yes that’s it. Once you get the main achievement or stats XML for a character, you will see that the page contains a set of category IDs tags like this:


    Stick a “&c=130” on the end of the URL for the main (achievements or stats) page and you will get a new XML page with the detailed information for the things in that category set. Do this repeatedly for every category ID you find on the main pages.

    I’ll put up a post with an example now.

    I’m keen to build my own database here, but what I am interested in is splitting up the work of analyzing the data.

    It would be good to have different sites that concentrate on different bits of the data – PvE stats vs PvP stats, combat stats vs skills/professions, population and gold; there’s a million things we can analyze.

    What you find is that the armoury crawler ends up being the easy bit – writing all the SQL, XSLT or whatever to get real information out of the data ends up being the tricky bit.

  5. zardoz Says:

    Oops, I see WordPress ate my XML in the above reply. Nvm there is an example in the latest post on the blog.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: