Greedy Goblin

Thursday, January 20, 2011

Does size matter?

The Pug update: On Wednesday, after reset, in a 3 hours raid we killed all 4 farm bosses (Omnitron, Magmaw, Valiona, Halfus). Those who were here for all 4 (many changed), got 3K pot. Are you still sucking in 3 hours long heroics with your "friends"? Join now! Just read rules!

I wrote about this in December, and found that the bigger the faction, the smaller the chance you find raiders (and consequently, the larger the chance to bump into M&S). Many commented that the early date was inappropriate as only the HC guilds killed anything back then. So I did it again, this time including all EU and US data, except Russian servers, because for some reason warcraftrealms did not track them.

The data acquisition was the same: wowprogress has a list of all realms, and it has a data field that count "raider population", people who killed at least one boss. It also has faction breakdown. Warcraftrealms has a similar list for server population, also with faction breakdown. I ignored the factions below 2000 population as it's hard to tell if they are really that small or simply no one ran the census addon. Unlike last time, when I plotted raider %, now I plot raider number vs population.

Obviously there is large noise, but if we place a trendline on it, one would expect that a 2 times larger faction have 2 times more raiders, or actually more, as the common belief is that on larger factions it's easier to recruit, there are more guilds, easier to find good pugs, and so on. Well, the facts are pretty different from that:
An average 2000 person faction have 1000 raiders. An average 20000 person faction has 2000. The "y = 0.0551x" part of the fitted equation means that if a faction gains 1000 people it gains only 55 more raiders. You think it's nasty? Then look at this! Wowprogress give the whole servers scores and rank them according to these scores. Behold the score vs population chart:
"R2 = 0.0012" means that the two variables, score and population are uncorrelated. It says that increasing population does not increase score at all. You just added warm bodies. This seems to be in contradiction with the previous chart, the solution is that large population servers have 2 pages of 1/12, 2/12 guilds, while on low population servers they are rare. Such guilds add to raider count but not much to score.

This result needs explanation. I mean people start playing on random servers, not knowing anything about raiding progression. When I started playing, I just saw another RPG with huge world and nice graphics that I will play with my girlfriend for a few months until we complete the game. Random distribution is random, so each server must have the same chance of getting someone who will be a good raider once he learned the game. Also the chance of someone remaining casual and collecting minipets and silly things like winter hats is the same on all servers. All servers should be the same! With two times more players you must have two times more raiders, therefore two times higher score!

I believe the above part is true. All servers have almost the same amount of raiders and casuals. What turns the system upside down are the M&S. Morons and slackers are not casuals. They are not just "bad players". They are bad players who refuse to learn to play, yet they feel entitled to rewards. Raiders learned to play, casuals ignore epics and just play. M&S suck and want to be carried.

In a small faction they soon visit every single guild. The raider guilds ignore their applications at best, send it to me for "morons of the day" at worst. The casual guilds accept them, but the M&S is not happy there, as casuals can't boost them to epics. There are few obnoxious M&S guilds (/2 xXxDethkillazxXx need moar active geared tankz and hilz 2 start raiding whisper ipwnudie 4 inv) but after a few attempts they tend to dissolve or transform into an alt-boosting lolguild. When the M&S runs out of options, he reaches the conclusion that "this server suxxs cokk hard", and moves to another one. Since larger population servers have more guilds, it takes much longer for him to get rejected from all the raider guilds and to figure out that the countless casual ones can't boost him. Also, on such servers the M&S guilds spawn and dissolve fast enough for making the impression that there is always another guild to try out, despite they are just permutations of the same M&S base. So these servers act as black holes, sucking up all the M&S.

Conclusion: if you are looking for a server to raid or to casually play, avoid the high pop ones like the bubonic plague! They have only 4 things that the low pop ones don't: server queues, mobcamping, unusable trade channel and legions of M&S.

PS: I don't want to imagine how Tol Barad looks like on a 20K+ server. I just don't.

30 comments:

Anonymous said...

Squishalot here.

I'm not convinced by your theory, interesting as it is.

Aside from the fact that there are discrepancies in your data (for example, EU-Alleria, with ~25,000 players according to WarcraftRealms, has ~2,900 raiders according to WoWProgress, no data point on your chart), your conclusion assumes that raiders and casuals will only play on the servers they start on.

In reality, raiding guilds will transfer across realms in order to get Realm First! achievements and other such plaudits. As a result, it is entirely expected to find a relatively even distribution of high-achieving guilds (as evidenced by WoWProgress score)across servers.

Consider what an abnormally high score on a single server means. A high score on a single server means that there is a lot of competition for progression on that single server. A raiding guild with little chance of getting Realm First! achievements will transfer so that they can make the claims.

Gevlon said...

Alleria is in the data, I just cut the chart plot (and not the calculation) at 1500 because only a few points above that.

Since the guilds on WoWprogress are ranked worldwide, there is no point transfering to other server, but even if it happens, it does not affect my conclusion: the progression of servers are even, all servers have about the same raider community, bigger servers just have more M&S

Nils said...

Blizzard tries to balance the servers for active players. The bigger servers may have more different people, but may be similarly active.

I guess there is not one single explanation and you should perhaps try to avoid the "If the only tool you have is a hammer, ...."-pitfall

Not everything has to do with M&S :)

Gevlon said...

@Nils: Blizzard can't really balance servers as they don't forcibly move players between servers.

chewy said...

As with your last post on this subject I'm still neutral because I don't believe you have enough control over the data to formulate any absolute conclusions.

The distribution of players across servers is not truly random. I can think of at least two factors that will interfere with the random distribution. Firstly the nationality servers which will skew the data towards a language group and secondly the technical/commercial bias that dictates Blizzard will try and load servers to an optimal utilisation (providing a balanced response time).

You could argue that M&S transcends nationality groups and that could be true or indeed that if Blizzard fill servers sequentially only providing new ones when the last one is full that this is still a random distribution (partially true) but to be sure you'd need to have the ingest data numbers, how many new players are joining per week, where are they joining, how long do they stay and most importantly as Squish points out - how much migration goes on.

Ðesolate said...

If I take a fast look at my statistic I get similar results.

When I compare Frostwolf to Malorne (full server to almost dead) I get following data:

Frostwolf: 284 total
17 good player - 5%
104 average - 37%
163 M&S - 57%

Malorne: 84 total
5 good player - 6%
62 average - 74%
17 M&S - 20%

Of course compared to the playerbase theese are small numbers. Another thing is it is mostly taken from the LFG-tool (some frostwolf data are from a personal friend of mine).

Anonymous said...

A lot of what has been said here doesn't agree with my experience.

I started on a small server and moved to a big one. The original server had 4 raiding guilds on my faction, though only one did ToC hardmodes and two did any ICC hardmodes. On the new server, there were at least 20 guilds doing ICC hardmodes; the number of guilds on my faction is huge.

For players that want to raid, there is one very important metric that is overlooked in Gevlon's original post: number of players/guilds with raiding goals similar to your own. Proportion of raiders does not address this issue.

Consider: on my old server, I really only had one guild option for performance-oriented raiding. If that guild already had enough players of my spec/class, too bad for me. If I had a personality conflict with the guild leadership, too bad for me. If their raid nights conflicted with my schedule, too bad for me. My choices were very limited. By contrast, I have been able to find a guild on my new server that very closely fits what I want out of raiding. This is no accident; I was able to select from ~10 guilds that were a close match to find one that fit particularly well. The population of M&S has no effect on the number of good guilds for me. The ability to have multiple options in terms of guilds is a major reason why high population servers are favored over low population servers for raiding.

I will say, though, that Gevlon's claims appear to be about right when talking about random trade PuGs. On both servers, they were pretty bad. But anybody that server transfers solely to raid with random trade PuGs is wildly misguided.

Riptor said...

@ Squishalot: Do you have any example, for your Theory of Raiding Guilds moving to other Servers for realm first Achievments?
Sure there was a certain raiding Guild that switched Realms as they claimed unbearable Lags on their overpopulated home Servers cost them a world first kill, but other than that, can you name one renowned Raiding Guild that has ever done that?
Sure there probably were some semi casual Groups that used free Character Migration as they had l33t Skillz but their Real Live kept them from getting a real first but they do not count really. If you are a competitive Raiding Guild it’s the World ranking your interested in. If you switch Servers to get a Realm First you just as good as that Pet-Collecting Casual.
Honestly, I don’t know what it is with the Bottom 95% of Raiders today. They even seem to think that a 10 Man Raid should be held equal to a 25 Man. Just because Blizzard gave them a “you are worth as much as any other Player that raids” Mode and plasters everybody with the same Item level does not make it equal. It is like winning the Hot Dog eating Contest at your local Fairground. Just because the Medal looks the same, it’s still not the Olympics and you’ll never be like Usain Bolt (sure.. Apples and Oranges, but honestly.. Apples – Oranges.. 10 Mans – 25 Mans.. it fits)

Vinnz said...

I rather agree with your conclusion. However, I don't agree with the demonstration part...

I'm not really sure that your trend line on first graph should be used.
R²=0.22 is still really low. I would consider that there's a way to fit your cloud (the distribution is not purely random), but that the linear fit is not the appropriate tool.
Possibly the data is too wrong/noisy (there's at least one data point saying that a 2000 population server has more than 2000 raiders.... ; and I cannot find a way to check the quality of each measurement). Moreover the extrapolation at x=0 population (y = 896.63 raiders)is a bit strange : I know taht my extrapolation is out of the cloud, but I cannot imagine a totally empty server with around 900 raiders being a sampling artefact.

I suggest a simpler description of the data : "Whatever the server population, the number of raiders will most probably stay between 500 and 2000.

Grim said...

@Riptor
What are you trying to suggest that 25 mans are real raids? 40 man or gtfo noob!

*shakes cane*

Currently in terms of gameplay 10 man raids are actually harder than 25 mans.
So why should 10mans be considered inferior?

Anonymous said...

Gevlon,

I really like your analysis. One of the things that I hate about most of these debates in Wow-land is the lack of real objective data to frame discussions.

Yes I could quibble about the R^2=.22 not being very correlated. But, meh the data is solid and the point does not depend on R^2.

To everyone who says "your analysis does not fit with my experience" posters. Please stop touting personal experience as being equivalent to what Gevlon has put together here. If you don't have data from a wide sample you really can't make any meaningful rebuttal to his data.

The conclusions that Gevlon makes are a bit far afield. I would say that "raiders" and "M&S" distribute themselves differently.

For example perhaps "M&S" accumulate like lint on servers as they age. But, raiders distribute themselves more evenly. It has been my experience on 3 servers that there always appears to be a finite number of good raiders. This number does not seem to change AND gear availability (like late Wrath) did not seem to improve the numbers.

Being M&S always seems to be a state of mind rather than anything that can be educated away.

Angry Gamer
Angry at Morons since Diablo

Carlton said...

There is problematic data within Warcraft Realms. If you look at the activity levels displayed in the Alliance/Horde ratios, most of the low (below ~15k) realm population data is flagged 'red.' This indicates that the server has received less than 50 snapshots within the past 30 days, and according to the site is unreliable.

One such consequence shows in Vinnz's comment in regards to more raiders than server population. A more reliable analysis would only be those factions with the 'green' qualifier, which Warcraft Realms considers ''fairly acceptable." However, this would eliminate much of the low population realms that are the required data points for your current analysis.

Anonymous said...

All you can do on a 20k server is imagine how tol barad is. because you only get into the battle about 1/20 times.

Sheldon said...

As Vinnz pointed out, some of your data points show some servers with more raiders than players. This is a clear indication that the raw data is unreliable. This renders the analysis suspect at best. I don't know about how things function in your world, but in my line of work we train people to never publish results, let alone conclusions, unless you are damn certain of the underlying data.

Gevlon said...

@data quality whiners: care to find better data? No? Then we must work from what we have. It's unreliable, true. But unless we assume it has systematic error, it's useful. Random error is random.

Anonymous said...

Considering this data is (most likely) based on # of toons rather than active acounts, this can interfere with the data as well.

For example: a M&S or casual kills Tol Barad boss with a trade pug and gets counted to the raider population. Now he will replicate that for his 9 other alt toons with 160% flying and 40 minipets each.

At the same time, a HC raider will focus to be optimal in one toon at a time, especially this early in the xpac. I would not be surprised to see that the ratio of "raiders" per M&S is actually higher than it 'should' be, with higher server populations having a larger influence from this phenomenon.

Sheldon said...

When your uncertainty bars are as wide as these are, you cannot "work with what you have" because you have nothing. If you are arguing that logic and conventional wisdom are wrong, then you need to back your words up with something solid. Calling me a whiner doesn't change the fact that all you have is a straight line drawn through a noisy cloud of crap.

Arguing that you don't know what the biases might be therefore we should just assume that there aren't any is silly and amateurish. Part of the analyst's job is to identify potential sources of error, not wish them away. Especially when the data pool is already known to be contaminated with points which are horribly wrong.

Carl said...

"They have only 4 things that the low pop ones don't: server queues, mobcamping, unusable trade channel and legions of M&S."

They also have legions of goblins infiltrating every niche market on the AH. I make way more gold on my 2nd server because it's the less populated faction and people buy anything off the AH and few people are there to compete with me.

T. said...

This can be considered to be true. However, it makes the vital assumption that EVERYBODY WANTS TO RAID. Which just feels counter-intuitive to me as a pvp-er. While I do agree with your conclusion that large servers have more M&S, it feels like your scope is too small and does not measure the right statistic. A better statistic would be gladiator:the amount of active arena teams on a server. This would use an already existing stat (glad). There should be a much lower ratio of players getting glad on high pop servers than low pop servers. It is a fairer assumption that everyone who does arena wants to win as much as possible than that everyone who plays wow wants to raid.

I have a feeling that the effect of social reasoning and actual benefits of playing on a larger server were not explained by you. Large servers did have more PuG's, so it was beneficial. People were lured in by successful PuG stories of servers where IC progress in PuG's was 11/12 HC. Mainly the high performance was advertised in channels (wow forums) so new players did not get the right idea of the general quality of players. You actual had to have the right connections and amazing performance to belong to these PuG's, they however did not know this. But because the content was piss easy there were many PuG's formed by idiots were still semi-successful. This gave these bad players something to do, while not doing much under for their terrible peers. When content is less pugable, like it is now, the statistic might slowly change back.
Some of the bad players are forced to learn the skills required or quit.

It is easier for a good player to transfer to a low pop realm, because regardless of pop, his performance will be valued and he will be successful (and have fun).
However, I would like to debate the point you made about M&S being rejected by guilds. I reckon that both M&S and average players are more hesitant to change to a different (low-pop) server.

2nd part: social reasoning
Let me tell you a story about a server called RealmA. It had a tight community without having too many players. A lot of raiding officers were actually into contact with each other. The quality of the average player was high. Pvp was in demand and popular. Players like PlayerI, PlayerZ and PlayerG became well known because of released movies.

When X-Realm was opened the difference of quality of the player base was noticeable. There were several premades that have been undefeated versus all other teams on the other realms. It gained a good reputation as one of the best PvP realms (along with Stormscale), partly because of immature bragging and flaming.

When X-realm transfers opened up the real problem began. A lot of players wanted to be part of that feeling, to belong to something bigger (or to be competitive). A lot of players transferred over because of the idea of belonging to something better. The average quality of players dropped. Only the already good players were capable of joining the already formed high quality communities. However, for the bad pvp-ers there was something to do. I had an alt in a social (either as not requiring 2200 rating or just social chat) guild to see what it was like. Comments like 'I just dueled -insert famous player-, he is so amazing or 'i just saw -insert player name- in org' were common in this chat. They felt like they belonged to something bigger and this enhanced their gaming experience. It boosted their ego's.

Squishalot said...

@ Riptor - Aside from your horrible capitalisation, consider the fact that 'renowned raiding guilds' won't need to change servers, because they're the ones getting the Realm First achievements in the first place. So by definition, the ones changing servers are second and third tier guilds. And yes, I have seen a number of those on the Oceanic realms that I play on. (Not always for the sake of Realm First achievements, but for any number of reasons, even just shifting PvP to PvE.)

Squishalot said...

@ Gevlon - I was referring to the first chart. There is no data point matching with 25000 players and 2900 raiders. There is one that appears to be at around 25,500 players and 2750 raiders.

Anyway, what is interesting is that I ran the same analysis you did on the US realms, and it resulted in a positive coefficient for server score vs population size. So in the absence of any statistically meaningful data, it's fair to say that we can't get much from these at all.

Anonymous said...

The coefficients of the regression are actually downward biased if there is measurement error. So if the data has substantial amounts of measurement error, you will find a slightly stronger effect of size on raiding then the small effect in your equation.

jaakkeli said...

The problem is that you can't separate alts and mains. Right now most of my guild has their mains raiding and then a bunch of level 85 alts that we don't play enough to raid on yet. Leveling the alts with rotating rested XP is easy but most of us are too burned on the gear grind on mains to even get the alts up to the ilvl to queue for a random hc.

My guild migrated just before Cataclysm from one of the lowest pop, "recommended to new players" servers to a fairly high population one. We were "the top guild" of the old server for much of ICC and most of the applicants we got there were new players who had leveled just that one character. When we left we took with us over 100 max level toons from a horde population of slightly over 3000 max level toons so it does have a huge influence on the numbers.

Individual raid progression of good players would pretty much always go the same way on that low population "new player" server. You'd just eventually find yourself in one of the "top guilds" on the server without much effort since the population was so low that you'd just end up knowing most people who raided through alts and pugs and soon you'd be known to be a good player. Then you'd be stuck raiding in a "top guild" with some people who sucked beyond belief just because you needed their class or buffs and then you'd find that a boss like LK25HC before 4.0.1 simply requires good players of certain classes or you just can't do it.

You'd spend your time farming the 11/12 easy hardmodes, level alts and so on and then eventually just find a higher population server to raid on. During ICC the "top guild" position changed 5 times, twice because the top guild disbanded after most of their raiders migrated and three times because the whole top guild migrated... so there was a constant stream of people migrating their raiding mains and their stable of not really raiding alts to higher population servers while new players who reached the max level would be constantly killing a boss or a few while trying to start raiding on their one character.

Sean said...

The graphs and the conclusion where adding more population leads to diminishing number of raiders is correct. However, I fail to see the proof where these other players are M&S. They could be casual players.

Also, another thing to point out is that the larger server does have more raiders and hence more choice.

I think I prefer to stick to a high population realm.

Anonymous said...

Gevlon says:

"@data quality whiners: care to find better data? No? Then we must work from what we have. It's unreliable, true. But unless we assume it has systematic error, it's useful. Random error is random."

When the error bars are bigger than the effect size, you probably need to reject your pet theory in favor of the null hypothesis.

Gevlon is the one making the claim. Therefore, the burden of finding good data falls on him. Labeling critics as "whiners" when they rightly point out a flaw in his methodology suggests that the conclusion is driven more by ideology than data. I think Gevlon would get fewer arguments by improving his data collection procedure and presenting new results.

Furthermore, it is not hard to believe there is a systematic error in the Warcraftrealms data. For some servers, there were very few snapshots in the last 30 days. It is possible that a large proportion of snapshots for small servers came from a single player. Depending on what time of day that player is typically logged in, you will see some portions of the player population but not others.

I've provided a lot of criticism, but hopefully the following suggestions can make it constructive:

-Throw out data points with low reliability. Pick a couple different thresholds for minimum numbers of snapshots for server data and see if the result changes.

-Break the data down into factions. Where raiding is concerned, the network effects of having other players around are based on faction and server, not server alone. This should help to make up for the loss of data points from my previous suggestion, too.

-Break servers up into PvE and PvP groupings, too. There are probably too few RP-PvE and RP-PvP servers to make it worthwhile to do them, too, but it could be interesting.

-Did the results you found hold true in the late-Wrath environment? You may be able to find more data elsewhere, particularly in some of Zardoz' stuff.

-You attempt to use the raider/character comparison to draw general conclusions about M&S populations across servers. If this is true, we might expect to see a similar effect with arena player populations. Are there proportionally more good arena players on small servers?

-Is there a way you could validate some of the Warcraftrealms/Wowprogress data? There are ways to harvest large amounts of data from the Armory. Somebody might have already collected the data for you, in fact; you would just have to write the queries to dig the answers out.

Gevlon said...

@more data quality whiners: You absolutely wrongfully claim that "since I question the consensus, I must provide stone-solid data".

There is NO scientific consensus on this field. There isn't a single post about the values of higher population realms that are considered accepted canon, like the first post of the "Restoration shaman in Cataclysm" thread in EJ.

Actually there are no posts about the values of large realms AT ALL.

I'm questioning no one's results, simply because I'm the first on this field. Try to find other posts discussing it! So my data, unless COMPLETELY useless, are the best available to decide an undecided question.

What I'm questioning is a "folk consensus", a widespread belief among random people (as opposed to researchers). It has NO DATA to back it up at all, it's just superstition, random beliefs coming from sheep mentality. Again, you are free to link ANY original research or review article on the topic that serves as the basis of the current consensus.

Example: Einstein challenged the consensus of the Newtonian mechanics. The Newtonian mechanics existed, it was backed up with data, it was believed by all authors. So Einstein needed stone-solid data.

On the other hand the ancient greek scholars in Pythagoras's school who - based on the fact that from a ship you first see the towers of a city - came to a conclusion that the Earth is a sphere, did not question any preceding SCIENTIFIC system, just the beliefs of peasants who thought that the Earth is flat. So despite their data was pretty weak, their theorem was accepted among the educated Greeks.

sonickat said...

I think something easily overlooked in your data but which is impossible given your sources to correlate is how many characters are unique accounts.

Most people I know who raid do not do so on multiple toons, however most people I know who raid have multiple characters they play at the level cap.

This means your data gets skued when it comes to population data to raider ratios.

The better correlation is unique accounts to raiders.

I think the net result is that your applied logic is backwards. You don't have more raiders because you have a higher population server. You have a higher pop because you have more raiders on the server. Because different people enjoy different amounts of diversity the amount of population increase you get varies.

Sheldon said...

Comparing yourself to ancient Greek scientists doesn't fly. Their data was completely reliable and accurately predicted what would happen every single time. This is why educated intelligent people agreed that they were right. If they had tried to show people data that said that when sailing into Memphis you can see 125% of the towers before you see the rest of the city, they would have been (correctly) laughed at.

Having limited data is not the same as data which is demonstrably wrong. Calling people who point that out to you whiners is puerile.

Anonymous said...

Gevlon says:

"You absolutely wrongfully claim that "since I question the consensus, I must provide stone-solid data"."

No, this is not a correct interpretation of what I mean by "null hypothesis". Consensus plays no part in why I have criticized your analysis. Rather, in the absence of strong data the only statistically valid conclusion that you can draw is "my data does not support the hypothesis that there is a relationship between the two variables." You might still be right, but you haven't shown it.

Also, another quote:

"So my data, unless COMPLETELY useless, are the best available to decide an undecided question."

Remember: garbage in, garbage out. As I and other commenters here have said, there are obvious problems with your data (eg, servers with more raiders than players) that you have not taken care to correct. I have proposed ways to clean or subdivide your data to provide additional validity tests. So even within the limitations of Wowprogress/Warcraftrealms data, you aren't taking reasonable steps to provide valid results.

All of that said, I don't want to totally discount what you are trying to do here, Gevlon. As you correctly note, you're trying to do something original and interesting in analyzing server populations. But I think it would be a shame if you aren't careful about what data you use and how you analyze it because that effort would be for nothing.

Also, I did a little digging and found a program for data collection on the armory: https://github.com/Lewisham/wowspyder Paired with some statistical analysis tools, you could have a lot of fun there.

Anonymous said...

I agree with the post who said "you don't transfer to play with trade PUGs, you transfer for a guild".

Server population doesn't interest me as much as just finding a suitable guild. It can happen to be a high-pop. realm since the more guilds, the bigger the chance I'll find something fitting my criteria.

I used to play on Argent Dawn EU. it's THE biggest EU realm population wise unless something changed recently. Most people whom I asked told me "it's an RP realm, you can't compare it like that", no idea whether they're right or not, but I was unable to find a raiding guild that would fit my rules.

I found it now on a different pve realm. They suit me in all aspects, be it raid size, class/role they were recruiting (spot I could take), raid times and frequency, loot rules, atmosphere in raids, progress.

But I don't claim I "just knew this guild will be perfect", it's always a risk, it's the second guild I tried after starting my search outside "first pick" realm.

I raid 10 man, maybe that's why I have less problem with the M&S problem, bigger raid sizes kinda create more opportunities for M&S to sneak in and hide in the crowd.

Funnily enough, my previous guild, the one I've left for current one, had bigger tendencies to accommodate M&S than my current one, despite their "higher" progress on the ladder. Their approach to play favouritism towards "friends of friends" was most significant factor why some M&S people sneaked in.