This request is silly. The opinion of random people is worthless because:
Unless someone does really serious opinion study with proper sampling, getting opinions about a fact is useless. The funny thing is that exact answer can be given to the question. All we have to do is look at WoWprogress.com that tells how many guilds done the achievements. There is a reason why Supermassive have been achieved less times than Siege of Ulduar.
So the difficulty list (copied on VI.24):
There is an obvious error in the list, more people have One light in the darkness (Yogg-3) than Two lights in the darkness (Yogg-2). The same error exists with FL+3/4. Most probably the system reads wrongfully these consequential achievements, maybe ignore guilds which get 2 of them at the same time. This undermines the reliablity of the data. However I checked my own guild and Larísa's and all achievements were in place. So (until someone finds a serious difference between an armory profile and the site) we can assume, that the database is correct, except for step-by-step achievements.
It says "The Siege of Ulduar (H): 16999 (31.92%)". From this, the 100% is 53255 guilds. If we assume 35 people/guild we get 1.8M people as 100%. We know that 11M people play WoW. The site just tracks EU and US servers, but it's still around 7M. The solution is that they don't track non-raiding social guilds. So 1% should be read as "1% of those who even attempted to raid". If you have an achievement and want to know how rare is it in the whole playerbase, divide the listed number by 3.
Iron Dwarf medium rare is not harder than FL+4, just silly, so no one bothers to do it. There can be other silly achievements on the list. They are not on a wrong place, they should not be on the list at all. The question "is dwarfaggeddon harder than Kiss and make up" is unanswerable, as neither is hard, just silly.
So, if we validated the data, and filtered out the silly nonsense, we get an exact answer to the question of outDPS.
There is always data somewhere. Find it. Use it. Opinions are unreliable.
- Opinions can come from people who haven't even seen Ulduar (unreliable)
- Their opinion does not represent the opinion of the whole population (unrepresentative)
- "achievement I have" is more valuable than "achievement I don't have" (biased)
Unless someone does really serious opinion study with proper sampling, getting opinions about a fact is useless. The funny thing is that exact answer can be given to the question. All we have to do is look at WoWprogress.com that tells how many guilds done the achievements. There is a reason why Supermassive have been achieved less times than Siege of Ulduar.
So the difficulty list (copied on VI.24):
- Alone in the Darkness: 0.00%
- Supermassive: 0.00%
- He's Not Getting Any Older: 0.00%
- Conqueror of Ulduar: 0.01%
- If Looks Could Kill: 0.02%
- Set Up Us the Bomb: 0.02%
- Shadowdodger: 0.05%
- Glory of the Ulduar Raider: 0.06%
- They're Coming Out of the Walls: 0.06%
- Observed: 0.06%
- Getting Cold in Here: 0.07%
- Firefighter: 0.08%
- Unlock Algalon's room: 0.08
- Lumberjacked: 0.09%
- Two Lights in the Darkness: 0.12%
- One Light in the Darkness: 0.14%
- Deforestation: 0.19%
- 3 x Knock on Wood: 0.23%
- Can't Do That While Stunned: 0.32%
- 2 x Knock on Wood: 0.45%
- I Love the Smell of Saronite in the Morning: 0.49%
- Iron Dwarf, Medium Rare: 0.66%
- Nerf Scrapbots: 0.68%
- With Open Arms: 0.75%
- Three Lights in the Darkness: 0.91%
- Con-speed-atory: 0.95%
- Getting Back to Nature: 1.02%
- Rubble and Roll: 1.03%
- I Choose You, Steelbreaker: 1.13%
- Nine Lives: 1.42%
- Must Deconstruct Faster: 1.65%
- Lose Your Illusion: 1.86%
- Not-So-Friendly Fire: 2.06%
- Crazy Cat Lady: 2.06%
- Nuked from Orbit: 2.49%
- Knock on Wood: 2.80%
- I Could Say That This Cache Was Rare: 2.84%
- Orbit-uary: 2.97%
- Heartbreaker: 2.98%
- Cheese the Freeze: 3.61%
- Siffed: 3.80%
- Stokin' the Furnace: 4.92%
- Unbroken: 5.19%
- Don't Stand in the Lightning: 5.21%
- I Choose You, Runemaster Molgeim: 5.78%
- Drive Me Crazy: 6.50%
- Descent into madness: 7.29
- Orbital Devastation: 8.99%
- Disarmed: 10.59%
- The keepers of Ulduar: 16.28
- I'll Take You All On: 19.31%
- Orbital Bombardment: 21.42%
- I Have the Coolest Friends: 22.35%
- A Quick Shave: 23.34%
- Shattered: 25.11%
- The Antechamber of Ulduar: 27.22
- Nerf Engineering : 28.63%
- Shutout: 30.19%
- The siege of Ulduar: 31.99
- Take Out Those Turrets: 32.78%
- Nerf Gravity Bombs: 39.32%
There is an obvious error in the list, more people have One light in the darkness (Yogg-3) than Two lights in the darkness (Yogg-2). The same error exists with FL+3/4. Most probably the system reads wrongfully these consequential achievements, maybe ignore guilds which get 2 of them at the same time. This undermines the reliablity of the data. However I checked my own guild and Larísa's and all achievements were in place. So (until someone finds a serious difference between an armory profile and the site) we can assume, that the database is correct, except for step-by-step achievements.
It says "The Siege of Ulduar (H): 16999 (31.92%)". From this, the 100% is 53255 guilds. If we assume 35 people/guild we get 1.8M people as 100%. We know that 11M people play WoW. The site just tracks EU and US servers, but it's still around 7M. The solution is that they don't track non-raiding social guilds. So 1% should be read as "1% of those who even attempted to raid". If you have an achievement and want to know how rare is it in the whole playerbase, divide the listed number by 3.
Iron Dwarf medium rare is not harder than FL+4, just silly, so no one bothers to do it. There can be other silly achievements on the list. They are not on a wrong place, they should not be on the list at all. The question "is dwarfaggeddon harder than Kiss and make up" is unanswerable, as neither is hard, just silly.
So, if we validated the data, and filtered out the silly nonsense, we get an exact answer to the question of outDPS.
There is always data somewhere. Find it. Use it. Opinions are unreliable.
19 comments:
There's also one thing to note with the data. Even after guilds disband, they're often still listed on WoWProgress. I'm sure there are countless guilds that died as they struggled with Ulduar progression, so the total # of guilds may be an overestimate.
It's also a bit skewed by raiding alliances, where you'll have members of more than one guild at a kill. This won't happen so much at the top end but the number of guilds who have gotten an achievement doesn't equate to the number of raids who have.
eg. I'm not in a raiding guild, but we have people who raid in different alliances (some of which are more hardcore than others) so the guild will show as fairly progressed even though we don't do guild runs.
You got an essential error in your thoughts, because there are achievements the raids will wipe for and there are achievements noone cares about.
"If Looks Could Kill" for example is just to uninteresting to wipe the raid just to achieve it but it's definitely not as difficult as the hard modes.
Data being fact is absolute, but data without correct interpretation can also be unreliable.
Someone who doesn't know that one achievement is harder than the other can look at this data and interpret it incorrectly.
As Timo says, some achievements may be so boring no one bothers with attempting them, this then makes them appear as completed by less people and thus to the uneducated just appears to be harder than everything below it.
err... he did mention about analyzing the data correctly..
"So, if we validated the data, and filtered out the silly nonsense, we get an exact answer to the question of outDPS."
There are still some methodological problems here:
-The unit of analysis is guilds, but guilds don't complete achievements -- groups of players do. This could skew the data.
-Only completion of achievements, not attempts, are considered. Is an achievement still easy if the wipe rate is very high? Is a hard achievement still hard if the wipe rate is low?
-In general, players do not get the same number of attempts to learn various boss fights. Early bosses are generally better understood than late bosses. Algalon is a very good example of this; he can only be fought for 1 hour every week.
-Difficulty is not univariate. Different fights require different things of a raid group. Is the boss fight a dps race, a fight for survival, or something else? Different groups may experience very different rates of success as the type of fight varies. A multivariate approach is needed to capture this phenomenon.
-Gear, spec, player skill and group composition are not being controlled for. A group in T8 will not perform the same as a group in greens.
Nice article, I agree with you totally. Obviously, any analysis will need to have proper methodology; you obviously just came up with a 'possible' one (and not neccessarily complete), and invite people to think for themselves.
I've read so many complaints on the forum on my class sucks, etc. because of this, this and this, etc.
Then someone links in arena representation numbers which invalidates their argument and they say that statistics are meaningless, LOL.
There are even people that say, 'X class is top of this battlegroup, so that class must be doing good, L2P!'
They'll say that even if the said class has terrible arena representation. One outlier doesn't mean anything!
Better question is to list the hardmode achievements in terms of difficulty. Since more than half of the achievements listed are either Vanity achievements (wow we killed FL+0 without repairing, thats hard) or Meta achievements (Glory) no one is really bothering to do them unless it's definately necessary. We won't be doing "Iron Dwarf Medium Rare" for example until we have all the other hardmodes for the Glory complete.
So the list should really be of the following achievements:
Alone in the darkness (one light etc, alone is also currently bugged according to the guilds trying it).
Firefighter
I thought this cache was rare
Knock,knock,knock on woods
I smell saronite in the morning
Crazy Cat Lady
Siffed
Heartbreaker
Orbit-Uary
Disarmed
Iron Dwarf, Medium Rare
Stokin' the furnace
I choose you steelbreaker
Of those the true hardmodes are those that gives extra loot which removes:
Iron Dwarf, Medium Rare
Stokin the Furnace
Disarmed
Crazy Cat Lady
And then we have the final achievement, to defeat Algalon:
Observed.
Now the list will start getting interesting, since you can not just check the amount of guilds that have downed the different achievements for a correct list since Algalon is easier than Firefighter and Freya+3. However, you can't reach Algalon without killing Freya+3 or Firefighter which makes the list harder to Balance.
I wont do my own list since it still just would be anecdotal (however I believ it would be a more true list since I would base it upon hearing from the top 20 guilds in the world which achievements are the hardest.)
Sometimes the Armory just doesn't record intermediate achievement.
Example: http://eu.wowarmory.com/character-achievements.xml?r=Al%27Akir&cn=Giskler&gn=Apex
The DK got "One Light in the Darkness" without "Two Lights in the Darkness". The previous achievements are missing.
The fact doesn't influence the scoring but may spoil the achievement statistics, so probably I'll add a workaround.
> there are achievements the raids will wipe for and there are achievements noone cares about
Absolutely, you should keep in mind that difficulty and rarity aren't the same things.
The points made in former comments are mostly correct, but I'd say they aren't relevant in really meaningful achievements.
I am confident there is no guild alliance, pickup group, ungeared or highly unbalanced raid that has done Heroic: Firefighter or Heroic: Knock Knock Knock on wood. Thus we can, according to the data, say that Heroic: Firefighter IS harder than Heroic: Knock Knock Knock on wood, at least in some sense.
[i]"-Difficulty is not univariate. Different fights require different things of a raid group. Is the boss fight a dps race, a fight for survival, or something else?"[/i]
I would also say that guilds that are ready to attempt any of the hardmodes should have sufficent DPS to manage the DPS-requirements once they have *learned the encounter*. Hodir hardmode requires about the same DPS-capacity as Mimiron but learning how to stay alive to deal that DPS is waay harder in Firefighter. I think the fourth phase actually really is hard to avoid all the stuff while doing your job in the same time (I must admit I quit before finishing Heroic: Firefighter, but having experienced 10% wipe in fourth phase gives me credibility to say that) whereas Hodir is more of "stay in starlights, bring storm power to other players, avoid avoidable damage so that you can do it with few healers".
In other words the meaning of "hard" mainly is how long it does for whole raid to learn the encounter. Whether it is DPS-intensive or l2p-based doesn't really matter.
Auriaya (spelling?) isn't difficult to beat, but the Crazy Cat Lady Achievement shows only 2%; probably because it is takes too damn long.
Why waste time on it when you could be killing another boss?
Re: Outdps and his request for opinions; if you really believe that the 'opinion of random people is worthless', then why allow comments on your posts?
Most of what I write is silly. Tongue in cheek, at any rate. I was doing this impromptu survey to "win a conversation".
That said, thanks Gevlon! I didn't know about that site. I assumed it would be possible to get that type of data, but hadn't seen it in the wild :)
Playing devil's advocate.
Isn't most of what you write about opinion? Not everyone agrees with you. I certainly don't on most of your philosophical and political points.
Besides, Outdps did just want a general consensus from the people who read his blog. Sure the data wouldn't have actually won him the argument, but still...
Wait you're throwing out some of the data based rankings on how hard some achievements because the data makes them harder than you think they really are in a post called Data > Opinions?
"Filtering Out Silly Nonsense" is an opinion function.
The data say they are hard--the reward for them isn't worth the risk.
You sir, are trying to pull a fast one.
Actually you're pulling two fast ones:
The title of the blog entry on the site you reference is "Ulduar encounter difficulty levels!"
The word achievement appears no where in his post.
And a 3rd fast one: there are opinions on the source of the data too.
So really your thesis is opinions + data > wild ass guess.
And that we can't trust your what you say about posts you link to.
Opinions about a fact are worthless. Find the fact.
There are several things (like world economy, politics, philosophy), where hard facts are not only not available but not even exists (as no one can know the future). So opinions are all we have. This case opinions are worthy.
If opinions about facts are worthless how are you discarding the silly achievements?
The data you draw upon doesn't differentiate between silly and non-silly. You are making a judgment (rendering an opinion) as to the relative value of each piece of data.
Mark Twain wasn't far off the mark when he said "Lies, damned lies and statistics." Choosing your data sources the correct way and you make the statistics sit up and bark.
If you don't have opinions about data I can feed you any old data set and have you draw whatever conclusions I want you to draw.
Data analysis is not as easy as it seems because most people seem to lack the ability to properly analyze despite their own assertions otherwise. Even when done properly, there is no such thing as a perfect data analysis and sometimes it can be easy to get sucked into the whole process of analyzing the data while forgetting the whole point of the analysis.
I guess what I am trying to say is, is it even important to know what is really the hardest achievement?
Yes, ego and laziness cause people to want to offer opinions instead of analysis of facts. But "opinions are worthless" is clearly wrong, especially in areas where there are few facts (predicting the future) or where the opinions are the facts - opinions matter when you are surveying consumer or voter opinion. "The Wisdom of the Crowd" was the book title; "wisdom" might be overstating it a bit. You wouldn't put a lot of faith if you ask a M&S neighbor or co-worker about the stock market's future or a future sporting event. But it seems a reasonable premise that asking a 1000 or 10000 could be more reliable; or that the the latest betting line/stock price is really just the net of millions of opinions.
Besides, for writers in general and bloggers in particular, it really is about being read not right?
Post a Comment