Guild Wars 2 is an incredible massively-multiplayer online role-playing game. The goal of Guild Wars 2, developed by ArenaNet, is to save the world, complete quests, and get awesome loot; objectives similar to that of many other MMORPGs such as World of Warcraft. However, Guild Wars 2 differentiates itself from its spiritual predecessors with its dynamically-changing world, incredible graphics, and the elimination of annoying and tedious time-wasters that have plagued the MMORPG genre.
The best items in Guild Wars 2, naturally, require a large amount of effort and money to obtain. In order to create Legendary weapons, players must obtain large amounts of an item known as Globs of Ectoplasm. Globs of Ectoplasm (abbreviated as “Ecto”) are obtained randomly when a player salvages a Rare item using a Salvage Kit. Due to their high demand, Ecto effectively serve as a second currency in Guild Wars 2.
Using the best Salvage Kit (that doesn’t require paying real-world money), players received, on average, 0.9 Ecto per Rare Item Salvage, a value that has been well-documented and well-tested. But after a patch released a couple days ago, players on the Official Guild Wars 2 forums and Reddit began to notice that the drop rate for Ecto was much lower, with no mention in the patch notes.
How do you prove something that occurs at random now occurs less frequently?
What is “Random”?
The first rule of any official MMO forum: never accuse the developers of making implicit changes to the game unless you have proof. Such tactics are commonly employed by forum trolls and it makes building a community more difficult.
The second rule of any official MMO forum: never accuse the developers of manipulating probability. As with normal gambling, players can hit a streak of bad luck with loot drops, enemy critical attacks, etc. But the anecdotal data from one player is not sufficient proof of any probability change. (an old meme from the World of Warcraft era mocked the userbase’s inability to comprehend the definition of random)
On May 28th, a player noted on the official GW2 forums that out of 166 salvages, he only received about 80 Ecto, or about 0.5 Ecto/Salvage. This is approximately half of the original rate of 0.9 Ecto/Salvage. Since the rate of new Ecto entering the market was halved with no change in demand, the Auction House price for Ecto began to steadily rise.
The price increase in the short-term was not economy-breaking, but when you need 250 Ectos to create a Legendary weapon in the long-term, it can add up very quickly.
Other players provided their experiences in the thread, with limited results such as “noticed this too, yesterday i salvaged 9 rares, got 4 ectos” and “20 rare at the end of my day, i save them before go bed: 0 ecto. wtf?”. Such sample sizes are much too low to satisfy the Law of Large Numbers and draw any remotely accurate conclusions.
Regina Buenaobra, ArenaNet Community Manager, eventually responded:
Changes to important things like salvage rates would be listed in the update notes if they have changed. If it’s not in the update notes, then no changes have been made by the development team.
So Word of God implies that everyone is wrong. Issue over?
Of course not!
No Such Thing As Too Much Data
Despite Regina’s dismissal, players continued to post in the thread and began quoting even larger sample sizes, such as a salvaging 500 Rares resulting in 286 Ectos (0.57 Ecto/Salvage) and salvaging 775 Rares resulting in 402 Ecto (0.52 Ecto/Salvage). With those sample sizes, the probability that the true salvage rate is unchanged from 0.9 Ecto/Salvage is essentially zero.
John Smith, ArenaNet’s resident economist, replied that he’ll “look into this and get back to you.” A day later, John made a promise that “if we change anything involving ectos, there will be public information on it.” Sure enough, on May 30th, a new patch was released with only one patch note:
Fixed a salvaging bug introduced in the last patch that sometimes caused salvage kits to yield incorrect results.
And that was that, the price of Ecto almost immediately dropped. All was well.
John left one final message: “Thanks to everyone for your help identifying the issue and providing great data for me to start with in my research.”
Hi! I am currently looking for a job in data analysis/software engineering in San Francisco. If you liked this post and have a lead, feel free to shoot me an email.
Since I currently do not have a full-time salary to subsidize my machine learning/deep learning/software/hardware needs for these blog posts, I have set up a Patreon, and any monetary contributions to the Patreon are appreciated and will be put to good creative use.