Greedy Goblin

Tuesday, March 12, 2013

API Coincidence detection against botters and a quick answer to Shadoo

You might noticed I don't like botters since they essentially cheat the game. They also flood the market with cheap things, supporting risk-free play (you don't care losing cheap stuff).

Today I post an idea of mine that could help CCP fighting botters and other cheaters by identifying their non-violating mains. The CCP policy is to ban the player by banning all his account, including non-violating ones. However to do so, they must be able to link accounts. In case of legitimate players it's not a big deal: same IP, same login pattern, often same e-mail or similar account name. However cheaters - for obvious reasons - try to hide their main account from the bot accounts by proxies, maybe even different computers.

The idea - as the title says - is API Coincidence detection. What is this? Several third party programs query your API key (that's why it's there). For example EVEMon asks the API using your keys to read the skill progression of your accounts to update the skill training plan. Accounting programs ask for your market API keys. The idea is that if the botter is lazy (and most people are), he doesn't have various instances of EVEMon. So when he starts EVEMon, it queries all his accounts at the same time, legitimate and botter alike. Using the API server log, CCP can calculate API coincidence: Qboth/min(QAcc1,QAcc2), where Qboth is the number of events when both accounts were queried within a time window. This number is zero for accounts that never query together and 1 for accounts that are always queried together. Of course accounts can be queried together by blind chance or by a third party site that queries lot of APIs (like killboards). However I believe that alt accounts will have a significantly higher coincidence number than random accounts, allowing detection of alts. The best thing is that this can be used on gathered old data, finding alts that were not found before, even if they stopped linking behavior long ago.

A quick answer to the post of Shadoo who is thinking about wargames to replace the "horrible" Sov grind. I was there with him on Sov grinding three regions and didn't find it horrible, because I'm one of the "few who've not actually ever ran or been in the FC side of running a large block war." The problem isn't Sov grind. The problem is that it waits for the same veterans who did it zillion times and don't want the Sov at the first place. Pandemic Legion which is leaded by Shadoo has 19 system Sov, practically staging systems in various points of space. The Sov they grind won't be theirs. They don't need it. They don't want it. Making someone grind down hundreds of multi-million HP stationary structures they don't want at the first place with no fight (besides bombers killing their fighter-bombers) is indeed horrible.

But the solution isn't simpler Sov grind. Without Sov grind, a weaker alliance would have no chance to rally troops, or find allies before losing everything. HBC would conquer all nullsec overnight just because they can. The solution is making people want to have Sov. Making them wanting to live there. A Sov war would be much more fun for PL if the new owners of the Sov would grind it in dreads, PL would only be cynoed in if someone shows up with force threatening the dreads. But why would anyone want Sov or even moons? Just buy a battleship V pilot and run lvl 4s in highsec. I'm damn sure that the man hours needed to capture and hold a even a Tech moon would be better spent missioning.

The solution is making living in nullsec profitable enough that "carebears" flow to it, grind it down and pay to PvP-ers to protect them. As long as nullsec Sov is nothing but bragging right - or even worse, to trigger a "good fight" - only a handful of bored nerds would try to get it who simply have nothing better to do.


Anonymous said...

Whos to say team security isn't already using API details to determine who the bots are?

If Sreegs is to be beleived their methods are highly effective with a vanishingly low false positive rate and a large pile of nuked botter and RMTer accounts to their credit.

Anonymous said...

Nullsec itself is profitable enough to support one's humble living.
Otherwise, people wouldn't be bothering themselves living in Curse, Syndicate or Stain.
If you'd ever travel there (and I barely believe you will) you might have found it pretty populated.
The problem barely lies within profits, it lies within people unwilling to commit to the fights, avoiding risks at any cost.
Same shit with nullbears: they simply dock up upon when neutral appears in local, they warp their minefleets to pos and remain there until the intruder disappears.

Anonymous said...

Makes no sense, because

a) X is lazy, so doesn't use proxy
b) X is not lazy, so uses proxy and also set of extra tools. (VMWare)

Of course there may be an odd occurence where you are right, but I doubt it.

On the other hand I'm pretty sure something like this is done already, as even some corps do similar things on their forums/jabber/irc and check API keys, and CCP monitors API usage anyways.

So in the end: Nothing to see here, move along.

maxim said...

I don't think CCP would have any trouble catching lazy botters, should it choose to.

Against non-lazy botters, the whole thing will just escalate into a security cold war, which will be a net loss for everyone.

Sugar Kyle said...

It seems that your null sec ideas of paying PvPrs has cone full circle.

Anonymous said...

funny fact: i got access to API keys of most of my hostiles.

now if ccp would ban 'em for fetching info through api on the time as a botter i'd LOVE that feature.

all i would need todo was to get a bot, and get caught on purpose (maybe reporting myself), and alot of my hostiles would have a problem.

this would get funny.

Hivemind said...

Setting aside whether a paranoid botter who's using VMware and proxies to avoid detection by IP and hardware bans would run all his accounts through a single EVEMon client, I'm not sure your suggestion would work from a technical perspective. The API is queried once an hour, that is once every 3,600 seconds. There are something like 500k+ active EVE accounts, and I don't think it's a stretch to imagine that half of those are linked to EVEMon or a similar client. That means on average every second 69-70 accounts' APIs will be simultaneously queried and those same 69-70 accounts will continue to be queried simultaneously so long as their instances of EVEMon continue running. In cases where players have computers running for a significant length of time that can rack up a very high concurrence without any actual links between the accounts.

The other problem I can see from checking EVEMon for my own accounts is that there’s a difference of 0-10 seconds between the query timer for each account, even though all the accounts were on the program when I started it. Checking account APIs for concurrence based on simultaneous queries would not tie together all my accounts, even though I've done nothing to obfuscate them. Presumably looking for concurrence at the millisecond level rather than per second to avoid false-positives would further reduce concurrences for actual alt accounts.

On the wargames/sov grind side of your post, grinding sov for PL (which I can verify having been on the receiving end of it) consists of deploying SBUs and keeping a cloaked cyno alt in system, dropping a counter fleet if SBUs are threatened. When the SBUs are online they drop in Titans and DD the station, TCU or Ihub straight into reinforcement. Complaints about the grind are less about shooting things specifically at and more about needing to keep a counter-hotdrop fleet active and ready to jump in to support any attack, protect the relevant SBUs, meet the various reinforcement timers (which will usually be set for inconvenient times) and do so repeatedly across multiple regions.

While in theory you can try and draft nullbear renters to do the shooting, in practice most of them will not have the ships/skills to bring capitals (at best they may have carriers but dreads are unlikely) and they will take longer than the more powerful alliance/coalition would do with supercaps, meaning more time that the alliance/coalition has to keep their support fleet ready to jump in and save the renters. Meanwhile all the logistics and organisation that is the main complaint will still have to be handled by the alliance/coalition as it's likely to be beyond the scope of the renters-to-be. It’s also worth noting that shifting an unenjoyable gameplay experience on to another group of players by making them grind multi-million HP structures isn’t the same as making it an enjoyable experience.

That being said, your continued assertion that nullsec isn't worth living in is still demonstrably wrong; I've pointed out that ISK/hour in nullsec is noticeably higher than the same activity in highsec and that the activities tend to offer quality-of-life benefits over their hisec equivalents (larger asteroids in null so less movement or need to switch targets, no need to return to station and hand in missions between anomalies etc). As for them actually being perceived as desirable I believe that the HBC gets some of its income from renting space to nullbears who would otherwise be in hisec, the N3 coalition get most of their income from their rental programmes and even PL are getting involved now, renting out systems they've taken in Malpais via the Brothers of Tangra alliance. For rental programs to be this widespread implies that they are profitable, them being profitable implies that there is demand for the space from non-sov-null entities.

Gevlon said...

@Hivemind: the API cashing happens once an hour (the transfer of data from game server to API server), the API query is instant.

Anonymous said...

Gevlon, that isn't quite how the API works.

The data is cached after you pull the information. There is a slight delay of a few minutes but for arguments sake, take the locations api query. Stick a thing that in your hanger then wait a minute then query the API - it will be there.

Also, in many cases the API doesn't obey its own cache timers - it simply tells YOU to cache for it (where you are the person writing the app to consume the API). They will of course ban you if you thrash the ever living crap out of it but on many of the calls you can get fresh data outside of cacheuntil values.

The API really doesn't work the way it says so on the tin. After you work with it for a while you get to understand its nuances. 2 people querying the same query for the same data (api key/vcode) at slightly different times can and will get wildly different information on a number of the calls.

Sorry if this is a bit off topic, but it is worth the clarification I think.

Tharre said...

It's indeed technically possible. However the problem is that you could quite easily create false positives, and that's definitely not something you want to do. Players are going to exploit everything they can.
And the other problem is that it's not a long-term solution: The moment this becomes public every botter will just avoid to access multiple APIs with the same IP.

Hivemind said...

@ Gevlon

I am not a programmer so I don't know the nuances of the API's inner workings, but I assume that you would be looking for coincidence between when the API cache is accessed across multiple accounts. EVEMon (and, I assume, similar programs like EVEHQ) make a new query every hour per account, whether that goes to the cache or the API server. You're still looking at concurrence in the times those queries are made across multiple accounts to detect a pattern.

The problems are still: 1) That the API server/cache presumably gets many queries every second which will keep on showing concurrence as the programs behind those queries keep querying the same accounts at the same time every hour, until one of the programs is closed down. 2) That it seems like EVEMon (and presumably other API-using programs) actually has some drift away from simultaneously querying all accounts on a single client, likely in response to the server/cache taking more or less time to respond to each API call that offsets the API call for each account by a little time, meaning that alt accounts would no longer get concurrent queries.

Anonymous said...

I hope they don't rely too much on IP address as several people in an apartment and hundreds in a dorm could have the same IP.

I think time is a big problem for Sov. You can't really make things too much simpler or there is the gb2w/my little pony comments. Many forum posters like the slow, inconvenient travel times. OTOH, when competing products like LoL or WoT (or console games, including DUST) allow you to jump on, play for 30 minutes, kill some people, and log off. This is exacerbated by the fact that the "killers" are less likely than explorers or PvE to want to endure several hours of mundane.

Since almost all of the null residents care far more about "good fights" than who has sov, EVE is asking them to expend a lot of effort doing things they don't enjoy on behalf of something they don't particularly care about. This is not a recipe for a mass market game.

Tharre said...

@Hivemind: CCP logs (most) API accesses already. So if you find a botter the only thing you need to do is grab every IP in his access log and search for it in the hole access database. And then you need some sort of algorithm that decides if the secound entry that was found is really the main account of the bot or just fake.

Anonymous said...

You dont need to check any APi acess if you can run executable in boters computer. Eve client can consist of spy tool which could detect botting software or activity. You can even scan eve mon directory for added api keys. Which would be much more accurate than API query log. Even this surely can be countered as anything else.
My point is: Imagine what you could do if you could smuggle executable to someones computer? Answer is almost everything with his computer.