[SOLVED] Logs with massive deduplication

Post Reply
Batata
Posts: 2
Joined: Wed Oct 19, 2011 11:28 pm

[SOLVED] Logs with massive deduplication

Post by Batata » Wed Oct 19, 2011 11:43 pm

So, every so and then I find some logs that have the tag
"Info: Statistics are calculated after deduplication. Duplication: x.xx% global (size=xxxxxxx), max xx.xx% sliding window (size=10000). (2)"

What about those? I mean, the DPS/Heal values are most of the times inflated thus "not valid" for a ranking point of view.

I have seen a lot of those, I kinda don't remember them or they lost their importance since they are kinda new, but a recent one is: http://www.worldoflogs.com/reports/rt-t ... 999&e=4247

If you see the combat log, there are a lot of "duplicated" abilities:

Code: Select all

[20:14:29.085] Silverfoxy Obliterate Off-Hand Baleroc *18057*
[20:14:29.085] Silverfoxy Obliterate Off-Hand Baleroc *18057*

[20:14:29.539] Shantál Mind Flay Baleroc *15010*
[20:14:29.647] Shantál Mind Flay Baleroc *15010*

[20:14:29.873] Silverfoxy Frost Strike Off-Hand Baleroc 7749
[20:14:29.901] Silverfoxy Frost Strike Off-Hand Baleroc 7749

[20:14:30.206] Quenios Deep Wounds Baleroc 2844
[20:14:30.245] Quenios Deep Wounds Baleroc 2844

[20:14:30.275] Tador Arcane Blast Baleroc 33186 (he doesn't have the staff, even if he did, the staff procs are 0.5s~ delayed)
[20:14:30.275] Tador Arcane Blast Baleroc 33186 
I know some of these logs edits/deduplication aren't done for a malicious purpose, because I just have seen a LOT of them and I doubt people have that much of spare time.
But what causes logs to deduplicate? Can I "report" them?

cremor
Posts: 381
Joined: Mon Jun 15, 2009 11:06 am

Re: Logs with massive deduplication

Post by cremor » Thu Oct 20, 2011 7:48 am

Combat log lines are duplicated on the WoW servers, it's a Blizzard bug. That's the reason why in-game damage meters are affected too.
WoL removes that duplicated lines for its calculations so that the DPS/HPS numbers are correct again. At least it tries to, sometimes the deduplication doesn't work (when the combat log lines are not exactly the same after duplication). This seems to be the case in some of the examples you posted (the timestamps are slightly different), so these duplicates are not removed by WoL.

You can report invalid reports here: viewtopic.php?f=15&t=2553

lewick
Posts: 5
Joined: Mon Feb 22, 2010 7:40 am

Re: Logs with massive deduplication

Post by lewick » Tue Nov 08, 2011 11:09 pm

I have determined that it is no longer feasible or possible to actually continue posting the logs with duplication in them. There are so many of them at this point it's literally not worth the effort. I started at the top of the Baleroc 10 heroic survival hunter logs and stopped after the first five because each one of those logs had duplicate damage values with slightly skewed timestamps. I honestly don't know how the WoL folks are going to deal with this, but it's definitely getting worse.

I noticed today that the top 9 Baleroc 25 heroic parses are all hacked logs with a guild called "FIX RANKS", all from China. The 10th rank is a BM hunter doing 50k dps, with his pet doing 35k dps. None of it is real, and since so many services use WoL to try and track things now, it's painting an unclear picture of what's going on with specs in the game.

World of Logs is a great tool, but it's falling victim to poor input data now. How this gets fixed I honestly don't know, but eventually something will need to be done.

Maihem
Site Admin
Posts: 2747
Joined: Sun Apr 05, 2009 4:14 pm

Re: Logs with massive deduplication

Post by Maihem » Wed Nov 09, 2011 10:58 pm

Since our input is simply text files with data that may or may not be authentic, it's fairly hard to programmatically decide which reports are genuine and which are bogus. Up until now we've been manually removing obviously bogus reports from the rankings. I've removed the guild in question and its reports from the rankings just now. In the mean time, we'll keep thinking on other solutions, perhaps even crowdsourcing this issue by letting (certain) people "flag" reports.

Muspel
Posts: 2
Joined: Sun Nov 13, 2011 3:37 am

Re: Logs with massive deduplication

Post by Muspel » Sun Nov 13, 2011 3:40 am

lewick wrote:The 10th rank is a BM hunter doing 50k dps, with his pet doing 35k dps.
That's actually the result of some kind of pet exploit. I don't know how it's done, other than that it involves BM hunters somehow, but I've seen it done on live servers.

ArcaneMagus
Posts: 1518
Joined: Wed Aug 26, 2009 7:56 pm
Contact:

Re: Logs with massive deduplication

Post by ArcaneMagus » Sat Jan 14, 2012 3:42 am

The log in question has been flagged as do not rank (From the Invalid Reports thread).

Post Reply