Query language [draft]

Query language [draft]

Postby Miles » Wed Oct 14, 2009 3:38 am

Why didn't I start messing with ANTLR earlier? After a few days, it seems so logical now; draft of the query language, dynamic graphs variant:

Code: Select all
ID: ('a'..'z'|'A'..'Z'|'_') ('a'..'z'|'A'..'Z'|'0'..'9'|'_')*
INT :   '0'..'9'+
STRING: double quoted string with escaping for quote

query: SELECT amount FROM type WHERE where GROUP BY grouping
amount: ID
type: ID
where: andclause (OR andclause)*
andclause: (clause AND clause)*
clause: (ID '=' value) or (ID 'IN' '(' value (',' value)* ')'
grouping: ID


amount
just 'amount' for now; can add tricks like buff stack count later.

type
pre defined filters. use 'any' for unfiltered, or 'damageDone' for damage done on hostiles, 'damageTaken', 'healingDone', 'healingTaken', 'friendlyFire'

grouping
Group by source, target (mobs: choice between merged or unmerged), spellId. LF more ideas.

where
Basically a 1:1 translation of the current filterset (@ log browser) to plain text. a = b is just a shortcut for a in (b).


Can generate AST in debugger/java version, just need to generate it to a python target, pray it isn't too slow and write the handler. where -> FilterSet converter, extend servlet with a new chart data handler and hack it into a new page.

WHERE sourceName="Miles" AND spellName = "Fireball" is way better than the [{"sourceNames":["Miles"], "spellNames":["Fireball"]}] variant; can make it linkable by just pasting it behind ?q=.
Miles
Site Admin
 
Posts: 1509
Joined: Mon Mar 30, 2009 7:12 pm

Re: Query language [draft]

Postby gcbirzan » Wed Oct 14, 2009 8:30 am

Miles wrote:type
pre defined filters. use 'any' for unfiltered, or 'damageDone' for damage done on hostiles, 'damageTaken', 'healingDone', 'healingTaken', 'friendlyFire'
I know it's a retarded metric, but. Spell casts. This would probably need another filter, by class, because it doesn't really make much sense otherwise. And maybe spec too, since you have that detection in somewhere.
grouping
Group by source, target (mobs: choice between merged or unmerged), spellId. LF more ideas.

Class/spec. This might need the amount divided by the number of members of that class, to provide a comparison between classes, not between the number of classmembers. :-)
where
Basically a 1:1 translation of the current filterset (@ log browser) to plain text. a = b is just a shortcut for a in (b).

What I mentioned above about class/spec, but I'm not sure if you don't plan to have another thing like this for the log browser. :-)

WHERE sourceName="Miles" AND spellName = "Fireball" is way better than the [{"sourceNames":["Miles"], "spellNames":["Fireball"]}] variant; can make it linkable by just pasting it behind ?q=.

Personally, I don't have a problem with the latter, since I wrote my own query builder, but that's another story. :-P

P.S. You're not using INT anywhere.
gcbirzan
 
Posts: 157
Joined: Mon Apr 13, 2009 3:31 am

Re: Query language [draft]

Postby Miles » Wed Oct 14, 2009 2:00 pm

Forgot value = STRING | INT | BOOL

Hmm... Class comparison. I'll probably hack it in later, sounds complex.


amount: 'eventAmount' (read: damage done / healing done etc), count(*)
grouping: add by class, class-spec
where: add class="bla" / spec="bla"

later: add hack in browser side, just do [v/count(players with spec) for v in values]

spell cast count: SELECT count(*) FROM any WHERE eventType="SPELL_CAST" and sourceInRaid=true and sourceIsPlayer=true GROUP BY class

Still sounds like a weird metric to me :P
Miles
Site Admin
 
Posts: 1509
Joined: Mon Mar 30, 2009 7:12 pm

Re: Query language [draft]

Postby gcbirzan » Wed Oct 14, 2009 2:20 pm

Miles wrote:spell cast count: SELECT count(*) FROM any WHERE eventType="SPELL_CAST" and sourceInRaid=true and sourceIsPlayer=true GROUP BY class

Still sounds like a weird metric to me :P


Not group by class, but using the class as a filter. Also, yeah, I didn't have enough coffee at the time, so ignore that bit.
gcbirzan
 
Posts: 157
Joined: Mon Apr 13, 2009 3:31 am

Re: Query language [draft]

Postby Miles » Sat Oct 24, 2009 12:59 am

Poked a few holes in the current filter / where clause implementation... There's no way to implement show me all lines with overkill without hacks or hits over certain amount.

With only exact or IN matching, you're kinda limited in what you can or cannot do. So the idea is to rewrite the thing to a generic everything is allowed expression.

Root: returns boolean
Unary Operators: !
Binary Operators: =, !=, <, >, <=, >=, IN, AND, OR
Fields: eventAmount: int, eventType: int, sourceId: int, targetId: int, etc etc
Parentheses: ()'s modifies execution order.
Expression: (Expression Operator Expression) or Field; so you can do things like (a=b or c>d) and e<f and g IN (h,i,j)


eventAmount >= 9000 anyone? Or just readable queries instead of the current weirdish Filter/FilterSet thing.

Crap. That just added like, 20 rules to this lexer/parser experiment and 20 classes to implement the action. I hope it will be fast enough :P
Miles
Site Admin
 
Posts: 1509
Joined: Mon Mar 30, 2009 7:12 pm

Re: Query language [draft]

Postby Miles » Sun Oct 25, 2009 1:20 am

Parser v0.1 done; It only needs some better error reports. Language is kinda complex tho, but you can do everything in it.

Operators and precedence

1: OR
2: AND
3: =, !=
4: <, <=, >, >=, IN, NOT IN
5: +, -
6: *
7: NOT, (unary) +, (unary) -

Most operations require that the left and right hand side have the same type.

Fields

Fields have a type: integer, string, boolean or list. They are either a constant or a reference to a field in the current row. Examples:

1: the number 1
sourcename: name of the event source
overkill: overkill amount of the row

Reference identifiers

Note: Identifiers are case insensitive

Actors: sourceid, targetid, sourceuid, targetuid, sourcemobid, targetmobid, sourcename, targetname
ActorType: sourcetype:String, targettype, sourcereaction, targetreaction
Event type: type, subtype, fulltype (will add more info later)
Amounts: amount, absorbed, overheal, overkill
Spell: spell (name), spellid
Short cuts: damagetaken, healingdone

Actor type is a string: "Player", "Pet", "Creature", "Vehicle" or "Unknown". Detected using GUID masks.
Actor reaction is an int, calculated by flags & 0xF0 (see combat log format for more info)


Examples

You can do almost everything within this language, so have fun finding new uses. When used as a filter, the root expression type must be boolean. Some examples:

type=3 and overheal=amount: rows with 100% overhealing
overkill>0: all killing blows
x=type=3 and sourcereaction=64: healing done by enemies (hi Faction Champions 25H :P)
((targetreaction=64 and fulltype=701) or (sourcereaction=64 and type=3)) and sourcename != "Healing Stream Totem": healing done by enemies, interrupted enemies plus spam filter for healing steam totem.

(up to here works in current build, not deployed yet)


When used as amount for the (non existing) dynamic graph, the root expression must be an integer:

amount-overheal: effective healing
healingdone-damagetaken: delta graph, positive is healing catching up, negative is damage taken not healed in same tick.

(not sure if this is such a good idea, need some benchmarking first; otherwise I'll just hard code the common views; not implemented yet anyway)
Miles
Site Admin
 
Posts: 1509
Joined: Mon Mar 30, 2009 7:12 pm

Re: Query language [draft]

Postby gcbirzan » Sun Oct 25, 2009 1:19 pm

Might want to add

amounts: blocked (does the log count it as absorbed? Iirc, no, but I could be wrong) and resisted.
spell: school.
gcbirzan
 
Posts: 157
Joined: Mon Apr 13, 2009 3:31 am

Re: Query language [draft]

Postby Miles » Sun Oct 25, 2009 2:33 pm

Yeah. A/B/R are separately noted; and all 3 have a SPELL_MISS variant for full absorbs.

School is a good one, it will be in the WoW school format (some 0xFF masked iirc). Uh oh. It's a bitmask and there is no bitwise and/or operator yet, so dual school spells will be a pain to match...

Maihem is the grammar.g wizard here, go yell at him to get bitand / bitor implemented so you can do (school bitand 0x04) != 0 :P

Or screw the operator precedence, and overload the normal "and" to accept 2 ints -> boolean.
Miles
Site Admin
 
Posts: 1509
Joined: Mon Mar 30, 2009 7:12 pm

Re: Query language [draft]

Postby Miles » Wed Oct 28, 2009 2:48 am

List of constants

Code: Select all
# Short event types
TYPE_DAMAGE=1
TYPE_MISS=2
TYPE_HEAL=3
TYPE_AURA=4
TYPE_DEATH=5
TYPE_CAST=6
TYPE_DISPEL=7
TYPE_GAIN=8
TYPE_ENCHANT=9
TYPE_DURABILITY=10
TYPE_SUMMON=11
TYPE_OTHER=50

# Full Event Types
SWING_DAMAGE=101
RANGE_DAMAGE=102
SPELL_DAMAGE=103
SPELL_PERIODIC_DAMAGE=104
DAMAGE_SPLIT=105
DAMAGE_SHIELD=106
ENVIRONMENTAL_DAMAGE=107
SPELL_BUILDING_DAMAGE=108
SWING_MISSED=201
RANGE_MISSED=202
SPELL_MISSED=203
SPELL_PERIODIC_MISSED=204
DAMAGE_SHIELD_MISSED=205
SPELL_HEAL=301
SPELL_PERIODIC_HEAL=302
SPELL_BUILDING_HEAL=303
SPELL_AURA_APPLIED=401
SPELL_AURA_APPLIED_DOSE=402
SPELL_AURA_REMOVED=403
SPELL_AURA_REMOVED_DOSE=404
SPELL_EXTRA_ATTACKS=405
SPELL_AURA_REFRESH=406
PARTY_KILL=501
UNIT_DIED=502
UNIT_DESTROYED=503
SPELL_INSTAKILL=504
SPELL_CAST_START=601
SPELL_CAST_SUCCESS=602
SPELL_CAST_FAILED=603
SPELL_INTERRUPT=701
SPELL_DISPEL=702
SPELL_DISPEL_FAILED=703
SPELL_STOLEN=704
SPELL_AURA_BROKEN=705
SPELL_AURA_BROKEN_SPELL=706
SPELL_ENERGIZE=801
SPELL_DRAIN=802
SPELL_LEECH=803
SPELL_PERIODIC_ENERGIZE=804
SPELL_PERIODIC_DRAIN=805
SPELL_PERIODIC_LEECH=806
ENCHANT_APPLIED=901
ENCHANT_REMOVED=902
SPELL_DURABILITY_DAMAGE=1001
SPELL_DURABILITY_DAMAGE_ALL=1002
SPELL_SUMMON=1101
SPELL_CREATE=5001
SPELL_RESURRECT=5002

# Spell types
PHYSICAL=1
HOLY=2
FIRE=4
NATURE=8
FROST=16
SHADOW=32
ARCANE=64

# Reaction in flag
REACTION_FRIENDLY=16
REACTION_NEUTRAL=32
REACTION_HOSTILE=64
Miles
Site Admin
 
Posts: 1509
Joined: Mon Mar 30, 2009 7:12 pm

Re: Query language [draft]

Postby Zuramed » Wed Oct 28, 2009 5:07 pm

Is there a way to see if a spell is beginning to cast, didn't succeed, but was not interrupted through an interrupt ability (player moving/ getting CCed/getting stunned)?

fulltype=SPELL_CAST_FAILED yielded no results in faction champs when I tried it. I also didn't see a way to look at stuns.
Zuramed
 
Posts: 10
Joined: Tue Aug 25, 2009 6:34 pm

Next

Return to Suggestions & Feature Requests

Who is online

Users browsing this forum: No registered users and 1 guest