Melee Set Testing Results

[Galaxy Brain]

Hey all, just copying my Melee Tests from beta to a new thread for ease of access / more varied discussion than just 1 set!

 

---

 

(The rules of this test are better described here: )

 

 

Overview & Biases:

The above link will go into a bit more detail, but the goal of this test is to provide benchmark tests for Scrapper Primaries in a more or less neutral environment (no gimmicks, no enemy resists, etc) that aimed to emulate a standard mission experience. This means a 4-floor defeat-all mission with slight variations to mob placement, size, and level room to room as well as a mix of enemy ranks with a mandatory defeat of 2 boss spawns and a final elite boss spawn.

 

With /Willpower and no pools/epics besides Combat Jumping (for the SO test, IO's had hasten), the enemies and difficulty setting were just enough to get "Safety" as a metric as many sets have clear design intention of trading damage for utility or mitigation, or vice versa, and allowing myself to actually experience or mitigate damage I felt is a worthy point of interest. Given this is an "everyday" sort of benchmark, ignoring the impact that Ice Melee's safety provides I feel would be unjust compared to if I just made myself invincible and went  to town to track raw output. Likewise, a set like Fire Melee really showed where it can struggle if it's not constantly killing.

 

This test has less mobs than a farm, but more to deal with than a Pylon, so results may vary wildly from the expectations provided by those popular trials. Both of those I feel do not provide a full picture between the mix of AoE and ST output, favoring the extremes on either end. The office map simulation just as well does not highlight certain sets in the way you'd expect either. Something like the new Energy Melee may massacre Pylons but end up on the low/mediocre end here or vice versa with other sets.

 

I hope you find the following enlightening in some ways, and let the results speak for themselves when it comes to the changes to multiple melee sets we see here in Beta.

 

---

 

 

 

Results on SO's only:

All sets were played with a SO build with /Willpower as a secondary and no pool powers/epics except Combat Jumping, in order to isolate the primary set as much as possible.

 

(except for Claws and Kinetic Melee which were given Overwhelming KD IO's + a lvl 50 damage IO to have the same slotting + knockdown on their ranged cones. These were ran due to the incredibly common practice of slotting for KD, and only 2 melee sets actually "needing" it.)

 

 

Results without the IO in Claws can be seen here:

Spoiler

 

As you can see by some of the names, we have been testing these changes internally for quite a while now to make sure things land in the desired spot!

 

Breaking down this chart, each column represents a different aspect:

• AVG = the average clear time of the mission from all 10 runs (minutes : seconds : milliseconds)
• SWING = How far away from the Total Average the set performed
• SD Deviation = Standard Deviation from run to run, measuring how consistently each set performed despite the 10 runs having slightly different mob formations and placements
• BEST TIME = the best run time of the set, shows the max potential
• WORST TIME = the worst run time of the set, shows the minimum potential
• SAFETY = This counts the number of deaths in the 10 runs. A score of 0.50 means that I took substantial damage but never died, a 0.00 means I was practically invincible!
• SAFETY ADJ = Each point in the Safety column is multiplied by 20 seconds, and then added to the average to show an Adjusted Safety Average.

 

With those all factored in, we can rank the sets against each other on more than just the average clear time:

 

 

From left to right, we have the results with Claws KD, Claws "Pure", and the average between the two. 

 

 

 

Now, lets see how this changes when we jump to an IO build:

 

Results on a "Mid Level" IO Build:

All sets were played with a n IO build with no purples, winter sets, or "special" procs beyond a single Damage Proc like Mako's or Obliteration that *any* Melee attack could slot. 

All sets achieved 72.5% global recharge (142.5% most of the time with Hasten up, only a 20s cooldown there). 

All sets had the normal uniques like Numina, Kismet, etc slotted

All sets ran the tests at +3/x5 difficulty

 

 

With recharge and procs thrown into the mix, the top sets become a LOT closer in clear times, and safety really isn't a concern outside of Energy Melee where rapid ET's kinda hurt lol.

 

Like above, lets see how all the factors weigh in:

 

 

The margins are so close for the "top tier"  that with more than 10 tests a piece I'm sure they'd all be near dead even! 

 

 

Combined Results:

 

 

 

Edited October 27, 2020 by Galaxy Brain

[summers]

I'm very surprised to see Staff, which has an abundance of AOE, do worse than Martial Arts, which has just the one? What was your observation here?

 

Edit: Ditto for War Mace, it has very good AOE, yet somehow Martial Arts is coming #2 in "Mid Level" IO builds. This does not match up with my experience...

Edited October 27, 2020 by summers

[Sif]

Looks like most of the images aren't working, I'm only seeing the Results without the IO in Claws image.

[Safehouse]

You know, somebody always has to come last, but my boy kinetic melee... ya hate to see it with a set you love 💔

[Galaxy Brain]

1 hour ago, summers said:

I'm very surprised to see Staff, which has an abundance of AOE, do worse than Martial Arts, which has just the one? What was your observation here?

While Staff has more AoE, it has slower AoE.  Overall, I feel staff does fantastic but it lacks a way to quickly dish out damage. As a fight goes on you can build up and unleash sure, but then the animations to do so end up being a smidge lengthy which then feeds back into the building up phase.... you get the idea.

 

MA in contrast has a lightning fast PBAoE in Dragon's Tail, and the new Eagle's Claw hits like a truck to your main target and very decently well to side targets. Combined, you can wallop a close-packed crowd, kung-fu your way through some other targets, repeat, very consistently. 

 

1 hour ago, summers said:

 

Edit: Ditto for War Mace, it has very good AOE, yet somehow Martial Arts is coming #2 in "Mid Level" IO builds. This does not match up with my experience...

So, one thing to note with the average times when going by that is the actual cluster of times. Lets look at the top sets:

 

Claws	6:39:44	Difference:
Martial Arts	6:40:58	0:01:14
Titan Weapons	6:41:35	0:00:37
Spines	6:41:44	0:00:09
Savage Melee	6:43:44	0:01:59
War Mace	6:46:45	0:03:01

 

 

MA is ~1s slower on the 10 runs than Claws

TW and Spines are both milliseconds within MA's average

 

Savage is *just* under 2 seconds slower than Spines

 

WM is *just* over 3 seconds slower than Savage, and in total 7 seconds slower on average than Claws, and about 5:50 seconds slower than MA. Within that 10-second bracket is where I would honestly account for X factors such as a few attacks missed here or there, a regen tick on the Elite Boss, or even maybe 1 more boss spawned for War Mace, or there was one smaller spawn for MA which skewed results a bit. Running each ten times averages out a lot, but those things could still happen! Given they are so so close though when you highlight them as a group, I would not put as much weight on that part.

 

 

 

[Troo]

Excellent stuff.

 

Will link & poach from here.

 

Edited October 27, 2020 by Troo

[Saikochoro]

I’m very impressed with how MA has ranked up. I may be recalling incorrectly, but I thought MA was on the bottom half of the original rankings. 
 

I find it Interesting that making  eagles claw a cone boosted it so far up. I guess it was a good change. 

Edited October 27, 2020 by Saikochoro

[Lines]

Also staggered by how high MA climbed (too high?). I'm going to be giving that a go on beta next.

[Maxzero]

 

How come Dark melee is rated so high even though its clear times are average?

[Hew]

2 hours ago, Maxzero said:

 

How come Dark melee is rated so high even though its clear times are average?

Not sure what you are looking at? It clearly excels on SO slotting. On a mid-slotted IO build, it is solidly in the middle of the pack? 

 

Something (missing?) here is the rating of the sets based on safety (or that might have been included here). Safety rating affects the overall rating of the set, and factors in how much time you spend dying trying to clear a map (1 death, 2 deaths, 0 deaths). 0 deaths clearing the map bumps the safety, and overall rating of the set, up significantly. Dying drops the rating.

 

edit: for completionism, GB died A LOT running these tests. 🙂

 

Edited October 27, 2020 by Hew

[Maxzero]

4 minutes ago, Hew said:

Not sure what you are looking at? It clearly excels on SO slotting. On a mid-slotted IO build, it is solidly in the middle of the pack? 

 

Something (missing?) here is the rating of the sets based on safety (or that might have been included here). Safety rating affects the overall rating of the set, and factors in how much time you spend dying trying to clear a map (1 death, 2 deaths, 0 deaths). 0 deaths clearing the map bumps the safety, and overall rating of the set, up significantly. Dying drops the rating.

 

 

In the mid IO table Dark Melee is near the middle time wise (7.13 average for 9th place). Everything but EnM has the same suvivability score. Yet the overall IO score is 12.34 putting it third. Claws KD is over 30 seconds faster on average is almost equal in the ratings.

 

Is standard deviation rated that highly? DM melee has 21 seconds between its highest and lowest while Claws KD has 25 seconds.

Edited October 27, 2020 by Maxzero

[Bionic_Flea]

Galaxy Brain, thanks for all the time, effort, thought, and presentation you put into your charts.  It's really amazing.

[Maxzero]

 

I really question the value you are placing on std deviation.

 

Take for example Martial Arts and Dark Melee in the mid tier IO table.

 

MAs WORST result is 6.57. DMs BEST result is 7.03. Yet DM is rated above MA (albeit narrowly) because it is consistant? Consistantly worse maybe.

 

If a set is on average 30+ seconds behind on each on run just how much valube do you place on the fact that sometimes its only 10 or 20 seconds behind? Your still behind.

 

There are several sets (TW, Spines, WM, Ice, Rad) who's worse time AND best time are better then DMs worst and best but because they vary between being say 5% worse 20% of the time to 20% better 80% of the time they get a lower rating overall?

Edited October 27, 2020 by Maxzero

[Gobbledigook]

If a set is low on AoE and clear time but may have good single target for EBs/AVs etc, it is easy enough to bump that clear time by picking the right Armour or epic pool choice.

Rad armour/shield/bio etc all provide extra AoE. Ball lightning, Dark Obliteration, Fireball etc will also boost that AoE. Trash is the easiest to clear.

 

But a set with low single target dps but good AoE like Staff. How do you boost their damage? They can clear trash fast but meet an EB/AV and hit a wall.

 

If i was to roll a new Energy melee brute for example i would combine it with Bio, Rad or shield etc and Ball lightning, DO or fireball etc. Benefiting from the excellent single target damage whilst boosting its AoE/Clear time greatly.

 

Some sets are as good as they get. Others can be boosted further and further with the right choices. All sets need looking at thoroughly.

Edited October 27, 2020 by Gobbledegook

[Bill Z Bubba]

Yea.... I've loved claws for a long time. Now if only I could make a shield/claws tank.

[Galaxy Brain]

 

@Maxzero

Thank you so much for that catch! Turns out I had grabbed the incorrect screenshot for the IO portion of the tests (Earlier build....)

 

The correct results :

 

   

[csr]

18 hours ago, Galaxy Brain said:

Claws	6:39:44	Difference:
Martial Arts	6:40:58	0:01:14
Titan Weapons	6:41:35	0:00:37
Spines	6:41:44	0:00:09
Savage Melee	6:43:44	0:01:59
War Mace	6:46:45	0:03:01

 

 

MA is ~1s slower on the 10 runs than Claws

TW and Spines are both milliseconds within MA's average

 

Savage is *just* under 2 seconds slower than Spines

Ah, that second colon is supposed to be a decimal point then.

[Maxzero]

2 hours ago, Galaxy Brain said:

 

@Maxzero

Thank you so much for that catch! Turns out I had grabbed the incorrect screenshot for the IO portion of the tests (Earlier build....)

 

The correct results :

 

   

 

All good man. Just want to say I appreciate the work you have done. Would of been quite the grind.

 

One question: why is MA so good now did the new small cone make such a difference? Claws and TW I can see but MA?

Edited October 27, 2020 by Maxzero

[Galaxy Brain]

27 minutes ago, Maxzero said:

One question: why is MA so good now did the new small cone make such a difference?

Recharge. 

 

Despite being a small cone, Eagles Claw is a tremendous ST attack and even a portion of the main damage being spread out quickly + dragins tail adds up to a set that can take on crowds and bosses equally well.

[Maxzero]

14 minutes ago, Galaxy Brain said:

Recharge. 

 

Despite being a small cone, Eagles Claw is a tremendous ST attack and even a portion of the main damage being spread out quickly + dragins tail adds up to a set that can take on crowds and bosses equally well.

 

Seems strong on a set that also has Storm Kick (Tanker).

Edited October 27, 2020 by Maxzero

[Maxzero]

 

Just noticed that on BETA the power details for Eagle's Claw lists the damage bonus being only in effect under Fiery Embrace.

 

It does not show this on live. Possible bug?

Edited October 28, 2020 by Maxzero

[Lancek]

Is Street Justice not that great? 

[Hew]

11 hours ago, Maxzero said:

 

Just noticed that on BETA the power details for Eagle's Claw lists the damage bonus being only in effect under Fiery Embrace.

 

It does not show this on live. Possible bug?

It is more like splash damage. Pretty sure it wouldn't show up as a result, if it is actually just splash?

[Vanden]

4 hours ago, Hew said:

It is more like splash damage. Pretty sure it wouldn't show up as a result, if it is actually just splash?

"Splash damage" doesn't really have an actual meaning in in terms of CoH mechanics.

 

4 hours ago, Lancek said:

Is Street Justice not that great? 

It's only great on Stalkers.

[Arkterusss]

I guess my question is, if I read the old scrapper results correctly, why did Claws get such a huge bump upwards?  They didn't change anything to that set for this beta did they?