Feedback: Testing Melee Set Performance

[Hopeling]

Sure, I just don't want to get through a bunch of testing and realize half our data is garbage. If somebody thinks a certain set is being badly represented, let them show us how it's done.

 

If we're testing at 50, a similar argument applies to the choice of epic pools: a set like Claws may want to take Moonbeam or Fireball, while a set like War Mace or TW would rather have Conserve Power. Rather than making everybody take the same thing, we'll get more representative results if we let players use the one they think is best.

[Galaxy Brain]

1 minute ago, Hopeling said:

Sure, I just don't want to get through a bunch of testing and realize half our data is garbage. If somebody thinks a certain set is being badly represented, let them show us how it's done.

 

If we're testing at 50, a similar argument applies to the choice of epic pools: a set like Claws may want to take Moonbeam or Fireball, while a set like War Mace or TW would rather have Conserve Power. Rather than making everybody take the same thing, we'll get more representative results if we let players use the one they think is best.

I mean yes, but at the same time that muddles it a bit... like should they all have hasten too?

 

I think just sticking with Primary / Willpower (least invasive secondary, also the vast majority of secondaries provide a bit of end management) will give the cleanest results?

[Hopeling]

I was going to say they should all have Hasten if we're including epic pools, yes. If you prefer, I don't mind having the default test condition be "SOs only, just primary and secondary powers".

[Galaxy Brain]

36 minutes ago, Hopeling said:

I was going to say they should all have Hasten if we're including epic pools, yes. If you prefer, I don't mind having the default test condition be "SOs only, just primary and secondary powers".

All having hasten or set pool powers is doable, or if they all have the same Epic. I just want to see if we can narrow it down to just primary performance is all 🙂

[Hopeling]

Let's leave it at "Combat Jumping, but no other pool or epic powers" for now.

 

I've got an AE arc which I am in the process of testing out right now, which I think should suffice. It's 5 identical missions in a row on the "Space Island" map, which accommodates 15 enemy spawns, in roughly the distributions you laid out in your earlier post, plus one EB fight. Unless something goes wonky on this testing pass, I'll publish it within the hour, and then try to get it up on Justin too.

[Hopeling]

@Galaxy Brain, if you didn't already have a map in mind to use for testing, I just put mine up. Arc ID 15832, titled "Asteroid Test 1". It uses a custom enemy group called "Punching Bags"; I was going to give them no attacks, but that resulted in them just running away a lot, so instead they have one attack, the t1 power from Street Justice.

 

I also took my TW/WP Scrapper through it, on a second build with the conditions mentioned above: +0/x3, SOs only, no inspirations, no pools besides Combat Jumping, no epic powers, no Incarnate powers. Specifically, I used this build. I didn't turn on the Fighting toggles nor Strength of Will, just had to take something at those levels until I had enough slots.

 

My clear times were 6:31, 7:33, 6:13, 6:32, and 6:25, for an average time of 6:39. I did not do any herding, since that didn't seem to be in the spirit of the test, just fought every group where it stood. Endurance was a mild problem, but not terrible; I sometimes had to wait a few seconds to get back above 40 endurance before engaging the next group, but never went dry mid-fight. Running Fighting toggles would definitely have created more issues though. Runners were a mild problem, but that's probably just due to /WP's weak taunt aura.

Edited September 18, 2019 by Hopeling

[Galaxy Brain]

Awesome! Yeah, I had no map in mind since they are randomized... so even just saying "office map" would yield different results per run. Unless there is a specific set that can be used for an "average mission" environment?

 

Also, is this on Justin?

Edited September 18, 2019 by Galaxy Brain

[Hopeling]

I published it on Justin (well, Pineapple, since they renamed it) and also on live.

[L0ngdistance]

The way I see it level a TW to 50 and 6 slot all the powers with SOs and test it that way. Don't cry wolf on a power because people are smart enough to make it great through set bonuses and incarnates. Like someone said on page 1, that's how you get power sets like EM that become worthless because of rapid nerfing cause a group of people cry OP. And to the person with data reading TW was number 1 and had claws at the bottom clearly has no grasp on the game. Because based on activation time and recharge speed claws is near the top of the class for DPS for melee. Most of the attacks have a <2 second activation time and all the powers with hasten come back within 2-3 seconds. If the set is broken and OP on scrappers than tune the damage or recharge or whatever on scrappers. Don't break it for every single melee archetype. 

[Galaxy Brain]

4 minutes ago, L0ngdistance said:

The way I see it level a TW to 50 and 6 slot all the powers with SOs and test it that way. Don't cry wolf on a power because people are smart enough to make it great through set bonuses and incarnates. Like someone said on page 1, that's how you get power sets like EM that become worthless because of rapid nerfing cause a group of people cry OP. And to the person with data reading TW was number 1 and had claws at the bottom clearly has no grasp on the game. Because based on activation time and recharge speed claws is near the top of the class for DPS for melee. Most of the attacks have a <2 second activation time and all the powers with hasten come back within 2-3 seconds. If the set is broken and OP on scrappers than tune the damage or recharge or whatever on scrappers. Don't break it for every single melee archetype. 

Exactly, which is why we are trying to test TW vs other sets in a more realistic environment using SO's.

[Indystruck]

2 hours ago, Vayek said:

 I mean, Psionics does incredible damage at a much faster rate. 

 

People need to stop making things up. Or at least be more specific with which Psionic set you're talking about.

[Hopeling]

1 hour ago, Hopeling said:

My clear times were 6:31, 7:33, 6:13, 6:32, and 6:25

By the way, if anyone is wondering about why that second run took much longer than the rest: the mission generally spawns with a whole lot of "2 lieutenant, 5 minion" groups, a couple "4 minion, 1 boss" groups, and one EB group. On that second run, it was almost all "4 minion, 1 boss" spawns. This seems to just be random; you see it in regular missions too occasionally. On the one hand, it's nice that this measures ST damage a bit more than usual; on the other hand, since it's random, it kind of disrupts testing. If you exclude that run, the average time was 6:25.

 

An important question to address at this point is: does this methodology actually work?

 

I ran a BS/WP Scrapper through the same test on Pineapple and got clear times of 7:40, 8:27, 8:02, 8:21, and 8:02, for an average of 8:06. I did not get any all-boss maps this time. Endurance was never anywhere close to an issue. I picked Broadsword because it's a set that I'm  confident is underperforming, and I wanted to see if this methodology will detect that. It looks like the answer is yes; TW clears in about 80% of the time. In other words, it does 25% more DPS than Broadsword in this test, even accounting for travel, downtime between spawns, etc, a substantial difference. This is good news; at first I wasn't quite sure if the test was going to be sensitive enough to tell apart the performance of different sets.

 

I also ran it with my TW/WP using his regular builds with IOs and Incarnate powers, still on +0/x3, no inspirations, to see if there is any kind of ceiling effect. I got 3:49, which suggests there is not.

[Hopeling]

Oh, one more thing worth testing: is this repeatable? In other words, if someone else does the same thing I just did, will they get about 6:30 for TW and about 8:00 for Broadsword, or does it depend on fiddly little playstyle quirks? I can't test that myself for obvious reasons, but if somebody else wants to repeat a TW/WP or BS/WP run, that would do it. Or just test whatever powersets you want to compare and I'll try to duplicate a couple.

[Galaxy Brain]

21 minutes ago, Hopeling said:

Oh, one more thing worth testing: is this repeatable? In other words, if someone else does the same thing I just did, will they get about 6:30 for TW and about 8:00 for Broadsword, or does it depend on fiddly little playstyle quirks? I can't test that myself for obvious reasons, but if somebody else wants to repeat a TW/WP or BS/WP run, that would do it. Or just test whatever powersets you want to compare and I'll try to duplicate a couple.

I think it is definitely repeatable, but maybe there needs to be a set amount of tests? The arc is 5 missions, running it at least twice would give us a solid 10 mission run to get a better average.

[Indystruck]

1 minute ago, Vayek said:

apparently you havent been in PVP at all lately

PVP is an entirely different beast, yes, however the vast vast majority of this topic is talking about PVE changes, since PVP ones can be made independently, and the team has shown they are in fact willing to do so.

[Galaxy Brain]

If you read the last page since you posted first, those times are a basic mission designed to test run-through in the most "average" settings possible.

 

I really should rename the thread at this point... but in all fairness I was wrong about TW due to the metrics being used + anecdotal evidence both being questionable. There are a ton of examples of outliers and data on the super duper top end, but there is zilch on the "basic performance" of powersets, which is why Hopeling has designed a mission format that attempts to mirror regular encounters you'd see in missions. 

 

If we say, run a scrapper build on an SO level with basic slotting you'd expect from the "average player", with the only variable being the primary set (Willpower is the secondary) and compare performances over 10 missions a piece, I think we can get something more solid for how a set functions on "normal play" than on a Farm or Pylon.

Edited September 18, 2019 by Galaxy Brain

[Galaxy Brain]

1 minute ago, Vayek said:

Seriously testing with broken will power? Super reflexes would be a more fair test. also, it would be betetr to test with a power set that actually has more AOEs than broadsword. I'm actually surprised broadsword did it in 8 minutes.

1) Willpower has no effect on your primary set via Click Powers to interrupt you mid-fight, nor does it improve DPS via a direct offense power like a damage aura or a +Recharge ability.

 

2) 9/13 secondaries provide some sort of endurance management tool, making it far more common that not. WP happens to have Quick Recovery + is non invasive for testing only the primary power in our tests.

 

3) We will be looking at every melee primary, including Broadsword and even Battle Axe, to record performances against one another. I find it a bit disheartening that you would say that a test with TW is anecdotal / bad, but then say the same test with BS is not good enough to use 😞 

[Haijinx]

You can slot a couple taunt durations in RTTC for the taunt issue. 

[Galaxy Brain]

4 minutes ago, Vayek said:

because Broadsword doesnt measure up to titan weapons. in damage or even AOEs and Cones. 

Besides all of that, again, the averga play testing is showing a difference of 1 to 2 minutes. that is nothing to worry about being over powered.

That is with 2 tests, so it is not enough to call an average yet. 

 

What we're looking for is to see if there is consistent performance issues, so after multiple sets go through and we notice a pattern we'd have a talking point.

Edited September 18, 2019 by Galaxy Brain

[Infinitum]

13 minutes ago, Galaxy Brain said:

1) Willpower has no effect on your primary set via Click Powers to interrupt you mid-fight, nor does it improve DPS via a direct offense power like a damage aura or a +Recharge ability.

 

2) 9/13 secondaries provide some sort of endurance management tool, making it far more common that not. WP happens to have Quick Recovery + is non invasive for testing only the primary power in our tests.

 

3) We will be looking at every melee primary, including Broadsword and even Battle Axe, to record performances against one another. I find it a bit disheartening that you would say that a test with TW is anecdotal / bad, but then say the same test with BS is not good enough to use 😞 

To be fair though you should at least test with a non +end secondary also to see if the results are dramatically off.

[Haijinx]

Just now, Infinitum said:

To be fair though you should at least test with a non +end secondary also to see if the results are dramatically off.

WP can be made a non +end secondary with no survival impact, since quick recovery is stand alone. 

 

 

Just run the same build without it 

 

[Infinitum]

I think you should also test it on brutes and tanks to see how the various inherants skew the results fury vs crits vs nothing with tanks yet until the changes hit perhaps

[Infinitum]

Just now, Haijinx said:

WP can be made a non +end secondary with no survival impact, since quick recovery is stand alone. 

 

 

Just run the same build without it 

 

but to simulate how someone may choose something in game i dont see how running a different secondary could do anything but add to the data thats being collected - if you really are wanting the most accurate data that simulates what you would see in game thats one suggestion

[Galaxy Brain]

1 minute ago, Infinitum said:

but to simulate how someone may choose something in game i dont see how running a different secondary could do anything but add to the data thats being collected - if you really are wanting the most accurate data that simulates what you would see in game thats one suggestion

Which set then? The only other non-performance one I can think of is Invuln, but that has +ToHit (others have damage auras, or +Rech/Movement, etc)

 

Running on Brutes and Tanks could be done, but Tanks are in an odd spot and Brutes would probably need more tests run for averages due to Fury being more inconsistent than Crits... we can certainly try tho.

 

That said, I would like to keep it to one AT at a time to see what testing works before it's more repeatable.

Edited September 18, 2019 by Galaxy Brain

[Infinitum]

1 minute ago, Galaxy Brain said:

Which set then? The only other non-performance one I can think of is Invuln, but that has +ToHit (others have damage auras, or +Rech/Movement, etc)

I think invul would be fair, its one of the older sets so it would give you a good competing data set.