Galaxy Brain

That is with 2 tests, so it is not enough to call an average yet. What we're looking for is to see if there is consistent performance issues, so after multiple sets go through and we notice a pattern we'd have a talking point.

1) Willpower has no effect on your primary set via Click Powers to interrupt you mid-fight, nor does it improve DPS via a direct offense power like a damage aura or a +Recharge ability. 2) 9/13 secondaries provide some sort of endurance management tool, making it far more common that not. WP happens to have Quick Recovery + is non invasive for testing only the primary power in our tests. 3) We will be looking at every melee primary, including Broadsword and even Battle Axe, to record performances against one another. I find it a bit disheartening that you would say that a test with TW is anecdotal / bad, but then say the same test with BS is not good enough to use 😞

If you read the last page since you posted first, those times are a basic mission designed to test run-through in the most "average" settings possible. I really should rename the thread at this point... but in all fairness I was wrong about TW due to the metrics being used + anecdotal evidence both being questionable. There are a ton of examples of outliers and data on the super duper top end, but there is zilch on the "basic performance" of powersets, which is why Hopeling has designed a mission format that attempts to mirror regular encounters you'd see in missions. If we say, run a scrapper build on an SO level with basic slotting you'd expect from the "average player", with the only variable being the primary set (Willpower is the secondary) and compare performances over 10 missions a piece, I think we can get something more solid for how a set functions on "normal play" than on a Farm or Pylon.

I think it is definitely repeatable, but maybe there needs to be a set amount of tests? The arc is 5 missions, running it at least twice would give us a solid 10 mission run to get a better average.

Exactly, which is why we are trying to test TW vs other sets in a more realistic environment using SO's.

Awesome! Yeah, I had no map in mind since they are randomized... so even just saying "office map" would yield different results per run. Unless there is a specific set that can be used for an "average mission" environment? Also, is this on Justin?

All having hasten or set pool powers is doable, or if they all have the same Epic. I just want to see if we can narrow it down to just primary performance is all 🙂

I mean yes, but at the same time that muddles it a bit... like should they all have hasten too? I think just sticking with Primary / Willpower (least invasive secondary, also the vast majority of secondaries provide a bit of end management) will give the cleanest results?

@Hopeling, thats a fair point, though "basic slotting" would cover 90% of attacks 🤔

Earlier on we had settled at the most basic of "Average Joe" slotting, that being: 3 Dam, 1 Acc, 1 End, 1 Rech at either SO or even lvl common IO level. Introducing more and more IO's would be a bit too variable, unless its just adding in incredibly common uniques / etc that *anyone* could slot. With the above, it would be a better showcase of the "Base Level" of the sets more or less as intended without outside variables.

So, in preparation for a new thread (Probably titled "Practical Melee Set Testing" or such), I am grabbing data on resistances based on enemy rank and level ranges. What would be best? lvl 5 - 20, 20-40, and 40+ as brackets? Or just Pre / Post SO's?

A scythe would be so sick... Take back everything, TW is UP until this.

I think at this point we need to gather data before going forward. TW is shown to be a huge outlier in what data we have, but those metrics have become questionable... so those results are then questionable. What I can do is: 1) gather data on enemy group resists and break it down by average and per rank to see if there are super significant changes mission to mission, or spawn to spawn that will be a factor for certain sets. 2) run data vs the example mission layout I posted earlier for a +0/x3 encounter with various enemy groups that could be fair to both AoE and ST damage. This would factor in the enemy resist averages (per rank) to give an *idea* of what results may be if you optimally ran a bunch of random missions. We cannot really account for runners, or map geography, and other rando X factors. But what we can do is at least make the measuring stick more attuned to what you would expect in average gameplay. Instead of a very specific Pylon target, or vs farm map targets, it would try to emulate the gambit of enemies youd face daily. With that done, it could at least show some more practical numbers to show what sets under or over perform... without random factors at the least. Randomness could then be discussed after the baseline is figured out?

If you want numbers just on specific ranks of enemies, I can gather that info as well. I did the group averages since missions give you an assortment. Sure, a Council boss may have high lethal resist, but if the boss is a vampire and not a werewolf it may not. The Minions and LTs also need to be cleared in a mission as well and would factor into performance. Would you rather we throw the hardest targets at each set and see if it still shines? Because we can emulate that. Pylons test ST, Farms test ease of AoE. Both are biased, every test will be in some small way, the best we can do is try to accommodate a wide variety of factors to let sets shine in different ways.

You're speaking my language 😉 Having just finished development on Borderlands 2.5 a solid month before BL3's release, diving into that code you can see all the revisions and how certain things got altered with time. Sometimes, nerfs are necessary if they effect the health of a game going forward. For context, Borderlands 2 is a First Person Shooter / RPG Hybrid with a massive emphasis on loot. Your character choice determined what skills you get, which were mostly passive abilities outside of the "Action Skill", which then interacted with the various gear to make certain gear better on certain characters, and so on. In CoH terms, you can think of the gear as powers / power sets and the characters as AT's. The example here that stands out to me is the "Beehawk". In the game, there are two items called "The Bee" and "The Sand Hawk". The Bee was a shield that, while it was at 100% capacity, would add a substantial amount of damage to every shot you made with your weapons. If you took damage and the shield went to even 99% capacity the bonus would stop, but the shield's capacity was very small anyways which made skills that replenished shields or boost shields improve the Bee dramatically. The Sand Hawk was a gun that fired a cool bird-shaped projectile, sort of like a " ^v^ ", using multiple bullets to achieve the effect. It was a unique weapon in that it was sort of like an Uzi with very high fire rate, but having x9 damage due to multiple bullets. Normally, the Bee would divide it's damage with shotgun-type weapons. So if the bee gave +1000 damage, a shotgun with 10 bullets per spread would be 1000/10 = 100 per bullet. This was not the case on the Sand Hawk, as the bullets were part of a special pattern that appears after the gun is fired so each of the bullets got the full bonus. Put those together with the high fire rate and well.... things would just die before they could even damage you to stop the Bee's amplification. All characters benefited from this, some more than others, but it was still degenerate as the combo straight up invalidated other strategies. This had to be nerfed in order to maintain the goal of the game which is incredible build diversity through unique gear combinations. Buffing the rest of the game to account for the incredibly power of the BeeHawk would simply mean that anything besides that combo, or similar combos, would just be an unfun chore at best. By nerfing that outlier (via slapping the Bee Shield's attributes), the strategy became more high risk / high reward + not as potent a reward as it was prior. This allowed other strategies to become viable as they could now compete with the BeeHawk, instead of needing to BeeHawk to compete. A similar philosophy could be seen here. If a specific powerset combo is incredibly strong, then it will always be a sticking point for game balance. If new content, or general difficulty tweaks / options get introduced, then sets / combos that are very strong may need to be eyeballed to fit the goals of content shifts.

Yes, this is based on Culex's but as you said it has over a thousand rows to look through. This one is condensed by enemy group, as it is exceptionally rare to have a mission where there is only 1 enemy type from a faction throughout. It's more realistic to look at the average resists throughout a fraction to see what resists you'd most ost likely face.

I went through and mapped out the average resists of enemy groups (61 of them from a data source on Discord) to see what damage types are actually most resisted. Smashing is resisted by nearly half the game to varying degrees, but usually is is not by that much: https://docs.google.com/spreadsheets/d/1arWBGWuwGCWqSrkLgnWK26xNTrILUO_QNKn8034A3L0/edit#gid=0

While the last point is true, 9/13 scrapper secondaries have some sort of endurance tool making it far more common to mitigate the end issues than not...

I think that a lot of good has come from the discussion despite the arguing, though I do think what has come to light is that the measuring sticks need to be looked at again which could either be it's own thread.... or I could rename this one and refocus on that.

Super anecdotal, but this is not an uncommon type of post from people on social media. I have another thread buried now that talks about making the game harder via advanced difficulty options as well which would be a great approach to this from the other side, but what powerhouse said still holds up as a valid question.

Have you guys already run metrics on the sets average performances somehow?

The time between shutdown and now is essentially dead zone due to a tiny tiny tiny amount of people playing if at all. So TW was "live" for a few months as a pay to get set, then it's been live again for a few more months as a set everyone has access to.

It was never my intention to cause a massive divide over this, and I for one have been bouncing back and forth on my opinion on where a so called "problem" even lies. Is TW too good? I'm not sure if it by itself is anymore, but what we have for results now shows that it is certainly a catalyst for incredible performance when built right and with the right power pairings. The question then comes, do other sets with the same pairings match up? If anything comes from this thread, I would like there to be an understanding of what makes a set good in some quantifiable way. Using actual gameplay attributes and metrics and not niche Farm maps or Pylons, and with the most basic slots possible. If we can do that, it should be clear that at a basic level certain things will just snowball when given extra juice from IOs or buffs. Om the flip side, we can probably ID what sets totally need help and in what ways if they struggle even at a base level.

If we wanna do pure average, we should refer to that chart floating around of all the resists and apply that based on commonality/ max res

Farming is a chunk of the game, but it is hyper specific to where it isn't "normal gameplay" as intended. It's almost like the opposite of a pylon test in that sense lol.