To hit roll may be bugged

[Audax]

I'm pretty sure to hit rolls are higher than average. My main toon is a high level Grav/Kin Controller, which means that hitting things is key to his survivability, and I've made sure his final to hit is clamped at 95%*, even against high level mobs. Thus, I should only be missing 1 out of 20 times, but I'm seeing about twice that against all mobs at all different levels.

 

Additionally, I have a low level Stalker and have been seeing the same thing there, so I think this is a universal problem and is not limited to hero/villain or a specific class or power set. However, I don't have the time to check every possible permutation. Can anyone else confirm if they are seeing this problem?

 

* For the record, I know how to calculate the final to hit using Accuracy and ToHit mods, and the numbers in the combat log check out. The problem seems to be with the raw to hit roll.

Edited February 14, 2021 by Audax

[SeraphimKensai]

Best way to check hop in an AE mission with a mob.that can't kill you and you put a power with a to hit check on auto that won't kill it and go to sleep. Check the combat logs the following morning.

[SwitchFade]

1. Perform said actions 500-1,000 times

2. Log data into excel

3. Perform analysis

4. Post results here

 

My assumption is, it's fine. Be sure to account for debuffs, streak breaker and buffs.

[Audax]

1 hour ago, SwitchFade said:

1. Perform said actions 500-1,000 times

2. Log data into excel

3. Perform analysis

4. Post results here

 

My assumption is, it's fine. Be sure to account for debuffs, streak breaker and buffs.

I've been tracking it, but wanted to find out if other people have seen this before investing the time. After all, one person doing something 500-1000 times is a rather small data set. If other people are seeing this, we can pool our research to see if this really is a universal issue or not. Want to help me prove/disprove the theory?

[Audax]

4 hours ago, SeraphimKensai said:

Best way to check hop in an AE mission with a mob.that can't kill you and you put a power with a to hit check on auto that won't kill it and go to sleep. Check the combat logs the following morning.

Interesting suggestion. Won't the combat logs eventually get purged, though?

[Bopper]

4 hours ago, Audax said:

I've been tracking it, but wanted to find out if other people have seen this before investing the time. After all, one person doing something 500-1000 times is a rather small data set. If other people are seeing this, we can pool our research to see if this really is a universal issue or not. Want to help me prove/disprove the theory?

Proof in comment section that there is no bias.

 

Edited February 14, 2021 by Bopper

[SwitchFade]

500-1000 is actually a large enough dataset to be statistically valid.

 

Go to AE, make sure you have 95% and no buffs or debuffs are present and auto-queue a ST attack.

 

Let it run for 1,000 attacks. Plot.

[Audax]

21 hours ago, Bopper said:

Proof in comment section that there is no bias.

 

Thank you, @Bopper! This is exactly the info I was looking for and I'm happy to see that I'm not the only person who thought RNG may have been biased. If I had an inspiration to give you, I would.

[Audax]

21 hours ago, SwitchFade said:

500-1000 is actually a large enough dataset to be statistically valid.

That dataset is only valid if everyone ran the same hero and powersets and on the same hardware/software config as me, but Bopper already linked me to the info I was looking for. Thanks for your advice here. Much appreciated!

[SwitchFade]

36 minutes ago, Audax said:

That dataset is only valid if everyone ran the same hero and powersets and on the same hardware/software config as me, but Bopper already linked me to the info I was looking for. Thanks for your advice here. Much appreciated!

Which validates that a sample set of that size is valid. Additionally, I included instructions on how to eliminate error and bias.

 

Please post your results here 😄

[wjrasmussen]

On 2/14/2021 at 9:10 AM, Audax said:

I've been tracking it, but wanted to find out if other people have seen this before investing the time. After all, one person doing something 500-1000 times is a rather small data set. If other people are seeing this, we can pool our research to see if this really is a universal issue or not. Want to help me prove/disprove the theory?

If other people have, then you don't need to invest time.  If you want to make your case, you have to do the work.

[Audax]

19 hours ago, SwitchFade said:

Which validates that a sample set of that size is valid. Additionally, I included instructions on how to eliminate error and bias.

If I was trying to verify an issue with my corner of the game, you are correct. If the problem is universal, a broader dataset is required. Fortunately, other people have already done that, so I can compare my results to theirs to disprove my theory. Thanks again for your input and helping point me in the right direction!

Edited February 16, 2021 by Audax

[Audax]

12 hours ago, wjrasmussen said:

If other people have, then you don't need to invest time.  If you want to make your case, you have to do the work.

You are absolutely correct, which is why I asked the question. I was hoping other people had noticed this, which it turned out they did. I'm just late to the party.

[SwitchFade]

2 hours ago, Audax said:

If I was trying to verify an issue with my corner of the game, you are correct. If the problem is universal, a broader dataset is required. Fortunately, other people have already done that, so I can compare my results to theirs to disprove my theory. Thanks again for your input and helping point me in the right direction!

Tohit isn't a corner of the game, it's universal. Please let us know when you have your sample and are ready to post it and your analysis. Be sure to include potential error factors, and your method of calculation. If you choose to convert to z score, please also include the unconverted data. Until then, tohit is functioning as normally as =RAND will allow, being that a computer can't actually choose a random number, it's fake.

[SwitchFade]

2 hours ago, Audax said:

You are absolutely correct, which is why I asked the question. I was hoping other people had noticed this, which it turned out they did. I'm just late to the party.

To be fair, your OP had pretty substantial observational bias, which prompted an ask for actual sample set, yet to be disclosed. I look forward to your data, I would love to find out that buried in the code the OG devs made a math/typo and we've all been to-hitting bugs 😀

 

Also, your simple postulation that you should only miss 1 out of 20 is a bit of an oversimplification, it's more like 1.7 out of 20. On the average. Because statistics, 'n stuff 🤪

Edited February 16, 2021 by SwitchFade

[wjrasmussen]

3 hours ago, Audax said:

You are absolutely correct, which is why I asked the question. I was hoping other people had noticed this, which it turned out they did. I'm just late to the party.

Then here is a song for you that I hope will apply to our situation in City of Heroes.

 

[tidge]

I'll make my usual complaint: I'd like to have Streakbreaker report the toHit roll for the forced hits.

 

While I completely agree that the RNG returns uniform (a.k.a. "flat") distribution of p between 0 < p < 1, I think Streakbreaker is forcing more misses at  ToHit = 95% than 1-in-20... but it is actually very hard to tell (because of the 'hiding' of a roll that is forced to hit by Streakbreaker. For consecutive toHit chances of p > 90, Streakbreaker assumes that after a first miss that the second roll will also be a miss and force a hit (throwing away whatever the roll actually was).

 

The simplest way to explain my thinking is this: Most RNGs "fill up" the available result space in a uniform manner... so a second roll of p > 0.90 is only 10%, but Straekbreaker acts as if this possibility is 100%... and then resets the streakbreaker count, such that when the next 'unlucky' roll occurs ( p > 0.9) you will see it, and thus the final percentage of actual hits (in aggregate) is likely to be below 90%. This is exaggerated when the final ToHit is p = 0.95, because chance of the second roll being also p  > 0.95 is 1-in-20, but again Streakbreaker acts as if it is 1-in-1.

 

I do believe in perception bias, but I don't believe that's what is happening at the extreme end where players have the (95%) ceiling on their toHit rolls. If Streakbreaker would return the ToHit result of the 'forced' hits, we could easily see if the fraction of those hits would have missed or not. And then I'd shut up about this.

[srmalloy]

1 hour ago, tidge said:

Most RNGs "fill up" the available result space in a uniform manner...

Actually, many RNGs take a simple approach that introduces bias, but the bias is at the low end of the range. The RNG algorithm computes a number, generally 32, 64, or 128 bits, and that number is distributed uniformly, but when converted to a result range, it introduces bias. Say you want to generate a 1-10 result; this is commonly done by generating a random number modulo 10 (getting 0-9) and adding 1. For this, only the low four bits of the number is relevant, but there are sixteen values -- 0:0, 1:1, 2:2, 3:3, 4:4, 5:5, 6:6, 7:7, 8:8, 9:9, 10:0, 11:1, 12:2, 13:3, 14:4, and 15:5. So the returned values have 1-6 appearing twice as often as 7-10. The bigger the range, and the closer that range is to a power of 2, the less effect this has -- a 1-1000 range is very close to 1024, for example.

 

From the graph above, though, that isn't what is being seen here, so the original devs were careful to avoid the easy pitfalls of random number generation.

[tidge]

Oh, I don't want to get into the specific details of the RNG, as I accept its behavior. It's Streakbreaker that is my bete noire.

 

I think that Streakbreaker was introduced not because of any 'defect' in RNG, but to try to address a perception bias. Until I can make a direct assessment of the rolls that Streakbreaker is actually tossing... it sure looks like characters that have a 95% chance to hit are missing more than 1-in-20 times.

 

Without seeing the actual rolls tossed aside by Streakbreaker, It's hard to set up a controlled experiement to test this... if there was a way to reliably set up multiple test targets in controlled patterns, this would help. The nearest I can come to approaching this from a different direction is:

• set up a character with no toggles that run 'to hit checks'
• avoid including %procs in powers (this should not strictly be necessary, I suggest it just to avoid more RNG calls)
• avoid DoT (and debuffs) <- again, not strictly necessary, but I want to avoid having enemies run away
• Have a ceiling 95% chance to hit

The one place where I've found I can get lots of enemies grouped in rather confined areas to experiment with AoEs and have a reliable 95% toHit is Tunnels of the Trolls (even the Sewer Network has too many walls and large rooms for my taste... but the explosives have to be watched for). The 'backwards' way of testing what is going on is to count the number of times Streakbreaker fires versus the number of rolls made against valid targets. I believe that if it all things are 'fair', Streakbreaker should be called only for 5% of all attack rolls... but my experience was that I saw it called much more frequently than that.

[UberGuy]

The streak breaker kicks in at 90%, not 95.

 

If you really have 95% change to hit against every target, then you should be missing no more than 1 in 40 times, since the streakbreaker will reject every other miss.

That's not correct. It will only kick in every time you miss two or more times in a row. I can do the math on that after work. 😛 

 

Any toHit check counts, including ones that are often invisible to players. This one has, I believe, been changed, but Tanker Gauntlet used to rely on hit rolls (taunt effects can miss AV/GM class entities) and this was messing with the streak breaker's perceived performance for Tankers.

 

Also, remember, you can't go blast a bunch of trolls a few times and get a good view of the streakbreaker's engagement from that. You have to make sure you have a good sample set. Random doesn't mean "evenly distributed", and random things can clump or spread such that short term views look very different than the long term average. You probably need to blow up several hundred, or even a thousand trolls or what have you to get a clearer view of what's really going on.

Edited February 17, 2021 by UberGuy

[UberGuy]

Try the AE. You can set up any foes you want, set them at +0 or even -1 and a fairly straightforward build should be able to cap hit chance against them. You can make yourself invulnerable in the AE's test mode, so absolutely any character should be usable.

[tidge]

I can tell you are trying to be helpful, but I don't think you read my post. For example, I specifically wrote to turn off all 'invisible' ToHit checks.

 

I am aware that Streakbreaker has the same behavior at 0.90 and 0.95, but it is

• far easier to ceiling to hit at 0.95, and since that is where MANY lvl 50+ players are,
• Streakbreaker is more likely to 'mess up' a 1-in-20 chance than it is a '1-in-10' chance.

If you go and test Streakbreaker (single target, against a practice drone) with a 95% chance to hit, you will see that whenever you miss, your next attack (assuming another 95% chance ToHit) will be forced to hit by Streakbreaker. It doesn't wait for another miss. It simply throws away the next roll (NO MATTER IF IT WOULD HAVE BEEN A HIT OR A MISS) and gives you a hit... and then it starts looking for the next miss.

[macskull]

1 hour ago, tidge said:

For example, I specifically wrote to turn off all 'invisible' ToHit checks.

There is no "invisible tohit check" to turn on. When the streakbreaker code determines the next hit is going to be forced, there's no check - it simply forces the power to hit (unless I'm misunderstanding how it works).

Edited February 17, 2021 by macskull

[UberGuy]

That's correct. I'm looking at the code.

 

* On a hit, (forced or not) the miss count is reset and the lowest toHit during the sequence of misses is set to 100.

* During the check on whether or not to force, the current hit chance is stored if lower than the current lowest value

* On an (allowed) miss, the miss count is incremented

 

The actual force check says:

* Pick the number of misses allowed by lookup based on the lowest hit chance recorded. It's broken into 10 ranges based on the "10s" digit in the hit chance.

  0,   1,    2,   3,  4,  5,  6,  7,  8,  9  // Chance 10s digit

  100, 100,  8,   6,  4,  4,  3,  3,  2,  1  // Allowed misses

* If the number of misses allowed is less than or equal to the number of misses, return "true", meaning force the hit.

 

The force check is evaluated before the actual hit check. The actual hit check says it's a hit if:

* It's an autohit power OR

* It should be forced due to misses OR

* The hit roll was successful

 

So if you have at 90%+ chance, when you miss once, on the next attack roll, the force check will be "true" and the next roll will be forced to hit.

Edit: for clarity, you don't have to actually miss a second time (1 in 400 chance at 95% hit chance) for the streak breaker to engage. With a 95% hit chance, you just have to miss once, (1 in 20 chance) and your next hit is guaranteed.

 

You can game this a little bit, since you can use it to force a hit with something that would have a really crappy hit chance. Note however that the crappy hit chance will be remembered for purposes of your next streak. If you throw in one attack with a 10% hit chance, you're allowed 100 misses in a row, no matter how high your hit chance with the rest of them, until you either manage a hit or the streak breaker saves you.

Edited February 17, 2021 by UberGuy
Brain-O

[UberGuy]

To address an earlier point, the streak breaker absolutely was introduced to address player perceptions, and has nothing to do with "correcting" any mechanical bias in the RNG. Players during beta reported desperately hating missing multiple times in a row, so the Cryptic devs threw them in a bone, introducing something that biases play towards hitting more often. We know this, because the devs told us on the old game forums.

Edited February 17, 2021 by UberGuy