Is the enhancement converter RNG Fair? or Does RNGesus exist?

[Ridiculous Girl]

This is something that I had asked myself as I started to carve myself a niche in enhancement crafting and selling. As a chronic altoholic, I needed a way to fund my severe altitis. Crafting is my jam, and my bread and butter. As of this moment, I have crafted 24,078 enhancements total, with 22,474 enhancements sold on the Auction House, 80-85+% of those were IO sets. (If you are curious as to how I know this, it comes from the information kiosk near the  train stations) I got interested in the random number generator (RNG) when I thought I was seeing certain trends in my converting large groups of the same enhancements at a time to another set. I could not tell if it was one of the cognitive biases, like the gambler's fallacy, that i was thinking I was seeing a pattern in the conversion results.

 

Since I was converting massive amounts of enhancements, I also wondered if there was a way to game the system. That certain sets converted better than others. So all in all, for the sake of this post, I tracked 4708 enhancement conversions, of one to another within the same category set. All of these conversions were of the "out of set" but within the "category." That is to say, conversion of one ranged damage enhancement to another ranged damage from the same category set. The nice thing is that if you stick with the same category sets, in this manner, your results are always the same.

 

I am going be deliberately vague with details, because I do not want to reveal too much of what I am doing for fear that I will lose my niche. But, I will explain as much as I can. Some enhancement sets will convert to only two or three different sets within the same category. This makes it very easy to track the odds of the conversion. So, for example, I have a ranged damage enhancement and I convert it in category, there are certain sets which convert to two or three different sets within the same category. If each of those sets have six varitions of enhancements each (ie. damage or damage/endurance, etc.) then the odds of converting to one specific type of enhancement is ONE out of SIX.

Spoiler

A level 36 Crushing Impact enhancement will convert to one of three other sets in the Melee damage category. In this case, converting an enhancement still has a one in six chance of a ACC/DAM enc, in any of the other sets.

If I convert a ranged damage (accuracy/damage) enhancement it would have a one in six change of being another ranged damage (accuracy/damage) in another set of the same category. (If you change within the set, and there are six different varieties in that set, you have a one in five chance of getting what you are hoping for. But that is another story, especially if you do not get what you were hoping for the first time. Seriously, look up the gambler's fallacy.)

 

My crafting niche was generally two completely different enhancement categories, buying in bulk recipes that I crafted and then converted regularly, to get certain resulting enhancements that I would sell on the AH. (For a decent price, of course. Seriously, you cannot sell 22K IO enhancements if you are trying to gouge players. I know the going rates very well, by now.) I will call these sets Set A and Set B, for simplicity and secrecy's sake.

 

When I first started tracking these conversions, Set A was more consistantly tracked, meaning that I did large blocks of the same recipe, around 70 at a time, and converted them wholesale. Set B was more of a mixed bag, where I would craft 70 of several varieties from the same set, 30 from Set B2 and 40 from Set B5 for example. But since they were always the same recipes, they would all convert in category to the same sets, so tracking was easy and consistant. Later, I found Set B to more lucrative and always sold much faster, so I switched to almost exclusively crafting Set B. At that point I started crafting and converting pure stacks of the same enhancement, for example 70 enhancements of Set B4 at one time.

In each of the three data groups that I collected, I would note one particular enhancement type (ie. ranged damage (acc/dam)) of the resulting conversions to track the odds of each conversion.

 

NOTE: I know there are a lot smarter people than me, especially with statistics, if I made any errors please tell me! My statistics professor in university was totally boring! 😄

 

Nitty Gritty details:
Best group conversion: 20 out of 70 (11.666 is 1 out of 6 odds for a stack of 70)
Worst group conversion: 4 out of 70

Set A (1-6):
    

Spoiler

Set A1:
    Set A1 12/50
    Set A1 6/69
    Set A1 16/70
    Set A1 9/66
    Set A1 11/70
    Set A1 4/50
    Set A1 10/70
    Set A1 5/30
    Set A1 Subotal: 73/475 (0.15368 vs 0.166666)

 

Set A2:
    Set A2 9/70
    Set A2 8/70
    Set A2 10/69
    Set A2 12/60
    Set A2 11/70
    Set A2 5/40
    Set A2 Subtotals: 55/379 (0.14511)

 

Set A3:  
    Set A3 15/69
    Set A3 13/70
    Set A3 12/70
    Set A3 14/60
    Set A3 Subtotals: 54/269 (0.20074)

 

Set A4:   
    Set A4 20/69
    Set A4 12/69
    Set A4 13/69
    Set A4 12/70
    Set A4 14/70
    Set A4 9/63
    Set A4 Subtotals: 80/410 (0.19512)

 

Set A5:
    Set A5 12/68
    Set A5 2/30
    Set A5 Subtotals 14/98 (0.14285)

 

Set A6:  
    Set A6 12/70
    Set A6 4/69     
    Set A6 18/70
    Set A6 8/70
    Set A6 Subtotals: 42/279 (0.15053)

 

Set A (1-6) combined totals: 318/1910 (0.16649 vs 0.166666)

 

Set B Mixed:

Spoiler

Set B mixed 12/70
Set B mixed 1/17
Set B mixed 8/68
Set B Mixed 7/48
Set B Mixed 8/48
Set B Mixed 10/79
Set B Mixed 16/88
Set B Mixed 12/57
Set B mixed 17/72
Set B Mixed 12/89
Set B mixed 6/70
Set B mixed 3/19
Set B Mixed 11/70
Set B Mixed 12/65
Set B Mixed 11/63
Set B Mixed 12/70
Set B Mixed 8/54
Set B Mixed 10/68
Set B Mixed 12/63
Set B Mixed 21/69

 

Set B (mixed totals): 188/1178 (0.15959)

Set B (1-6):
    

Spoiler

Set B1 8/30
    Set B1 13/70
    Set B1 14/70
    Set B1 7/70
    Set B1 Subtotals: 42/240 (0.175)

 

    Set B2 8/40
    Set B2 9/70
    Set B2 11/70
    Set B2 13/70
    Set B2 Subtotals: 41/250 (0.164)

 

    Set B3 3/30    
    Set B3 7/60    
    Set B3 13/70
    Set B3 7/70
    Set B3 10/70
    Set B3 Subtotals: 40/300 (0.13333)

 

    Set B4 8/70
    Set B4 12/70
    Set B4 13/70
    Set B4 13/70
    Set B4 Subtotals: 46/280 (0.16428)

 

    Set B5 10/70
    Set B5 16/70
    Set B5 6/30
    Set B5 14/70
    Set B5 Subtotals: 46/240 (0.19166)

 

    Set B6 20/100
    Set B6 15/70
    Set B6 9/70    
    Set B6 13/70
    Set B6 Subtotals: 57/310 (0.18387)

 

Total Set B (1-6)(pure groups): 272/1620 (0.167901 vs 0.16666)

 

Combined Totals:
 

Spoiler

Totals for Sets B (mixed and pure): 460/2798 (0.16440)
Totals for pure sets (Set A and Set B): 590/3530 (0.16713)
Totals for all sets: Set A and Sets B (mixed and pure) 778 out of 4708 converted (0.16525 vs O.16666 for 1/6 odds)

Results:
Set A.

Spoiler

Set A (1-6) combined totals: 318/1910 (0.16649 vs 0.166666)

There is a mixed bag of results from within each individual subSet, conversions from A2/A5 being the worst and A3/A4 overperforming, when all added together Sets A converts almost exactly to one in six odds.
 
Conclusion: The RNG is perfectly fair!

Set B (pure).

Spoiler

Set B (1-6)(pure groups): 272/1620 (0.167901 vs 0.16666)

Out of 1620 conversions, with a one in six odds, I should have had 270 of the correct enhancements. I was off by 2, which is a difference of less that 1%. Which is close enough in my book!

Conclusion: The RNG is perfectly fair!

Set B (Mixed).

Spoiler

Set B (mixed totals): 188/1178 (0.15959)

Out of 1178 conversions I should have had about 196 enhancements to met the one in six odds, but I was off by a margin of eight, which is a 4.25% error, which I guess is an acceptable margin. After a thousand conversions, it should have narrowed down more, but I think it is probaby still acceptable. This was why I started doing this all, because when I was doing mixed groups, I thought I was getting better results than when I was doing pure groups. But 4% difference would not really be that noticible, it was probably more of the cognitive bias kicking in. It is also a smaller set of conversions, so that error may just be statistical noise. I would probably need to do at least another 500 to 1000 conversions to see if that were true or not.

Conclusion: The RNG seems to be pretty darn close to fair! But... more work needs to be done.

Set A and Set B (pure) combined.

Spoiler

Set A and Set B: 590/3530 (0.16713)

Out of a total of 3580 conversions, I should have had 588 of the right type of enhancements, off by 2 enhancements! Which is a really super close.  

Conclusion: The RNG is perfectly fair!

Set A and Sets B (mixed and pure) combined.

Spoiler

Set A and Sets B (mixed and pure): 778 out of 4708 converted (0.16525 vs O.16666 for 1/6 odds)

As a mixture of all the data points, I should have had 785 of the correct type from the almost 5000 enhancements converted. I was under by 7 which works out to be less than 1% of a difference. That mixed set messed up the odds a bit, and for the moment I am not sure how I feel about it. But still with a less that 1% variation, the odds work out to be pretty close to one in six again.

Conclusion: The RNG is perfectly fair!

 

Ultimate Conclusion!:

Spoiler

With close to 5000 conversions tracked, the statistical odds of conversion were pretty much exactly as they should be. That is to say the the RNG is a reasonable approximation of rolling the dice. You are going to have lucky streaks and unlucky streaks, but over time the odds all converge to where they should be. Those good and bad rolls will over time even out, wearing your lucky shirt and converting on odd rainy days will make no differnce in the end when you are converting enhancements. The RNG is fair, but in the end the house always wins!

 

TL;DR version:

Spoiler

The RNG is fair.

 

 

BUT! (for the DEVs eyes only!)
 

Spoiler

Would the devs please contact me. In the more granular details of specific conversions of Set B, there was a significant variation of distribution of results, and I cannot tell if there is a bug or it was built this way and it skews the results deliberately.

 

[Cancrusher]

I don't convert very often.  When I do, the results vary.  Sometimes I get my Panacea (or whatever) on the first try.  Other times it may take me a dozen or more tries.

 

No idea whether the process is truly random or not.  I actually prefer just to buy the stuff I want, because converting and (even more so) crafting is boring.

[Ridiculous Girl]

7 minutes ago, Cancrusher said:

I don't convert very often.  When I do, the results vary.  Sometimes I get my Panacea (or whatever) on the first try.  Other times it may take me a dozen or more tries.

 

No idea whether the process is truly random or not.  I actually prefer just to buy the stuff I want, because converting and (even more so) crafting is boring.

that's the thing... it is random, and it is fair. so you may get the thing you want on the first try. or you may get, like me this morning, after 15 conversions, sometimes getting the exact same enhancement that you started with several times, before getting the one i wanted. it's random, sometimes it works in our favour, other times not at all. but if you do it often enough, it will work as it should. it's just very frustrating sometimes. 🙂

[Cancrusher]

I sidestep the issue by shucking out influence, so it's not a problem for me.  🤷‍♂️

[aethereal]

The kinds of biases that people imagine in the RNG would actually be really hard to put in.  We've seen what happened when the original devs wanted to bias the random number generator: they developed streakbreaker.  They didn't build subtle hidden biases into the RNG.  Like, why would you?  What would the point be?

 

(And CoH uses for the basis of its random number generator the random seed of the operating system, and besides that, basic straightforward methods of turning the random seed into numbers in a particular range.  There is very little possibility that there is any meaningful bias in it that the devs didn't intentionally build in.)

[Ridiculous Girl]

1 minute ago, aethereal said:

The kinds of biases that people imagine in the RNG would actually be really hard to put in.  We've seen what happened when the original devs wanted to bias the random number generator: they developed streakbreaker.  They didn't build subtle hidden biases into the RNG.  Like, why would you?  What would the point be?

 

(And CoH uses for the basis of its random number generator the random seed of the operating system, and besides that, basic straightforward methods of turning the random seed into numbers in a particular range.  There is very little possibility that there is any meaningful bias in it that the devs didn't intentionally build in.)

yeah, you are right. there is no reason, absolutely none.  but there is always the sneaking suspicion that when you try to convert to a proc, that it's working against you. but what i was really wondering was where those biases were, if they existed, and if i was seeing any or not. that is why i figured why not track a couple of thousand conversions and see if anything interesting pops. 🙂

[TheZag]

You probably need to do 50 to 100 thousand conversions to eliminate the majority of anomalies and determine if there was an actual deviation from random.  But even a random number generator is only as good as it was programmed and will repeat itself eventually.  One of the zelda games has its random numbers tracked and it turns out they repeat every 45 ish minutes after a few million random number calls.  Missing by 4% after 1178 conversions would be within the margin of error.  Even if it was showing the RNG is skewed,  having to convert nearly 1200 times to come out ahead by 8 is hardly exploitable.

 

This is great work though even if it took me a few minutes to figure out how your data tables were being presented.  Since there were no names i was wondering what the tables were intending us to see,  but i figured it out.

[lemming]

1 minute ago, TheZag said:

But even a random number generator is only as good as it was programmed and will repeat itself eventually.

So much this.

 

I suspect that there is something somewhat screwy with how the RNG is implemented in the game.   I'm betting on some oddness with making a pool of random numbers every so often and pulling from them with the pool being smaller than the amount of requests from various places in the game.   (Total theory, probably wrong)

I usually look at hit chances & miss chances and noting anomalies like missing three 95% chances in a row more often than a 1/8000 chance should be.  Pretty sure that's why streakbreaker was done to kludge around stuff like that.  Which makes me think the RNG was done by some clever programmer that implemented a neat way of doing something that turns out not to scale that great, but breaks horribly when other programmers poke at it.

Took over a programming project with this odd block of code that had the comment:

"I have no idea why this exists, or works, but many things break if you touch it"

That project was shut down in 2016, and that block of code still haunts me.

[Ridiculous Girl]

10 minutes ago, TheZag said:

Missing by 4% after 1178 conversions would be within the margin of error.  Even if it was showing the RNG is skewed,  having to convert nearly 1200 times to come out ahead by 8 is hardly exploitable.

 

This is great work though even if it took me a few minutes to figure out how your data tables were being presented.  Since there were no names i was wondering what the tables were intending us to see,  but i figured it out.

i know i was being vague. i wanted that way, because i do decent business to suport my altoholism, and did not want to give away too much. I figure it was random statistical noise that gave me that 4% "rror. unfortunately i do not have too much experience producing readable data. nor the tools. i'm not even using a spreadsheet, just text. 😄

[aethereal]

1 hour ago, lemming said:

So much this.

 

I suspect that there is something somewhat screwy with how the RNG is implemented in the game.   I'm betting on some oddness with making a pool of random numbers every so often and pulling from them with the pool being smaller than the amount of requests from various places in the game.

 

No.  It just calls urandom to get a random number.  The CoH devs did not roll their own random number generator.

[Yomo Kimyata]

The programming does not seem to skew the RNG, nor do any empirical tests seem to indicate that the RNG is anything but pseudo-random.

[Snarky]

It is not reallllllly random.  But over a VERY LARGE sample set it should come to about 1/5 chance. to convert in set to another of same set (of 6).  Calculate your math odds before beginning for other conversion dds.  I will not do your math, but I bet if you combine all your sample sets you would find the number start to converge to the predicted ratio.  Double your sample, moreso....

Edited March 27, 2022 by Snarky

[Without_Pause]

I think I read somewhere someone was trying to figure out the odds for something and admitted that there can be roughly a 3% variance in the testing. So even getting a 47/53 disparity would be within reason even when someone would be going, "See, it favors the one side."

[Ura Hero]

This topic has been beaten to death over in the Market section.  Multiple analysis have proven that the RNG falls into statistical limits; IE typical bell curve.  Small sample sets show sample bias.  Large sets show it is WAI. 

[aethereal]

50 minutes ago, Krimson said:

It's when you have to use 100+ Converters on three Enhancements that sews the seed of doubt. 😂 

For whatever it's worth, it's "sow" the seed of doubt.  Sew is what you do with needle and thread, sow is what you do with seeds.

[tidge]

2 hours ago, Ura Hero said:

This topic has been beaten to death over in the Market section.  Multiple analysis have proven that the RNG falls into statistical limits; IE typical bell curve.  Small sample sets show sample bias.  Large sets show it is WAI. 

Your point stands (no evidence of bias) but it is more like a pseudorandom even distribution, no opportunity for a normal distribution AFAIK.

[Xeres]

The RNG in this game has less R than I'd like.  Convert enough and you'll see patterns.

[Ura Hero]

15 hours ago, Krimson said:

It's when you have to use 100+ Converters on three Enhancements that sews the seed of doubt. 😂 

 

If that didn't happen I would be suspicious.  Anyone who has ever studied randomness knows that for every time you Roll a D20 and get a 1 on the first try, it's just as likely that you will take thirty rolls to get a 1 again.  RNG gods can be cruel, but they are fair.

[UltraAlt]

16 hours ago, Without_Pause said:

I think I read somewhere someone was trying to figure out the odds for something and admitted that there can be roughly a 3% variance in the testing. So even getting a 47/53 disparity would be within reason even when someone would be going, "See, it favors the one side."

 

Fortune always favors the bold.

The foolhardy on the other hand ....

[srmalloy]

22 hours ago, Ridiculous Girl said:

(If you change within the set, and there are six different varieties in that set, you have a one in five chance of getting what you are hoping for. But that is another story, especially if you do not get what you were hoping for the first time. Seriously, look up the gambler's fallacy.)

It's the same math that worked against bomber crews in WWII -- if you have a 95% chance of making it back on every mission, it compounded until you only had a 28% chance of making it through a 25-mission tour to get rotated home. The chance of succeeding is (or at least should be, with an 'honest' RNG) stochastic, which means that each try is independent -- it doesn't matter how many times you failed before, your chances on the next try won't change. The coin doesn't know that it came up heads ten times in a row; the next throw still has a 50% chance of coming up heads.

 

For the people who don't want to look at the math, the number that's important is the cumulative chance of not getting what you want in N tries -- the first time, you have an 80% chance of failure, failing twice is 80% of 80%, and so on; you still have about a one in three chance of not having gotten the one you want by five tries, and to reach 90% success rate takes ten. This doesn't mean that it will take that many tries -- you could get it on the first try -- but that 90% of the time, it will take no more than ten tries.

[iBot]

These sorts of human issues are why lotteries and casinos exist.  I will caveat by saying I've never looked at the CoH code.  However, I can say that given my experiences with coding in general there is nothing that would lead me to suspect that the CoH RNG is nothing more than: num = rand () % 100.  That simple line is all it takes to generate a series of random numbers based on whatever seed the system is set up to use.  And because that line of code has no memory it will always be independent for the purposes of probability.  It is the humans that add that memory and swear on all that they hold sacred that some nefarious force is skewing the RNG against them.

 

Another funny thing that always pops in these sorts of posts is that it is the negative that we howl about.  No one ever says "WOW I won that 4 times in a row.  The RNG must be coded wrong!"  More proof that it is the human code base that is skewed in that we tend to remember more vividly the failures rather than the successes and the failures are attributed to an external force rather than our own brains.  It is why game development is like 10% idea, code, and graphics and 110% psychology.  (and yes I know that doesn't make sense but neither do humans 🤪)

[SwitchFade]

20 hours ago, Ridiculous Girl said:

i know i was being vague. i wanted that way, because i do decent business to suport my altoholism, and did not want to give away too much. I figure it was random statistical noise that gave me that 4% "rror. unfortunately i do not have too much experience producing readable data. nor the tools. i'm not even using a spreadsheet, just text. 😄

Everyone knows all of the ways to market, at this point. I make about 100million every 20 minutes when I bother, so, there's really no stone unturned. And I regularly tell people how to do this in game.

[srmalloy]

1 hour ago, iBot said:

However, I can say that given my experiences with coding in general there is nothing that would lead me to suspect that the CoH RNG is nothing more than: num = rand () % 100. 

I'm pretty sure it's more complicated than that, but only to remove the modulo-induced bias (which gets smaller the larger the range you generate. For example, when generating 0-9 with a raw modulo algorithm, only the last four bits are important, but the 0-15 of those four bits produce 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5 -- 0-5 appear twice as often as 6-9. 1-100 requires 7 bits, 0-127, so a raw modulo function returns 0-27 twice and the other numbers once. 0-999 requires nine bits, 0-1023, doubling for 0-27, a much smaller part of the range. A simple algorithm to correct this is to check the returned value and reroll if it is outside the goal range, and there are a number of well-known approaches to the problem.

[PLVRIZR]

Cheap Doctored Wounds recipe/ingredients to enhancement, converted to another heal, was my bread-and-butter, back in the day, for just starting out toons.  Once I got a Num or Miracle or PrevMed (usually only a roll or two), it was probably an average of four in-set conversions, to get the proc.  Yes, sometimes my gut told me it took 10-12 rolls to get it, but hardly blinked when it popped, on the first click.  IIRC, 19 was the most I ever counted, to get a 1 in 6 chance... and, on that example, I still made 4M on a 1.5M investment.  Whether the RNG is perfect or not, is inconsequential to me.  Billions are out there just waiting to be collected...

[iBot]

4 hours ago, srmalloy said:

I'm pretty sure it's more complicated than that, but only to remove the modulo-induced bias (which gets smaller the larger the range you generate. For example, when generating 0-9 with a raw modulo algorithm, only the last four bits are important, but the 0-15 of those four bits produce 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5 -- 0-5 appear twice as often as 6-9. 1-100 requires 7 bits, 0-127, so a raw modulo function returns 0-27 twice and the other numbers once. 0-999 requires nine bits, 0-1023, doubling for 0-27, a much smaller part of the range. A simple algorithm to correct this is to check the returned value and reroll if it is outside the goal range, and there are a number of well-known approaches to the problem.

Well as I said I've not seen the code and yes my example was simplistic and really only meant to show that generating random numbers isn't a seriously complicated problem that requires hundreds of lines of code and specialized knowledge of the inner workings of the universe.  So long as the seed used is changed or tied to something that doesn't repeat for small samples the built in rand function is about as good as it gets.