Recipe Drops are screwed passed 99

[Bionic_Flea]

Threads drop like rain in the Amazon, so I doubt they are severely restricted, if restricted at all.  Shards, however, I could believe.  But I know that a good run on an ITF can generate 4 - 9 shards pretty consistently, probably because there are a ton of EBs in there.

[Faultline]

On 6/28/2021 at 1:04 PM, thunderforce said:

I am absolutely not saying this has happened, or is likely, but surely it is theoretically possible that something on the rewards side looks at what badges you have got and happens to do something inappropriate if you have the level 99 vet badge?

 

(Again, I am absolutely not saying this happened. I expect the answer, if Faultline indulges my curiousity, is that the reward side doesn't look at what badges you have got at all, or in very limited ways).

 

The reward side doesn't look at what badges you have got at all. Rewards can have requirements that go through a general evaluator which can check badges, the enemy group you defeated (Vanguard Merits use both), what powers you have (Windfall uses this), whether you are in a zone or mission map (Tour Guide tips), whether the map is Arena or Architect (used to disable things like Candy Canes in those) and a lot of other things.

 

But the rewards code just hands off that list of requirements to the evaluator (which is just a line of text, like "VanguardEnabled owned? onMissionMap? && Loot_Booster ownPower? &&"), it doesn't look at anything directly, and the evaluator doesn't look at badges unless the requirements passed explicitly ask for it. No rewards check for any veteran badges.

 

The same evaluator is used *everywhere* for many things, so any bug on it would be instantly and very obvious.

[Arbegla]

10 minutes ago, Faultline said:

 

The reward side doesn't look at what badges you have got at all. Rewards can have requirements that go through a general evaluator which can check badges, the enemy group you defeated (Vanguard Merits use both), what powers you have (Windfall uses this), whether you are in a zone or mission map (Tour Guide tips), whether the map is Arena or Architect (used to disable things like Candy Canes in those) and a lot of other things. But the rewards code just hands off that list of requirements to the evaluator, it doesn't look at anything directly, and the evaluator doesn't look at badges unless the requirements passed explicitly asks for it.

 

so.. something like this?

 

Evaluator:

if (badge == true)

{

  reward(badge);

}

 

and

reward(Obj Badge)

{

  Grant_Reward(Badge);

}

 

Repeat?

[Faultline]

No, that is too simplistic a view because it assumes the code is looking at badges directly, which is isn't. The requirements are a string which is basically a list of functions and parameters. For example, if the requirements are "onMissionMap? VanguardEnabled owned? &&", the evaluator code would:

 

- Run the "onMissionMap?" function and get a result of 1 or 0.

- Run the "owned?" function with the "VanguardEnabled" parameter and return 1 or 0.

- AND both values together to get a final result of 1 or 0.

 

The "owned?" function is the one that would check for a badge, but that's not something the evaluator knows, it just goes through the list of functions and gets results. And more critically, if the Requires text doesn't have any calls to "owned?", then the code that checks for a badge is never called at all for that evaluation.

[Arbegla]

Ah, makes sense to me.

[Faultline]

Here's a real-life example with the Enhacement Catalysts, since they were mentioned before. Every time you defeat an enemy of Boss rank, it triggers the Boss reward table (simple enough). That table has a lot of different rewards with various chances, this is one of them:

 

Chance 3 Everyone
{
    Requires level char> 50 == EnhancementCatalystToken TokenOwned? ! now EnhancementCatalystToken TokenTime> - 72000 > || &&
    DropGroup 100
    {
        ItemSetName PL.EnhancementCatalystDrop, RT.EnhancementCatalystToken
    }
}

 

The first line means that this specific reward has a 3% chance to trigger for everyone on the team. The second line is the actual requirements string. This is using what is known as Reverse Polish Notation, which will look bizarre if you never saw it before; it's a type of evaluator that is very simple to implement for a computer, and used in scientific calculators.

 

The rewads code sees that this reward has a Requires string (if it doesn't have one, it always triggers) and ships it to the evaluator, asking the evaluator, "hey, what is the result of this gibberish". The evaluator then does the following:

 

- Call the char> function with the level parameter. Let's say this returns 50.

- Call the TokenOwned? function with the EnhancementCatalystToken parameter. Let's say this returns true.

- Call the function now (no parameters) to get the current time (in seconds). At the time I'm writing this, it would be 678483161.

- Call the function TokenTime> with the parameter EnhancementCatalystToken to get the time (in seconds) that the token was awarded. Let's say it returns 678480000.

 

At this point, we've translated the Requires to look like this:

 

50 50 == true ! 678483161 678480000 - 72000 > || &&

 

- Call the == (equals) function with the parameters 50 and 50. This returns true.

- Call the ! (negate) function with the parameter true. This returns false.

- Call the - (subtract) function with the parametes 678483161 and 678480000. This returns 3161.

 

I'm now going to show the evaluator after every step since seeing all the functions piled up at the end often confuses people:

 

true false 3161 72000 > || &&

 

- Call the  > (greater than) function with the parameters 3161 and 72000. This returns false.

 

true false false || &&

 

- Call the || (OR) function with the parameters false and false. This returns false.

 

true false &&

 

- Call the && (AND) function with the paramters true and false. This returns false.

 

The evaluator is now done and tells the rewards code "the requirements evaluated to false". The rewards code goes "thanks bro" and doesn't give the reward.

 

What failed here? You were level 50, but you had a token marking that you recently received a catalyst, and the timestamp says you received it 3161 seconds ago -- which is less than the 72000 seconds allowed between drops (20 hours). The rewards code didn't look at your character level, didn't look at your tokens, didn't look at the current time; and the evaluator didn't either, it just called functions that did. This means that a bug in the evaluator would affect any functions it calls, because it doesn't know the difference (so no way it's just the badges one) and a bug in an evaluator function would only affect that one function (so no way for a reward that isn't checking for badges to be affected by the badges owned function).

 

I "solved" the above function a bit out of order to make it more legible, but RPN always works left-to-right;  Here's the same example, but assume that you don't have the EnhancementCatalystToken (and because of this, TokenTime> returns 0), in the way that it would actually be processed:

 

level char> 50 == EnhancementCatalystToken TokenOwned? ! now EnhancementCatalystToken TokenTime> - 72000 > || &&

50 50 == EnhancementCatalystToken TokenOwned? ! now EnhancementCatalystToken TokenTime> - 72000 > || &&

true EnhancementCatalystToken TokenOwned? ! now EnhancementCatalystToken TokenTime> - 72000 > || &&

true false ! now EnhancementCatalystToken TokenTime> - 72000 > || &&

true true now EnhancementCatalystToken TokenTime> - 72000 > || &&

true true 678483161 EnhancementCatalystToken TokenTime> - 72000 > || &&

true true 678483161 0 - 72000 > || &&

true true 678483161 72000 > || &&

true true true || &&

true true &&

true

 

The evaluator returns "the requirements evaluated to true", the rewards code goes "thanks bro" and gives the reward. The reward in this case is EnhancementCatalystDrop (which drops the actual catalyst) and EnhancementCatalystToken (which gives the token mentioned above, to check it the next time the reward triggers).

[Bionic_Flea]

Thanks, bro.

[Arbegla]

Huh, RPN seems very processor intensive, as it doesn't seem like you can short circuit it. But that's neat, and I appreciate you breaking down the calls that way for us.

[Faultline]

2 hours ago, Arbegla said:

Huh, RPN seems very processor intensive, as it doesn't seem like you can short circuit it. But that's neat, and I appreciate you breaking down the calls that way for us.

 

The opposite! It's actually very simple, because it does the parameters in order, left-to-right; no need to check for order of operations or to use parenthesis. It just needs a stack:

 

- If the next token is a value, PUSH it to the stack.

- If the next token is a function, POP parameters (as many as the function needs) then PUSH the result

 

At the end of the evaluation, the stack should have the final value, and only the final value - if more than one value is left in the stack (or you can't POP enough arguments for a function at any time), the requested evaluation was malformed.

 

If you never coded in assembly or have the concept of a stack it will seem odd, but this is super natural for a computer. You could easily write a RPN evaluator in assembly for any system with a stack. The performance gains from this simple parsing outweight short-circuiting gains on anything but the most insanely complex expressions.

 

[Arbegla]

But you still have to wait for the stack to clear before you can get your result, so you're stuck right? can't really short circuit a stack, unless you have 'Ignore everything under me' calls I guess. The PUSH and POP functions make it O(1) so that's pretty neat once all the evaluations are done. Just gotta get through all the evaluations.

[Faultline]

21 minutes ago, Arbegla said:

But you still have to wait for the stack to clear before you can get your result, so you're stuck right? can't really short circuit a stack, unless you have 'Ignore everything under me' calls I guess. The PUSH and POP functions make it O(1) so that's pretty neat once all the evaluations are done. Just gotta get through all the evaluations.

 

But think about all the processing power you'd have to waste parsing the expression in the first place. The equivalent expression in more legible code would be char(level) == 50 && (!TokenOwned(EnhancementCatalystToken) || ((now() - TokenTime(EnhancementCatalystToken)) > 72000)) and just parsing that at run-time would use more power than the few "wasted" calls. RPN means it just executes the function calls one after another, without having to first figure out what it should be executing first.

 

Think about it, what uses more processing power: fetching the current time, subtracting 0 from it, ORing and then ANDing it; or lots of string manipulation to find parenthesis and checking order of operations when there's not enough parenthesis? String manipulation may seem trivial nowadays, but try writing an evaluator for it in assembly and you'll quickly be singing the praises of RPN. MOV, SUB, OR and AND are a single processor instruction each. String parsing needs hundreds or thousands of instructions.

 

In other words: you spent ~1000 instructions parsing in order to decide that you don't need to run ~50 while performing checks. Sure, it's better than running all ~1050, but with RPN you just evaluate the whole thing using ~100. When functions are this simple (nothing the evaluator looks at takes more than 10 lines of code) it's just more efficient to let it run.

[Arbegla]

8 minutes ago, Faultline said:

 

But think about all the processing power you'd have to waste parsing the expression in the first place. The equivalent expression in more legible code would be char(level) == 50 && (!TokenOwned(EnhancementCatalystToken) || ((now() - TokenTime(EnhancementCatalystToken)) > 72000)) and just parsing that at run-time would use more power than the few "wasted" calls. RPN means it just executes the function calls one after another, without having to first figure out what it should be executing first.

 

Think about it, what uses more processing power: fetching the current time, subtracting 0 from it, ORing and then ANDing it; or lots of string manipulation to find parenthesis and checking order of operations when there's not enough parenthesis? String manipulation may seem trivial nowadays, but try writing an evaluator for it in assembly and you'll quickly be singing the praises of RPN.

 

But if you can short circuit it, you can cut all of the other calls out, and exit early.

For example, if char(level) = 35, the code would evaluate 35 == 50, return false, see the && and then just exit (because && returns true iff both sides are true, so having one side, that is easy to evaluate return false, you can ignore everything else)

 

From what you described of RPN, even if char(level) = 35, you would still have to evaluate everything else before exiting the stack. Even though the answer will still be false after you're done (due to the && operator)

[Faultline]

Just now, Arbegla said:

 

But if you can short circuit it, you can cut all of the other calls out, and exit early.

For example, if char(level) = 35, the code would evaluate 35 == 50, return false, see the && and then just exit (because && returns true iff both sides are true, so having one side, that is easy to evaluate return false, you can ignore everything else)

 

From what you described of RPN, even if char(level) = 35, you would still have to evaluate everything else before exiting the stack. Even though the answer will still be false after you're done (due to the && operator)

 

Again: you are not taking into account the expense of parsing the text string in the first place. Remember this is done at run-time -- there is no compiler translating the string into nice checks. Everything you said is true when you're writing code to be compiled, because then all the string parsing is done at compile time. But when the string parsing is done at run-time, the extra processing expense is much, much higher.

[Arbegla]

Ah, the run-time processing, vs having a compiler clean it up makes the difference. From my understanding, compilers (and really any regular language parser) uses a stack, so I was looking at it through that lens.

 

But you're right, doing the evaluating at run-time is much faster, assuming you don't get stuck in recursive loops (i/e, POP something that PUSHES the same something onto the stack, which POPs, then PUSHES the same thing so you never actually change states)

[krj12]

Just curious what the logic is for Cimerora - seems like the Devs back on live went on a crusade against farming, and completely stopped IO drops in that zone.

I used to enjoy farming the wall before that happened.

 

[Faultline]

4 minutes ago, krj12 said:

Just curious what the logic is for Cimerora - seems like the Devs back on live went on a crusade against farming, and completely stopped IO drops in that zone.

I used to enjoy farming the wall before that happened.

 

Invention recipe drops don't check for villaingroups at all. Invention salvage drops do, and Cimerorans are on the Magic group alongside a bunch of other groups. I'm not seeing anything that would result in them having less drops than any other group. Need more information on where the perceived lack of IO drops is.

[Ruin Mage]

What in the world is this thread.

[Luminara]

Just now, Shadeknight said:

What in the world is this thread.

 

There are a handful of savant-level geniuses who have isolated a bug that no-one else can replicate, because the rest of us aren't that smart, and that can't actually exist because the code isn't structured in such a way that would allow it, according to @Faultbro, and everyone is just "gaslighting" when they disagree.

 

Or, a shorter explanation: comedy gold.

[Hardship]

1 hour ago, Shadeknight said:

What in the world is this thread.

It is a peak behind the curtain and getting to see how things work...

[Piecemeal]

RPN. We meet again.

 

Anyone who wants to understand comedy gold should have been a fly on the wall when @Faultline taught me. Then retaught me. Then taught me again months later.

[Tahliah]

 

5 hours ago, Shadeknight said:

What in the world is this thread.

This thread is crazy town. If devs say the suggested/suspected level 99 recipe choke can't happen, it can't happen.  The end. 

[thunderforce]

5 hours ago, Tahliah said:

This thread is crazy town. If devs say the suggested/suspected level 99 recipe choke can't happen, it can't happen.  The end. 

I wouldn't go that far! I, and I'm sure many of us, have sat at the keyboard contemplating the following:

1. such-and-such can't happen, I am absolutely sure of it
2. nevertheless, such-and-such has happened.

I think we're mostly missing #2 here.

[Crysis]

On 6/26/2021 at 12:49 AM, Bill Z Bubba said:

Hi. I'm on the other side of the equation and often get far more drops than I would expect including purples and pvp recipes. And my main is only at 700andsomethin vetlvls.

 

This thread absolutely sent me down a WTFBBQ spiral of insanity so... ummm... thanks? I mean, chaos is good and all but wow. There is NO drop rate reduction past vetlvl 99. I have at least 5 characters past that threshold and have seen nothing of the sort.

Honestly the only reason to keep my AFK farmers going past lvl 99 is for the recipe drops. 
 

I can assure that at least for AE missions I’m def still earning recipe drops past vet level 99.

[Bill Z Bubba]

15 hours ago, krj12 said:

Just curious what the logic is for Cimerora - seems like the Devs back on live went on a crusade against farming, and completely stopped IO drops in that zone.

I used to enjoy farming the wall before that happened.

 

I'm guessing you never solo the ITF. I've never had a problem getting drops in Cimerora.

[Bionic_Flea]

I've got 99 badges but recipes ain't one.

                                  -Super JZ