The Homecoming of HeroStats

[_NOPE_]

Hmmm.... wasn't someone, I think it was either Number Six or Cipher, saying that they wanted to just put this type of functionality in-game anyways? Regardless, since this looks like something that's wanted, if it's okay with you Bob, I'd like to take a gander at your source code and the game's code and see what I might be able to do, since my other projects are effectively on hold due to lack of resources...

[ineffablebob]

 

https://sourceforge.net/projects/herostats/files/HeroStats - Stable/v7.6.1.0/

 

Version 7.6.1.0

Bug Fixes:
 - Fixed chat log reading functionality. You can enable it in Hero Options, and it will work with all the game client builds, but only a subset of HeroStats functionality will work.

 

18 hours ago, Unknown Magi said:

Can HeroStats leverage saved chat log files in order to parse out the requisite data regardless of client?

We can! I'd pretty much forgotten about that functionality, but it already existed from back in the pre-shutdown days. It needed some updating, of course, thus the release above.

 

17 hours ago, DocLiah said:

Perhaps it might be possible for the devs to dump more data to the chat log file?

If we want all the same functions as the direct-read method, we'd need that. Just to list a few of the very basic things, the chat log doesn't say which character is logged in, or what powersets they have, or what SG they belong to. 

 

[ineffablebob]

58 minutes ago, The Philotic Knight said:

Hmmm.... wasn't someone, I think it was either Number Six or Cipher, saying that they wanted to just put this type of functionality in-game anyways? Regardless, since this looks like something that's wanted, if it's okay with you Bob, I'd like to take a gander at your source code and the game's code and see what I might be able to do, since my other projects are effectively on hold due to lack of resources...

Yep, here it is: 

As for the source code, it's all on SourceForge.

 

Warning: technical stuff! If you want to look at the scanner part, it's in the CityOfHeroes project. Probably want to start with MemlocFinder (FindChatList) and CohMemoryScanner (ScanHeroData and ScanTabs). My current major problem with the new client is that I can't find string pointers in memory. The process works like this: a new client is released, and I need to update the scanner. I open up a memory debugger attached to the game client, and go find "HSData" (the name of my chat tab) in the client memory - say it's at 0x00e8ab34. In the old client, I could then look for "34 ab e8 00" in memory and that tells me where the chat tab name pointers are stored (and by extension, the larger chat tab info structure). That lets me adjust the code so it can find that stuff on its own. But in the new client, that search for "34 ab e8 00" returns nothing (or occasionally garbage). I've tried it with a bunch of different types of data - chat messages, character name, etc - and that same problem pops up every time. There's probably a simple solution, but it's certainly not one I know!

 

 

 

[Number Six]

1 hour ago, The Philotic Knight said:

Hmmm.... wasn't someone, I think it was either Number Six or Cipher, saying that they wanted to just put this type of functionality in-game anyways? Regardless, since this looks like something that's wanted, if it's okay with you Bob, I'd like to take a gander at your source code and the game's code and see what I might be able to do, since my other projects are effectively on hold due to lack of resources...

It was me, and it's still on my list, but the 64-bit porting project was a much higher priority and required the majority of our development resources. There are still some lingering issues that need to be fixed, but once those are taken care of this is something I want to look at doing.

 

The most likely form this will take is a listening socket on localhost with a tiny embedded HTTP server and a REST API for pulling stuff like character information in JSON form -- basically the same thing we already do internally for our server status API. We have the pieces in place to put it together so it's not a huge amount of work on our side, and there are a million libraries that add-on authors can leverage to talk to that.

 

Example flowchart:

1. Addon connects to something like http://127.0.0.1:[port]/api/, POSTs some json with stuff like the name and version
2. Player gets in-game prompt that an addon wants to access their character info, can accept or deny
3. Addon gets a token to add to the Authorization header of future requests
4. Addon can query URLs like http://127.0.0.1:[port]/api/character/info to get basic character info, maybe stuff about powers and build, etc
5. Addon can optionally upgrade the connection to a websocket in order to get a real time feed of combat logs, power recharge status, etc. (may require additional consent prompt if it wants chat messages)

[Number Six]

18 hours ago, ineffablebob said:

The bad news is that I have no idea how to get HeroStats working with the other builds, either 32 or 64 bit mode. I don't know what "based on modern development tools" in the patch notes means but whatever it is, it completely changed how the internal client memory structures look. I can't find the combat log messages that HeroStats relies on any more. If any developers want to clue me in, I'm happy to listen, but for now the other builds simply will not work.

It'll be a bit problematic. The information should all still be there, but the new compiler is much more aggressive when it comes to optimization, including eliminating unnecessary variables and re-ordering things for cache locality. So the memory layout may not exactly match what's in the code.

[_NOPE_]

4 minutes ago, Number Six said:

It was me, and it's still on my list, but the 64-bit porting project was a much higher priority and required the majority of our development resources. There are still some lingering issues that need to be fixed, but once those are taken care of this is something I want to look at doing.

 

The most likely form this will take is a listening socket on localhost with a tiny embedded HTTP server and a REST API for pulling stuff like character information in JSON form -- basically the same thing we already do internally for our server status API. We have the pieces in place to put it together so it's not a huge amount of work on our side, and there are a million libraries that add-on authors can leverage to talk to that.

 

Example flowchart:

1. Addon connects to something like http://127.0.0.1:[port]/api/, POSTs some json with stuff like the name and version
2. Player gets in-game prompt that an addon wants to access their character info, can accept or deny
3. Addon gets a token to add to the Authorization header of future requests
4. Addon can query URLs like http://127.0.0.1:[port]/api/character/info to get basic character info, maybe stuff about powers and build, etc
5. Addon can optionally upgrade the connection to a websocket in order to get a real time feed of combat logs, power recharge status, etc. (may require additional consent prompt if it wants chat messages)

I would love you forever if you made a restful API that allowed us developers to poll the game and servers for information.... character data, auction house.... it would be AMAZING, and it would open up SO many possibilities to not have to deal with the friggin' text parser and those custom insane C structs... ugh....

[Bopper]

Currently I can't get HeroStats to work on either Live or Beta. Any recommendations?

[ineffablebob]

I've pretty much given up updating the scanner because of the change in development - see the last dozen or so posts above. You can use the file log option if you like, but that's a pretty minimal set of functionality.

[Bopper]

7 hours ago, ineffablebob said:

I've pretty much given up updating the scanner because of the change in development - see the last dozen or so posts above. You can use the file log option if you like, but that's a pretty minimal set of functionality.

I don't blame you, I guess I needed to just get confirmation that it's dead. I'll save my chat logs and just code up a parser. Thanks for the great work.

[adarict]

On 8/23/2019 at 10:06 AM, Number Six said:

It was me, and it's still on my list, but the 64-bit porting project was a much higher priority and required the majority of our development resources. There are still some lingering issues that need to be fixed, but once those are taken care of this is something I want to look at doing.

 

The most likely form this will take is a listening socket on localhost with a tiny embedded HTTP server and a REST API for pulling stuff like character information in JSON form -- basically the same thing we already do internally for our server status API. We have the pieces in place to put it together so it's not a huge amount of work on our side, and there are a million libraries that add-on authors can leverage to talk to that.

 

Example flowchart:

1. Addon connects to something like http://127.0.0.1:[port]/api/, POSTs some json with stuff like the name and version
2. Player gets in-game prompt that an addon wants to access their character info, can accept or deny
3. Addon gets a token to add to the Authorization header of future requests
4. Addon can query URLs like http://127.0.0.1:[port]/api/character/info to get basic character info, maybe stuff about powers and build, etc
5. Addon can optionally upgrade the connection to a websocket in order to get a real time feed of combat logs, power recharge status, etc. (may require additional consent prompt if it wants chat messages)

Would this mean that something could be added for Badge tracking, rather than someone having to set up a web site like the old badgehunter site?  The spreadsheet is useful, but a pain compared to being able to just run a bind and hit a key  to cycle through everything.

[_NOPE_]

Well techincially, this program could be rewritten to keep track of badges... 

[adarict]

On 8/28/2019 at 5:56 PM, The Philotic Knight said:

Well techincially, this program could be rewritten to keep track of badges... 

True, but considering the effort he is already having to put in for the changes being pushed out, I didn't want to put any pressure for more new stuff in his program.  🙂

 

[DocLiah]

On 8/28/2019 at 10:35 PM, adarict said:

Would this mean that something could be added for Badge tracking, rather than someone having to set up a web site like the old badgehunter site?  The spreadsheet is useful, but a pain compared to being able to just run a bind and hit a key  to cycle through everything.

I wrote a small program to read in badges from chat logs and show lists of badges needed.

Could upload that somewhere if wanted... made it for my own use so it might be a bit undocumented.

 

Edited September 7, 2019 by DocLiah

[Ura Hero]

I would love that.  I just did my badges manually into a spreadsheet and it took me two hours.  

[MachineGunDude]

My Hero stats works using beta, but somehow it got borked, as it will not let me change my hero options. It keeps erring out with this error

 

Can you help with this? I tried uninstalling and reinstalling, but that did not fix it. It still collects data, but it will not show all that I want. Just the basic.

[ineffablebob]

Try shutting down HeroStats then deleting the two configurations files - HeroConfigurations.xml and HeroStatsConfig.xml. It will recreate them when you restart. But be aware that I've not been updating the game scanner (see messages above for why) so it may still not work.

[MachineGunDude]

That worked. I will just have to redo the settings, no biggie. Thanks. I was just really missing the inspirations timers the most.

 

[Swiftmage]

So does this not work anymore or am I just dumb? I hadn't used it yet, but wanted to track my INF.   I'm on an SS/Fire brute and it doesn't seem to register the character at all.  Name server origin etc are all blank.  Powersets are Unknown/Unknown and HP and Endurance are both 1.00/1.00

 

I've tried closing the client/hero stats and re-opening, tried making the HSData tab, tried deleting the two files above but nothing works.

[DocLiah]

7 hours ago, Swiftmage said:

So does this not work anymore or am I just dumb? I hadn't used it yet, but wanted to track my INF.   I'm on an SS/Fire brute and it doesn't seem to register the character at all.  Name server origin etc are all blank.  Powersets are Unknown/Unknown and HP and Endurance are both 1.00/1.00

 

I've tried closing the client/hero stats and re-opening, tried making the HSData tab, tried deleting the two files above but nothing works.

It wont work with the new builds of the game.

[Swiftmage]

3 hours ago, DocLiah said:

It wont work with the new builds of the game.

Crap.  Ok thanks

[Unknown Magi]

Herostats can still work, if you utilize chat log file parsing rather than direct memory scanning.  Just make sure you're chat logging in CoH, and adjust your settings in HeroStats to use that method and point it to he correct folder for the logs.

[KaizenSoze]

On 9/25/2019 at 11:54 AM, Unknown Magi said:

Herostats can still work, if you utilize chat log file parsing rather than direct memory scanning.  Just make sure you're chat logging in CoH, and adjust your settings in HeroStats to use that method and point it to he correct folder for the logs.

Thank you for pointing this out. I set the log file location, but nothing improved. Are there other settings I need to change?

[RoseClaymore]

On 9/25/2019 at 9:54 AM, Unknown Magi said:

Herostats can still work, if you utilize chat log file parsing rather than direct memory scanning.  Just make sure you're chat logging in CoH, and adjust your settings in HeroStats to use that method and point it to he correct folder for the logs.

How do I find the correct folder for logs? I assumed it was the \logs\game folder in the install directory, but nothing seems to be actually getting logged there.

[OldLostAlt]

You have to enable the in-game setting to write to a log file. It's in Options/Windows/Chat/Log Chat.

 

For me the logs folder is under the (tequila) install folder, in a folder called "<LogonName>/logs".

 

[RoseClaymore]

On 9/28/2019 at 8:45 PM, OldLostAlt said:

You have to enable the in-game setting to write to a log file. It's in Options/Windows/Chat/Log Chat.

 

For me the logs folder is under the (tequila) install folder, in a folder called "<LogonName>/logs".

 

Thank you! Found it.