The Chat Log System Needs Tweaking

[McSpazz]

A loong while back in SCORE, I was considering writing an automated parser for the chat logs. After looking into it some, I decided against it. The reason?

The current chat logs would be a MASSIVE pain to parse. I'm going to get to my suggestions in a bit, but I need some preamble to get to it.

First, here's a sample chunk of text from a chat log (provided to me by @CrystalDragon thank youuu!)
 

> 2021-01-18 11:29:39 [SuperGroup] Scions of Everlasting Message of the Day -- If you got a friend interested in the group, feel free to invite them to visit the base! Scions-3706
> 2021-01-18 11:29:41 You are flying!
> 2021-01-18 11:29:41 Welcome to City of Heroes, CrystalDragon!
> 2021-01-18 11:29:41 You may ask for help from other players by using the Help chat tab.
> 2021-01-18 11:29:41 <a href='cmd:gmotd'><b>Click Here</b></a> or type /gmotd to see the global message of the day again.
> 2021-01-18 11:29:42 Using global chat handle <@!281515951285862401>
> 2021-01-18 11:29:42 [Looking For Group] Baron Von Evil: <color #010101>Looking to JOIN a Farming Mission ..need the levels
> 2021-01-18 11:29:42 You have 2 unclaimed respecs available. Type /respec in the chat window to begin respecing your character.
> 2021-01-18 11:29:42 You don't have any available free tailor sessions.
> 2021-01-18 11:29:56 Entering Atlas Plaza.
> 2021-01-18 11:30:05 [Help] Dr. Fantasy: <color #010101>oi whhhhhyyyyyy!?!? Purples are still crazy expensive looks like.
> 2021-01-18 11:30:13 [Looking For Group] Lo-Pan: <color #000000>lvl 5 lf dfb or AE
> 2021-01-18 11:30:44 [Help] Unity Red Echo: <color #010101>yep!
> 2021-01-18 11:30:50 [Local] Darcy O'Toole: <color #010101>Anyone around that wants to help out a newbie?  I just started last night with another character, but didnt get far
> 2021-01-18 11:30:58 Shutting off Fly.
> 2021-01-18 11:30:58 You are no longer flying.
> 2021-01-18 11:31:02 [Help] Snixnix: 20m aint nothing compared to 250m
> 2021-01-18 11:31:57 [Broadcast] Darcy O'Toole: <color #010101>Maybe Maybe I should have asked in broadcast?  Anyone around that wants to help out a newbie? I just started last night with another character, but didnt get far
> 2021-01-18 11:32:08 [Broadcast] Snixnix: help how so?
> 2021-01-18 11:32:40 [Broadcast] Darcy O'Toole: <color #010101>I dont know!  Hehe
> 2021-01-18 11:32:49 [Broadcast] Darcy O'Toole: <color #010101>like, what to do next?
> 2021-01-18 11:33:17 [Broadcast] Snixnix: mmmmmm, well im happy to answer questions.  feel free to dm me
> 2021-01-18 11:33:38 [Broadcast] Snixnix: not gonna level a new player though 😄

 

To start, this makes the need for a parser pretty apparent. It's why I wanted to make one in the first place. However, things break down once you get to emotes, combat, and system messages.

In short, there is no reliable way of differentiating them from one another. Since logs are basically HTML files saved to a text document, the only thing I (or anyone really) would have to rely on is what is present in the text document. Unlike standard channels, I can't just look for the presence of [Name] after the date and time. That leaves me with trying to identify patterns. I could potentially get away with using RegEx functions (which are annoying to those not aware) to differentiate them if not for the fact that there is no pattern that exists that is exclusive to them.

So system actions can look like this:
 

> 2021-01-18 11:29:41 Welcome to City of Heroes, CrystalDragon!
> 2021-01-18 11:29:41 You may ask for help from other players by using the Help chat tab.
> 2021-01-18 11:29:41 <a href='cmd:gmotd'><b>Click Here</b></a> or type /gmotd to see the global message of the day again.

Combat actions can look like this:

2021-07-16 04:17:40 You create a Phantom Army of pure illusion!
2021-07-16 04:17:44 You activated the Blind power.
2021-07-16 04:17:44 HIT Soldier! Your Blind power had a 95.00% chance to hit, you rolled a 5.36.
2021-07-16 04:17:44 You Hold Soldier with your Blind.
2021-07-16 04:17:44 You hit Soldier with your Blind for 52.92 points of Psionic damage.
2021-07-16 04:17:45 Readying Arcane Bolt.

And emotes can look like this:

2021-03-15 00:29:28 Rock Lyte removes his helmet, which would visibly vanish as if either being digitized or melding right into his arm. "Only if you can explain why you're having your bots scan someone without permission." He'd give a light grin in response as well.
2021-03-15 00:30:01 [Local] May The Sidekick: <color #010101><bgcolor #64aac3>"They scan everybody...sort of standard rules with my dad..."
2021-03-15 00:30:04 [NPC] Darrel: Can you believe Gentlemanly Giant rescued Positron?
2021-03-15 00:30:26 Adamantium:  Protector Bot protects you from all attacks with a Force Shield.
2021-03-15 00:30:53 Dalentium:  Protector Bot protects you from all attacks with a Force Shield.
2021-03-15 00:30:56 Protector Bot protects you from all attacks with a Force Shield.

Where can the confusion come in? Glad you asked! Some eagle eyed readers might notice that the last chunk of text actually contained a combat log.

Protector Bot is not a character. It's a minion casting force shield on the player. There is literally no difference between the text at the top of the chunk and the text at the bottom. And what about Adamantium and Dalentium? Are those combat logs? Emotes? Who knows!

While in most use cases, you probably aren't going to encounter a player character named "Protector Bot", "You hit", "Readying", etc. However, those are all valid names. The only consistency I can rely on while making a parser is the standard chat channels. Everything else requires manual consideration.

But this doesn't just apply to some fancy parser I want to make. If a player wants to take out a particular segment of chat logs, not only do they have to trim the fat of any channel they don't want (which is relatively easy to do in notepad with the find feature), but they also need to go through it manually to remove system, combat, and/or emotes.

So how would I fix this? Without knowing the likely horrific spaghetti code behind the chat box and logger, I can't say for certain. I know what I would LIKE to see. Even if [Emote], [Combat], [System] was not added to the text as it appears on the chat box, I would absolutely love it if those could be added when the text is saved to the chat log. It would make logs far more useful in my opinion.

[Greycat]

There was  a parser ages ago on live. Then there was some change to the chat format (I think the date/time portion of it?) and that broke - and to my knowledge was never fixed. But the combat portions and such... yes. Those have been an issue.

 

For a while, for one group, I took chat/combat logs and wrote them up, story-wise. That ... was a pain. I eventually stopped doing it - it was becoming another job, essentially.

 

With the mentioned tags, even if someone doesn't write a *parser* per se, it does let you do some cleanup with (as I recall) word processor macros - "find this, do this action" (with the action being "go to the beginning of the line, select to the end, delete.")

[GraspingVileTerror]

Somebody bopped me one of those that they were using during SCoRE.  Lemme see if I can attach it to the forum here . . . 

 

CoX-Log-Reader.zIp

[McSpazz]

6 minutes ago, GraspingVileTerror said:

Somebody bopped me one of those that they were using during SCoRE.  Lemme see if I can attach it to the forum here . . . 

 

CoX-Log-Reader.zIp 362.96 kB · 0 downloads

I'll look it over, but I doubt it can reliably handle combat/emote/system inputs. If it can, I'd love to figure out how...

[Greycat]

I wonder if that's the same one or an update of it...

 

Yeah, that looks similar to what was on live, initially, as I recall... I think perhaps with a bit more functionality. Nice!

[McSpazz]

23 minutes ago, GraspingVileTerror said:

Somebody bopped me one of those that they were using during SCoRE.  Lemme see if I can attach it to the forum here . . . 

 

CoX-Log-Reader.zIp 362.96 kB · 2 downloads

This is very well put together! It honestly does everything I wanted a parser to do!

...except it appears the creator of this encountered the very thing I realized before I even got started. It can't tell the difference between combat, system, and emotes. It's a shame, because if there was an [Emote] tag, it would be fairly simple to suss out and even filter.