City of Data v2.0

[UberGuy]

I'm happy to publicly announce that there is an up-to-date rebirth of the old City of Data site.

 

The site is accessible at https://cod.uberguy.net/

 

This is meant as a successor to Red Tomax's original City of Data, and borrows heavily from that site's organization and visual display of information. Quite a few things have changed, , though, both in the game and our understanding of its data, so it's not identical to the old site. It shows more information in a lot of cases, and lets you see more "raw" data.

This site would not have been possible without the extremely impressive work done by @RubyRedto create the Powers project and the coh.tips site. Ruby's Code Archeology blog post made me realize this project was achievable using Ruby's work as a starting point.

 

I also want to thank @Bopperfor help trying out the early versions of the site, finding bugs and making suggestions. @Captain Powerhouse also gave valuable feedback, and some of his suggestions are still in the pipeline as tweaks and improvements.

 

The site's data is extracted from Homecoming's game files using a modified version of Ruby's tool. This version exposes the raw data in a Python environment, where I perform additional transformations before exporting it as JSON. This version of the data acts as the backing for the website. The site is entirely dynamic on the client side - there is no server-side page rendering. This does mean the site requires Javascript to be enabled and a relatively modern browser. If you've updated your browser in the last couple of years, you should be fine. (If you haven't, I hope you have either an excellent antivirus or exceptional opsec skills.)

There are some ways of displaying things the old CoD did which aren't the most practical for this version. They're possible, but more work (or cause more traffic) due to design decisions I made about how the power and other data is structured. Old CoD built display pages on the server, where the new CoD is entirely client-side logic. This means how the site displays things is more tightly coupled with decisions around the structure of the files it pulls in to show you data. For example, every power is sent as a distinct data file, so displaying a whole powerset on one page would currently require pulling down every power in the powerset. Do-able, but not great. (A more compact whole-powerset display is possible, but ideally calls for a better slice-and-dice export of the powerset as one file.)

 

I've made some effort to check that the site works on mobile devices, but it is not really visually optimized for that. I have no way to test the site on Safari or iOS devices. I can test it on browsers that run on Linux, Windows and Android. Most of my testing was done in Chrome and Chromium-based browsers, plus Firefox.

 

Questions and feedback are welcome. Be sure to check out the "Help" pages on the site, which are accessed via the question mark link on the top right of every page.

 

Edited August 11, 2021 by GM Manga
Repaired URL link

[Bopper]

This is a fantastic site, and I find it to be superior to the old CoD in most ways. You pointed out one thing that I wish it did have (all powers displayed on one page) but that is merely a quality of life desire, far from a required one. The search feature is the best thing since sliced bread, and your showing of modifier tables when using the "None" AT option has helped me a ton when it comes to updating Mids and finding bugs. Great work, I really hope this gets the use it deserves.

[Apparition]

FWIW, the site works well using Brave Browser on iPadOS 14.4.

[Apparition]

I really, really like what I see here.  Very well done, @UberGuy.  Thank you!

 

Just one suggestion: at some point down the road, please implement a dark theme.  It'll look better (IMO), and save battery on mobile devices. 

[summers]

This is simply amazing, thank you!

[Bill Z Bubba]

Doom. The rabbit hole has returned!

 

Awesome work.

[Clave Dark 5]

Thank you to everyone who was involved.  I probably won;'t make too much use of it myself, but others will.  They will use it and design builds.  They will post these builds here on the forums.  And will steal them.  So thank you!

[Aberrant]

Thanks very much for this. I've replaced the old Red Tomax links on the hcwiki to now link here instead if that's okay.

 

Edit: Would it be possible to have a page that displays all of the top level power categories in a big list?

 

Edited January 31, 2021 by Aberrant
Edit: Power Categories request

[UberGuy]

3 hours ago, Aberrant said:

Edit: Would it be possible to have a page that displays all of the top level power categories in a big list?

There are 36,000 of them. I don't think you want that...  😲

 

Edit: That was a pre-coffee answer. That's the number of powers. But still, it's almost 200. Is that really useful?

 

And yes, totally cool to link to it from the wiki!

Edited January 31, 2021 by UberGuy

[Aberrant]

1 hour ago, UberGuy said:

There are 36,000 of them. I don't think you want that...  😲

 

Edit: That was a pre-coffee answer. That's the number of powers. But still, it's almost 200. Is that really useful?

 

And yes, totally cool to link to it from the wiki!

I would find it very useful since the search only shows a very limited number of results. I do a lot of looking for NPC powers to update enemy entries on the wiki and so far I've been working off a .txt file export of Issue 24 powers data so I've already been looking through a 20,241 record list, and 8+ years out of date! A lot of the naming conventions aren't always obvious so having a full top category list to work downwards through would make finding stuff a lot easier. I'm an edge case obviously 😛 

 

Edit: Even more useful to me would be a City of Data for the Villaindefs, but that may be outside of the scope for this project! As of Issue 24 that's only 7122 records! 😄

 

Edit2: Just realised that it already has all the entity data! There's just no way to search them. So yes, definitely, being able to search the Entities as well would be amazing!

Edited January 31, 2021 by Aberrant

[Sabrehawk]

Thank you for this. It is very helpful.

[UberGuy]

2 hours ago, Aberrant said:

I would find it very useful since the search only shows a very limited number of results. I do a lot of looking for NPC powers to update enemy entries on the wiki and so far I've been working off a .txt file export of Issue 24 powers data so I've already been looking through a 20,241 record list, and 8+ years out of date! A lot of the naming conventions aren't always obvious so having a full top category list to work downwards through would make finding stuff a lot easier. I'm an edge case obviously 😛 

If you can work with just a list (which you could use to input into the search box or, with a wee bit of figuring out, the address bar), use this.

2 hours ago, Aberrant said:

 

Edit: Even more useful to me would be a City of Data for the Villaindefs, but that may be outside of the scope for this project! As of Issue 24 that's only 7122 records! 😄

 

Edit2: Just realised that it already has all the entity data! There's just no way to search them. So yes, definitely, being able to search the Entities as well would be amazing!

This would be easy to do, but I want to see what the powers that be think of it. That's a bit spoilery, and beyond what old CoD offered. If Leo and crew don't object, I'll make it a thing.

[Bill Z Bubba]

Confused about something. What is that 0% chance of 18.7685 damage? It's not tagged with anything like fiery embrace. Side note, and I know this isn't your job, but dang, devs, swipe hasn't had the two quick swipe animation for a looooooong time.

 

[Aberrant]

13 minutes ago, UberGuy said:

If you can work with just a list (which you could use to input into the search box or, with a wee bit of figuring out, the address bar), use this.

That's very helpful, thanks!

 

Edit:

7 minutes ago, Bill Z Bubba said:

Confused about something. What is that 0% chance of 18.7685 damage? It's not tagged with anything like fiery embrace. Side note, and I know this isn't your job, but dang, devs, swipe hasn't had the two quick swipe animation for a looooooong time.

Could it be the leftover second tick of damage from when it had a two swipe animation? And instead of removing it they just gave it 0% chance?

 

Edited January 31, 2021 by Aberrant

[Bill Z Bubba]

11 minutes ago, Aberrant said:

Could it be the leftover second tick of damage from when it had a two swipe animation? And instead of removing it they just gave it 0% chance?

I wondered the same but slash, which is now the two slash animation, has both entries in the 1st section.

 

[Erratic1]

2 minutes ago, Bill Z Bubba said:

I wondered the same but slash, which is now the two slash animation, has both entries in the 1st section.

 

 

Because they both occur. The sections would have to be different if one part has 100% and the other 0% (unless they could set the 'after 0.5s' part to zero damage).

[Bopper]

1 hour ago, Bill Z Bubba said:

Confused about something. What is that 0% chance of 18.7685 damage? It's not tagged with anything like fiery embrace. Side note, and I know this isn't your job, but dang, devs, swipe hasn't had the two quick swipe animation for a looooooong time.

 

It basically is a removed effect. Looking at the design formula, Swipe's 2s recharge attack should do 0.68 scaled damage however Swipe is doing 0.76 scale damage (something a 2.5s recharge power should do). However, since Swipe has no secondary effects, it is common to give powers like this a 80% discount on the design formula, so that all lines up correctly (2s is 80% of 2.5s). 

 

Since all that is working correctly, it's safe to assume setting the probability to 0% is an easy solution, as it effectively removes the effect from the power while it also offers the effect slot for editing later if Claws ever gets revamped (for example).

1 hour ago, Bill Z Bubba said:

I wondered the same but slash, which is now the two slash animation, has both entries in the 1st section.

 

That's a new thing with the i25 engine: Effect Groups. On the days of live, all effects had their chances checked individually. This is a big reason why the Crit chance of a split damage power (e.g. Smash/Energy) would combine the damage into one type (Energy), because otherwise the 10% chance to crit would be done individually if there was a Smash crit effect and Energy crit effect. Now...that's a bad example as that practice (crit one damage type) is still done today, but if they wanted to, they could move the Smash and Energy crit effects into an effect group, and that group would all use the same crit roll.

 

So to better understand CoD 2.0, whenever you see a black outlined region with a probabiliy listed in the top row, that is an effect group. Everything below that probability are individual effects that exist within the effect group. The bottom row will show a "required" line, which tells you what is required for that effect group to be checked. In this example, the requirement is if the target is a critter (which implies it is a PvE only effect).

Edited January 31, 2021 by Bopper

[UberGuy]

1 hour ago, Bopper said:

It basically is a removed effect. Looking at the design formula, Swipe's 2s recharge attack should do 0.68 scaled damage however Swipe is doing 0.76 scale damage (something a 2.5s recharge power should do). However, since Swipe has no secondary effects, it is common to give powers like this a 80% discount on the design formula, so that all lines up correctly (2s is 80% of 2.5s). 

 

Since all that is working correctly, it's safe to assume setting the probability to 0% is an easy solution, as it effectively removes the effect from the power while it also offers the effect slot for editing later if Claws ever gets revamped (for example).

While that's true, that feels like a strange choice. This chance will be evaluated every time the power is used. While not even remotely wasted cycles worth being concerned about, it doesn't feel like something anyone working on the game would do intentionally for the sake of convenience in later adding a new effect in this spot, which would be a trivial bit of editing to do even with the old DEF file format.

 

My guess is that someone did this while testing variations in the power, probably back in the Cryptic Studios days, got in in a satisfactory state, and forgot to clean this up later.

 

It may be worth a pass to see if there are any other effects (in any powers) with a 0% chance but no tag on the effect or its parents.

[Bopper]

3 minutes ago, UberGuy said:

While that's true, that feels like a strange choice. This chance will be evaluated every time the power is used. While not even remotely wasted cycles worth being concerned about, it doesn't feel like something anyone working on the game would do intentionally for the sake of convenience in later adding a new effect in this spot, which would be a trivial bit of editing to do even with the old DEF file format.

 

My guess is that someone did this while testing variations in the power, probably back in the Cryptic Studios days, got in in a satisfactory state, and forgot to clean this up later.

 

It may be worth a pass to see if there are any other effects (in any powers) with a 0% chance but no tag on the effect or its parents.

It could be. Powerhouse can probably clear that up for us. I used to think this was a common thing back when I'd see Null effects, I thought those were old effects that got removed but had to remain for who knows why (makes database happy? I dunno). But I found out recently those are possibly there for animation purposes. I don't know if this would be a case of that. Probably not. It definitelly seems like an oversight, but then I see PvP having the delayed damage effect and wonder why there's a discrepancy.

[UberGuy]

Null effects are usually, as you mentioned, for FX purposes (and I try to detect that and display it in CoD v2) or, in some cases, simply to put an icon in your status tray (as in the case of the Scrapper critical hit "inherent"). I've seen some I couldn't figure out, though. But there's no useful equivalent to that, as far as I know, for a fully-fleshed out effect with no chance of actually activating (and no visible means to change that chance).

 

It's a bit like an effect with a condition that says 

if false

 

Edited January 31, 2021 by UberGuy

[Bopper]

2 minutes ago, UberGuy said:

It's a bit like an effect with a condition that says 

if false

 

I guess that's better than

while true

 

I think I've seen other cases of null effects that might be tied to set bonuses, but I could be misremembering. 

[UberGuy]

Actually, I found a bunch of other powers with this going on, and some are fairly new.

 

This is just the player-specific ones. There are NPC ones as well.

 

Corruptor_Ranged.Dual_Pistols.Bullet_Rain
Incarnate.Interface_Silent.Confuse
Incarnate.Interface_Silent.Immobilize
Mastermind_Summon.Mercenaries.Equip_Mercenary
Mastermind_Summon.Mercenaries.Tactical_Upgrade
Mastermind_Summon.Ninjas.Kuji_In_Zen
Mastermind_Summon.Ninjas.Train_Ninjas
Mastermind_Summon.Robotics.Equip_Robot
Mastermind_Summon.Robotics.Upgrade_Robot
Mastermind_Summon.Thugs.Equip_Thugs
Mastermind_Summon.Thugs.Upgrade_Equipment
Pool.Utility_Belt.Flying_Kick
Pool.Utility_Belt.Poisoned_Dagger
Scrapper_Melee.Claws.Swipe
Sentinel_Ranged.Dual_Pistols.Bullet_Rain
Stalker_Melee.Claws.Eviscerate
Stalker_Melee.Claws.Swipe
Stalker_Melee.Ninja_Sword.Head_Splitter
Temporary_Powers.Temporary_Powers.Opportunity_Lock

 

Edited January 31, 2021 by UberGuy
Sorted the powers list

[UberGuy]

The set-related nulls you're remembering are effects like LotG and Preventive Medicine. The 6th enhancement actually has no direct effect, so its only effect is a Null. That's because the special effect of slotting those is a set bonus, which is applied by external logic.

 

Which reminds me - that's an expansion of the site I have planned - one that lists the details of boost sets (which is the internal name for enhancements that give bonuses based on how many you have slotted).

[Captain Powerhouse]

5 hours ago, Bill Z Bubba said:

Confused about something. What is that 0% chance of 18.7685 damage? It's not tagged with anything like fiery embrace. Side note, and I know this isn't your job, but dang, devs, swipe hasn't had the two quick swipe animation for a looooooong time.

 

Back in the day, it was extremely common for powers that had effects removed to simply have their chances lowered to zero, or turned into "kNulls". You will see this, a lot, in powers that used to drop toggles in pvp. Many had their effects turned into zero chance nulls. 

 

There are some cases that have null effects with either VFX attached to them, or animation information for the target (Continuing|Conditional Bits.) The ones that are such and have zero chance of happening, are likely abandoned or removed due to Reasons™.

 

5 hours ago, Bopper said:

It basically is a removed effect. Looking at the design formula, Swipe's 2s recharge attack should do 0.68 scaled damage however Swipe is doing 0.76 scale damage (something a 2.5s recharge power should do). However, since Swipe has no secondary effects, it is common to give powers like this a 80% discount on the design formula, so that all lines up correctly (2s is 80% of 2.5s). 

 

Don't try to figure out why Claws does what Claws does. It uses it's own formula that has been lost to archeology.

[WanderingAries]

I don't think I've ever used the old version either, but if it helps, it loads and I can click through things fine in Opera and Safari.