Looking for data mappings

[TranceMagick]

I have a database of the enhancements currently in use.  I want to map the data items in the buildsave file back to this database, and I'm trying to avoid having to manually create the relationships/mappings myself since it's such a monster.  By chance has anyone else done this mapping and is willing to let me have it?  Or for that matter, am I missing the obvious and the relationships are there, I'm just not seeing it.   Thanks in advance,.

[AboveTheChemist]

On 1/10/2025 at 4:32 PM, TranceMagick said:

I have a database of the enhancements currently in use.  I want to map the data items in the buildsave file back to this database

 

I have a crude enhancement database but I am not sure if it is mapped the way you need. If you can provide an example mapping of what you need, I should be able to tell if what I have will work for you.

[TranceMagick]

Thank you for responding.  I contacted you a while back about your data asking permission to use it if I ever came up with anything interesting to do with it (I have not yet).  I imported your spreadsheet into a SQLite database with no issues to speak of, the data in the spreadsheet was laid out well, so it was a breeze.  The problem I'm working on at the moment is that the data you get in the buildsave file doesn't have a clear mapping to what the actual enhancement it's referring to.  For example, I have a Celerity +Stealth in sprint at level 1, but the buildfile entry lists it as Crafted_Celerity_C, which does not map back to my data easily.  Frankly, the options I've identified to resolve this would involve more time than I'm willing to commit to the project (the source for Mids is available, so I could figure things out from there, but frankly, I've looked at the code, and from what I've seen it would take a substantial effort to get what I need out of it).

 

I've attached the SQLite DB I created from your spreadsheet if you're interested.   I use DB Browser for SQLite, it's small, free, and easy to set up.

 

Also, the spreadsheet you have doesn't include the basic IO recipes, just the sets.  Would you know where I might find this data so I could merge it in with your data?

20250113-HC Recipe DB.zip

[Michiyo]

This: https://cod.uberguy.net/homecoming/raw_data_homecoming.zip is a zip of json dumps that City of Data uses, boost_sets\celerity.json shows:
 

    "boost_infos": {

      "crafted_celerity_a": {

        "computed_name": "Celerity: Endurance",

      "crafted_celerity_c": {

        "computed_name": "Celerity: +Stealth",

 

This may be of some help to you.

[TranceMagick]

13 hours ago, Michiyo said:

    "boost_infos": {

      "crafted_celerity_a": {

        "computed_name": "Celerity: Endurance",

      "crafted_celerity_c": {

        "computed_name": "Celerity: +Stealth",

 

Thanks for the reply.  This was incredibly helpful.  I had looked at this dataset before, and I'm not really familiar with JSON files, so the learning curve appeared steep, and I moved on looking for an easier solution.  Your snippet showed me exactly the relationship I was looking for, so while it looks like I'll have to start from scratch again, the couple of google searches I've run seem to show a lot of utilities for manipulating JSON files in the way I need.

 

Thanks to everyone who responded, I appreciate it.

[lemming]

Yea, I haven't dug into those JSON files myself, but that's easy to work with.

Weighing if I want to use them as is, or convert to a more compact form for my own use.

[AboveTheChemist]

On 1/13/2025 at 2:47 PM, TranceMagick said:

For example, I have a Celerity +Stealth in sprint at level 1, but the buildfile entry lists it as Crafted_Celerity_C

 

I am pretty sure I have that somewhere in a friendlier-than-JSON format, but not 100% sure. I had to map this relationship when I updated the freebies menu to show the actual names versus the internal names (with the letters), because I generated all that with a Python script. Let me have a look and see if I can find it. Also I totally do not remember you contacting me previously so I really hope that I am thinking of something other that what I gave you previously.

 

Even if I don't have it readily available, I've parsed JSON from City of Data before so I could probably whip up a script to read in those files and output something a little more user-friendly for you.

[AboveTheChemist]

I did have the data in friendlier-than-JSON format but it is nearly four years old and I didn't want to vet it for any changes since then, so I just parsed the latest City of Data JSON files (specifically the ones in individual subfolders in powers\boosts within the zipped archive) to get the boost codes. This includes every single enhancement, be they crafted, bought, dropped, attuned, etc. I've attached the data here as a semi-colon delimited file:

 

enh_boost_codes_types.txt

1. First column: Enhancement name as it appears in-game
2. Second column: Boost code for that enhancement
3. Third column: Enhancement type, one of Crafted, Attuned, or a catch-all Other that includes anything that was neither Crafted or Attuned (like bought and dropped enhancements)

On 1/13/2025 at 2:47 PM, TranceMagick said:

Also, the spreadsheet you have doesn't include the basic IO recipes, just the sets.  Would you know where I might find this data so I could merge it in with your data?

 

I missed this question earlier but I was in a scripting mood so I put together this dataset as well. I've attached it here:

 

invention_final.txt

 

It is also semi-colon delimited and follows the same basic format as my other recipe salvage dataset, with a few minor differences. The first column has the enhancement name, not the recipe name as in my other dataset (if you need the recipe name, I can modify that easily). Second and third columns are min and max level, respectively, as in my other dataset. These recipes have a max of three components, so the salvage names appear in columns four through six. Some enhancements require two of a component and in those cases, a component will be listed twice as it was in my other dataset. Note that I spot-checked this data against the invention recipes wiki page but I did not check it thoroughly, so you may want to check it against that page to make sure I didn't make any errors. This data came from the leaked source files.

 

Just let me know if you have any questions or need clarification on any of the data. Enjoy!

Edited January 15 by AboveTheChemist

[lemming]

Thanks ATC, that saves me from doing that myself. 😄

[TranceMagick]

Honestly, I'm a little overwhelmed you took the time to do this for me/us.  I can't thank you enough.  My coding skills are antiquated now (VB6, Unix shell, Transact-SQL), and I'm not particularly interested in learning a new language now.  I was doing this in VBA (not .net), and while I was finding things to parse the JSON file, they all seemed kludgy work arounds.  What you have given me seems to cover all bases, so all I need to do is parse the build file, and generate the SQL to run against the database.  I've already been able to import the data into the sql db.  I really can't express my appreciation for you (and the others) help on this.

[AboveTheChemist]

Happy to help! I know the input data well enough and it's been a little while since I did much coding. I've been looking for some small projects to help knock off the rust so it probably helped me as much as it helped you.