Progor

I started trying to parse boostsets as well, and found an awesome tool to help: https://ide.kaitai.io/# part of the https://kaitai.io/ kaitai-struct project. It gives you a DSL made specifically for parsing binary files, and more importantly, an IDE that highlights the section of the binary file that each parse covers. Once you've got the file defined in their simple YAML format, it generates code in a bunch of languages to do the actual parsing. The only difficulty is the parser output is pretty verbose, mentioning every array size, string length, and offset pointer you had to use to define the file. I fixed this by naming the important variable at each level "value" and replacing the parent object with the value field whenever I see it. I've got boostsets and attrib_names (wanted to start with something easy to evaluate the tool) in a new project: https://github.com/dpwhittaker/coh-parse7 With the following "API" available so far: https://dpwhittaker.github.io/coh-parse7/boostSets.json https://dpwhittaker.github.io/coh-parse7/attribNames.json I'll probably do client messages in this format next, then try to tackle powers to see if it ends up being even more straightforward than the rust implementation. The easier it is to find and fix format changes after a release, the faster tools that rely on it can get updated.

At least the recursion depth is limited by the size of the file, or I may have never emerged from the abyss!

There's also some additional files that I don't think any of the projects have parsed yet. One is boostsets.bin. I may try to reverse engineer that one myself based on ourodev code, but I don't have access to homecoming code, so I'm at a disadvantage. I'm not sure what information it will or won't provide.

Alright, I went ahead and did the defaults and split into categories. This made all the files around 1MB or less (a few big ones up to 5MB), which is much more doable for a web app that's pulling in around 5-10 files to get the primary, secondary, pools, boosts, and pets for a single build. If you're going to do much more than this you probably just want to load the 41MB powers.json with everything in it, including the powers possessed by Hellions in the Sewer trial. Maybe I'll make a powers.Player.json that has everything accessible to a player. There's also the powers.default.json. This file has the default values for any value that shows up in more than 20% of the powers. At each level of this object, the "sub" field represents the defaults of the object contained in that field. For instance: sub.effects.tag tells you the default value that should go in effects: [ { tag: ...} ]. This structure is more deeply nested than I thought (effects can have effects of their own), so I used a recursive function to find the defaults, and you'll probably need one to rebuild the original object from the defaults if you want to. Here's a shot of the files that are output (197 total). Feel free to look around in the github project, or you can access them API-style like this: https://dpwhittaker.github.io/sif_parser/powers/Tanker_Melee.json I'll work on the archetypes and powersets next to act as an index into these files. I'll probably end up adding some folder structure here, so don't start building anything on this yet 🙂 edit: removed the powers. prefix and moved them into a powers folder.

Yep, noticed that. So, got that figured out and made the first early, hard-coded, cobbled-together, command-line app. https://github.com/dpwhittaker/sif_parser Just "cargo run" it with bin.pigg and bin_powers.pigg in the parent directory to generate a powers.json in the parent directory. and here's the output for the powers.bin: https://1drv.ms/u/s!AuGvcX8-qHq2luNtIXUr_nVnmKheqA?e=nMZT8N So, next steps are: output the other bins as json come up with a "default value" for each field and only serialize the field if it is different from the default to cut down on file size. 242MB uncompressed is a bit much for a web app split it up into categories similar @RubyRed's API This post-processing will probably happen in a language I'm more comfortable with - Node or Python. In any case, this should get it down to something that fits comfortably in github pages limits, so I'll publish it there as a "static API" similar to RubyRed's.

I think there's value here as well as @RubyRed's API. This library can give you *all* the raw data, while RubyRed's API focuses on giving you the important bits for building tools without all the extras. I'll work on a command line application for exporting the results as JSON so it can be just as accessible from downstream apps. @Sif Alright, quick usage check. I don't know rust well enough to write correct syntax yet, but here's what it looks like I need to do to load powers data and write it out as json: let pigg = Pigg::new("path/to/file.pigg"); let messages = parse_messages::get_pmessages(pigg.get_data("bin/clientmessages-en.bin")); let powers = defs::decode<objects::Power>(pigg.get_data("bin/powers.bin")); foreach (power in powers) power.fix_strings(messages); writeFile("powers.json", serde_json::to_string(powers)); Does that look about right (other than slaughtering the syntax, I'm sure), or am I missing something?

Reposting @Sif's work here so we don't take over @The Philotic Knight's thread: GitHub (source): https://github.com/IndexObscurum/index_datamanip Crates.io (Rust package): https://crates.io/crates/index_datamanip Docs.rs (docs): https://docs.rs/index_datamanip/ This contains the code for extracting data from Piggs, enough handling around MessageStores to go from the P-strings to localized text (the rest of the data is parsed, but discarded currently), and for deserializing Parse7-encoded bins into the passed in data structures. Additionally, it contains all of the structures (in the objects module) used for deserializing Powers/Powerset/PowerCategories/Classes (Classes has something like 2 fields unimplemented). I have some additional high level documentation on the formats I've written up that I intend to merge into the library's docs. Some work would need to be done to make this a CLI / callable from non-Rust languages (I have some ideas for both, but... I also rolled a new Blaster...)

@RubyRed I was about to start working on what @Sif suggested using his library to create something like what you just did... but you've already done it, plus calculated scales, so I'll work with yours for now. Thank you both for your efforts! Two things I notice missing from the API that is useful for planner-like things are Pets (how much does the burn patch do over time? How much does a Protector Bot bubble for?) and Boosts (Enhancements). I'll add those power categories and see what comes out in the parse locally, but we may need to end up parsing boostsets.bin to get all the info necessary.

Yeah, that's why they stopped calling themselves UI (user interface) designers and started calling themselves UX (user experience) designers. It's as much about having a streamlined flow through the application as it is about looking pretty. We talked this afternoon and the first thing we worked out was user flow diagrams, not artistic concepts, so he's on the same page. "intuitive and easy to use" is often harder than people realize to get right. I can handle making the application do what it needs to do correctly and consistently, but having a UX expert will help achieve that zen-like flow where every button you want to tap just happens to be right under your thumb when you need to tap it.

Quick update: power selection is started. I still need to filter out powers you've already selected from the dropdown list so you can't duplicate, and add the pool powers, but it's coming along. It follows the same basic design for the Heroes screen: click the pencil to modify the power, select from a dropdown, click the info button if you want details. I'm even retaining a per-power notes box if people want to use it. Let me know if anyone wants to see a Tag Line on the powers as well. Another note: a generous forum-goer who happens to be a UX designer by day (I'll let them pipe in and identify themselves if they wish) has offered to help with the look and feel, so once he gets a chance to send me some sketches and I get them integrated, it should start looking nicer. The empty space below the power name will be filled with the slotted enhancements, and perhaps some basic effect details.

I can actually do one better on the first request - you could import a /buildsave file and it could compare which enhancements you have equipped to the ones in your build and highlight them. I could also add a manual checkbox to allow you to mark the ones in your inventory that have not been slotted. A "shopping list" report sounds like the next request, which would list out the enhancements you are currently missing, probably sorted by rarity and/or expected cost. Do you have a link to the slash commands on test server? That sounds pretty straightforward as well.

If a moderator wants to move it, that's fine. Otherwise I'll start a new topic there when I get to the alpha release.

Think this will cover your needs? The notes box at the bottom is not visible until you toggle it with the Notes button. It's 20 lines long and will scroll if you need more, so you can write a small novel in there if you want. The Tag Line shows up as small text below the name, and can technically be pretty much as long as you want as well, but it will blow up your hero list if it's too long, so I'd keep it short. If there's nothing there it takes up no space at all. Depending on the size of your phone, the entire "card" for a single hero may not fit on your screen, but scrolling up and down shouldn't be an issue. If you have a larger phone, it fits fine (this size simulates an iPhone X, for reference).

Yeah, notes would be easy to add. It's more a matter of screen real estate vs. general utility. So I'll hide the notes behind a button click, then give you a big multiline textbox you can put anything you want in. I could also add a 1-line "Tag Line" under the name that would show up in the collapsed view as well if you wanted some custom text visible from the "hero tracker".

Progress is being made: This is the blander "SFW" theme. If there's still too much color, let me know. I'll spice it up a bit after the basic functionality is working. Incidentally, if anyone has UX design experience, I'd love some input on what that should look like 🙂 Otherwise I'll mostly just round off some corners, add more color and more icons from the game, and have a blue hero and red villain theme, each on black or white background. So I'm working on ways to show you just as much information as you want without without cramming your tiny screen full of size 1 font. This is the Hero/Villain creation tab, that lets you create your stable of heroes and select their origin, archetype, primary and secondary. Once you've done that, you can click the Go To Build button, which will take you to the build tab to start adding powers and slots. Notice that there are 3 different levels of "expansion" here - the "Ice Baby" build is fully collapsed, showing just the basic information to remind you which build it is. On the right are edit and delete icons (don't worry, the delete shows a confirmation before actually deleting). when you click the edit pencil icon, it expands to look like "Hot T", and give you the option to alter the name, origin, archetype, and power sets. Altering the Archetype, Primary, or Secondary would mess up the build, so they will probably be grayed out after you add the first power unless you click the Clear/Reset build button. Finally, if you want more information about the archetype or power sets you are selecting, you can click the info icon to expand out the description of the archetype or power set. The plus at the bottom lets you add new characters. When you go to the build tab, you'll see the same block at the top (fully collapsed). This should make it easy when you are first creating a build to switch between power sets and look at the powers until you decide on one you like. The build tab will have the same basic flow. No plus icon, since it will be pre-populated with the 24 power slots, all empty. Click the edit pencil on one, select from a dropdown of powers available at that level. Info icons everywhere will give you the textual description of whatever you've selected. Another icon will let you drill down to the numbers. Click the edit pencil again to collapse everything down to just the level, name of the power, and enhancements slotted. For enhancements, I'm still thinking through what the interaction should look like. I know I want it to be more set-based than Mids' - it shouldn't take 50 clicks to change 6 SOs to a full set of IOs. That should be three actions: clear enhancements from power -> select set -> click select all. To handle frankenslotting, each enhancement in the set should also be able to be selected individually. As long as you haven't filled all 6 slots, then additional set selection dropdowns will be shown. TOs, DOs, SOs, IOs, and Hamis will just show up as additional sets, only instead of checking off the enhancement, you'll be able to select a number between 1 and 6 (through slider, spinner, 6 checkboxes, or just repeatedly clicking the enhancement, I'm not sure). Notice I didn't mention "adding slots" anywhere - I'm not trying to be faithful to the in-game experience like Mids' does. If you add an enhancement it automatically adds the slot. Once you've added 67 enhancements (other than the first in each power), you can't add any more enhancements. Either way, I'll put more screenshots up as I get more built. Hoping to have the first prototype ready for alpha testing this weekend (probably without any sort of totals calculation, just picking powers and enhancements). Let me know what you think.

I had thought about this one as well. If I keep out all the graphics, icons, and colors, and highlight with bold and italics, a casual glance from a coworker could easily mistake it for an email app. Of course if they look closer they would notice all your emails are about Plasma Shields and Fire Swords... If they still think it's an email, then I want to work where you work 🙂 I may actually build this view first. Then come back and add themes that let you ratchet up the color and graphics one step at a time. One theme would just add color coding to the text, the next would start enabling power and enhancement icons, and the last would be the full graphical experience in both Hero and Villain color schemes.

Updated the OP with suggestions to this point. Keep 'em coming.

Oh, that's another good idea SmalltalkJava. I had thought about including the buffs your pets put on you, but not about showing the attributes of your pets with your buffs. I'll have to think about where to put that in the UI (and see if I have enough data on the pets innate stats to build that view). Pets may end up getting a dedicated tab that looks like the Totals page for each pet. You have the right idea on what I'm looking to achieve with the automated optimization work. It's definitely more of an idea generator than a build perfecter. The workflow will likely be: put your must haves in the build to start with, generate some optimized builds for some of your wants. Then you'll either see you can't have everything you want and have to back off some values and re-optimize, or find out what you want is easy if you have a trillion inf, and have to add constraints on price, or add more wants if you have a few trillion laying around. Once the optimizer gets close to a build you like, you'll probably still want to go in and re-order powers for exemplar situations, perhaps move sets around, and tweak a few things here and there to really make it yours. Optimizers have a weird way of coming up with left-field suggestions that just happen to fit the requirements. For instance, you might be able to get perma-hasten if you 6-slot hover on a specific build, but surely those slots could be used better elsewhere and 99% uptime on hasten isn't bad.

Hi Redlynne, the main reason a lot of the build planners use the URL to store the build data is so they don't have to run a database to store them. It turns out to be a nice feature for sharing builds on forums and works as long as you manage your bookmarks well. I've built a few planners this way in the past, like https://dpwhittaker.github.io/hos-planner/ for the Heroes of Steel game by the Trese Brothers (check it out if you like Turn Based Strategy games). I'll simulate this feature by having two Create Link buttons, one for a Dynamic link and another for a Perma link. When the user clicks a dynamic link, they will come back to the most up to date version of the build, so as you update Local Man, the AR/Dev Natural blaster, the link will stay the same but the build will update with you. Other viewers will see a read-only view of the most current build, and be given the opportunity to copy it to their own build list if they want to edit it. A Perma link will function just like importing a build in mids from the forum - it will automatically be added to your profile and presented in edit mode, but that link will always refer to the exact same build, even if the original creator has long since deleted it or turned Local Man into a Fire/Fire magical scrapper (I guess Local Man had one too many run-ins with the Circle of Thorns). The main benefit of the database backed links is that you can start a link on the web page, keep working on it on your phone while you are out and about, then pull it up on your tablet when you are ready to respec in game, all without emailing links back and forth to yourself - just log in and they are all there. You also get the benefit of having a searchable central repository of builds. Want to find all the Tanker builds that have capped Fire resists and defenses and > 100 DPS against AVs? You can search all the public builds that have been made by the community with a dedicated search engine with attribute level filters a lot easier than trying to craft a search term that will bring up all the right posts on the forum, then look through them for build link, import it, and see if it actually matches what you are looking for. Yes, Proc percentages will be necessary to accurately calculate DPS, so I'll be sure to get that calculator built before I start working on the DPS calculator, and display the results in the power details. And yes, Rule of Five and ED will be enforced, and I'll find some way to visually represent when those rules have been violated (probably outlining the enhancements in red or yellow that violate the rules). Yeah, the Optimization features are definitely down the road after manually creating builds and analyzing them is feature complete. After all, you have to be able to fully create and analyze a build before you can let the computer loop through the billions of possibilities for you and search for the optimal build. I'll admit, it's more of a way to stretch my algorithmic muscles than anything... CoH has so many possibilities that I may even have to dust off some AI search algorithms I haven't been able to use since college. Yes, I'm a true geek that salivates over writing a branch-and-bound or minimax algorithm from scratch, and dream of finding an actual real-world application for a genetic algorithm.

That is exactly the kind of requests I was looking for, thank you! I had completely forgotten about the /buildsave command. I will definitely put it on the list, as well as Mids exports, data links, and save files (they are all basically the same thing as long as I can decode the hex block at the end) as ways to get builds into the app. You'll be able to sign into the web version of the app with the same id/password as you do on your phone and import them there, and it will show up immediately on your phone (should be basically real time). Is there a reason to save/export to the in-game build format? There's not a /buildload command I can use during respec or anything is there? Full interoperability with Mids' is certainly a worthy target though. I could even see reading and writing Mids' save files to a dropbox or google drive so you could just use your dropbox folder as Mids' save location and keep your builds in sync across apps that way. Keep in mind this feature would be added later in the process once all the features of builds are available. I don't want to delete your Incarnate selections every time you save because I haven't implemented that system yet. Donations would use a Paypal donate button. I have zero desire to keep track of your credit card numbers, personally identifiable information, or even passwords if I can help it (login will use google/facebook/etc. and password as a last resort).

Hi, I know suckerpunch was working on a mobile-friendly online planner, but I haven't seen anything about it since June. So I'm planning on building one myself in React Native so it will have a native feel on phones and tablets, but also be able to run on the web if you want to use your big screen. The UI will be significantly different from Mids' to fit comfortably on a phone (more tabs and linear lists), but I'll try to make the tablet/web view use the full screen as well. Right now the features I'm planning are below, but let me know if you think I missed something important. Build a list of Heroes/Villains with a specified selection of class, primary, and secondary power setsAdd pools, powers, slots, and enhancements (of course) Select incarnate slots Highlight Rule of Five and ED violations visually on the main build view Show effects of all buffs/procs/pet buffs/enhancements on player similar to "Advanced Totals" in Mids Make a build public (shared with everyone in a searchable build library) Import builds from /buildsave and mids hexblock (file upload or paste) Export to mids hexblock and text/bbcode Share a build by link to individuals - dynamic and perma link Copy shared build to your own list so you can tweak it to your preferences Multiple themes (SFW email client look, SFW+color coding, icons, full graphics, multiple color schemes) "Optimize" a build: - fill in the powers, slots, and enhancements you "have to have", leave the rest blank - specify targets for any attribute (e.g. I want 80% res(all), 40% def(all but psionic), and 1000% regen on this tank) - targets can be partially met. i.e. if hasten with 70% recharge is up 50% of the time, it will be counted as 35% recharge - specify max enhancement level (i.e. SOs, IOs, Yellow/Orange/Purple Sets) - specify average cost for enhancements at each specific level (cost can also be an attribute to optimize for) - specify cost for specific enhancements (e.g. the LotG recharge enhancement costs more than the rest of the set)- the optimizer will scan through the available powers, slots, and enhancements and find the builds that come the closest to your target attributes - a score will be given to each build based on what percentage of the target they received, and each build will be ranked by the percentage of the target that they reached (a build with 88% res, 44%def and 1100% regen would have a score of 110) - replace your current build with one of the optimized builds, or create a new hero from the optimized build Optimal DPS attack chain: - select a given enemy count, level, and mix of minion/lieutenant/boss/elite/AV (e.g. lvl + 4, 8 minion, 2 lieutenant, 1 boss) - require specific attacks at least every N seconds (e.g. to ensure double-stacked Parry on Broad Sword) - calculate an attack chain and total DPS for the entire group of enemies, including toHit % for each enemy - calculate proc %, include in DPS calculation, and display on power details - attack chain will include self heals, buffs like build up and hasten, etc. to calculate their contribution correctly - this may mean an attack chain is very long if it needs to include a slow recharge power like Parasitic Leech - calculate end "time-to-empty" or "blues-per-minute" to give you an idea of how much external recovery you'll need to keep going - can include time-to-empty and blues-per-minute in optimizable attributes to find end-efficient attack chains - goal is to inform you of the best attack chain whether you are jumping into the big mob or soloing an AV Build explorer: - select constraints on Archetype, power sets, and attributes (similar to build optimization) - you can select "any archetype" and/or "any power set" - select DPS scenario for calculating optimal DPS if you use DPS as a constraint - searches all possible builds for matches to your constraints and rank them by how close they come - I'm good at algorithms and optimization, but this may take some time to run, especially on a low end phone - target: the altoholic who is constantly looking for their next build and wants to build something that meets a specific niche Long term goals: - show effects of all buffs on each pet under your control in "Advanced Totals" fashion - fight simulator - enter a specific set of enemies or power sets you will be fighting against and calculate % chance to survive the fight (basically, the DPS calculator on steroids with defensive calcs included) - trial simulator - calculate % chance to survive an entire trial by specifying multiple fights with different enemy groups - share fights and trials so everyone can use it to optimize or explore new builds - temp powers and accolades - if I can get access to the game servers, I'll pull cost information directly from the auction house I'd like to keep at least basic usage free and ad-free, but I will need some sort of income from it to keep the database running if all 90,000 of you want to use it. For the next few months while I'm building it, I'll make it available as soon as its useful and take donations. As long as the donations keep the lights on (hopefully indefinitely), I won't need to charge for it. If they don't, I'll probably let you create up to 5 builds free, and have IAPs to unlock more build slots (one-time purchase) or enable access to the build database for sharing and searching (small monthly subscription). I certainly don't want to bait-and-switch, so just putting this out there from the start. One caveat: I don't own a mac or an Apple Developer id yet, so getting it in the Apple App Store will cost me about $500 for a used mac mini, iPhone, and the developer id. I'll probably create a separate donation box to take donations toward this cost. If it doesn't reach $500, Apple users will continue to use the web app on their phones, while Android users will get a native app from the start. I may be able to come up with a workaround that lets you use the Expo app in the app store to launch the app natively (in development mode), but no promises there. Either way, if you have any suggestions on features, pricing, or anything else related to this project, please let me know. Thanks!