Number Six

This is a technical deep dive of the new Homecoming Launcher. For download instructions, please visit the public beta thread. Terminology The launcher uses some terminology that, while it’s not necessary to know for normal use, can be helpful when reading log files or discussing features. Package: A package is a collection of files that lives under a particular root (which may be a subdirectory). Packages are versioned and stateful - a cancelled download will not leave a package partially installed or updated for example. Only once the entire transaction is staged and verified is a package committed. Packages can have dependencies on other packages. Packages are not directly visible through the user interface and are mostly an implementation detail. Profile: A profile, or launch profile, is what you see in the UI as the buttons on the left. A profile contains information about which packages it requires, as well as instructions on how to launch a program, and what configuration options can be set. News feeds and server status are also provided on a per-profile basis. Since there is a layer of separation, multiple profiles can (and in fact, do) share packages of common files. The launcher handles the details of figuring out which packages need to be installed or removed based on the profiles the user has indicated they want installed. Security Features The HC Launcher has multiple layers of security. The first layer of security is the cryptographic integrity protections. All launcher assets, scripts, code, profiles, and packages are signed using X.509 certificates. We are currently using 2048 RSA keys for the production infrastructure, and 384-bit elliptic curve (secp384r1) for our test/dev infrastructure. The exact algorithms used may change in the future once there is wider library support for ed25519. The launcher does not allow any code to run that is not signed, whether from a package or as part of a self-update. The signing keys are not accessible from the deployment infrastructure, so even a compromised download server could not cause users to run untrusted code. By running the launcher, you’re still trusting the HC Team, but you can at least know that’s who the files are actually coming from. There are some additional but less effective layers. One example is that each package has its own VFS with the package root as /, and processes all file access within it, so the package files themselves can’t reference absolute paths or escape out of the VFS. This doesn’t matter that much when we’re talking about downloading program code anyway, and is more useful for preventing accidents rather than trying to block something malicious. Finally, the launcher has the ability to log in to our account system and retrieve an OAuth token. While there is not currently much use to doing this, we have near-term plans to use this functionality to log into the game itself. This allows us to address a common feature request and create a secure method to “remember password” that doesn’t actually save the password. File Structure The new file layout is designed to make the game client cleaner and more organized, as well as reduce the chance of potential problems with name collision. Some of it will look familiar and has not changed from the traditional structure, while some of it will not. From the top level, here’s a breakdown of what is inside the HC: CoH install folder: accounts - Each account you log into will get a subfolder in here (moved from the top-level). The per-account subfolder contains chat logs if you have that option turned on, visited maps / fog of war data, playerslot.txt, and settings.json, which contains certain account-wide UI settings such as cooldown timers and chat timestamps. architect - Contains all AE-related data. Depending on what you have saved, it may contain subfolders for missions, storyarcs, and critters. assets - All of the packed game assets are contained in subfolders. Renamed and reorganized as while the assets are currently packed using the pigg format, that may or may not change in the future. bin - The game executables. It contains one subdirectory per platform -- currently win32 and win64. Inside you will find the launcher itself, as well as a subdirectory for each profile (live/beta/pre) with the game executables. cache - Temporary data that may be deleted with no ill effect. The shader cache, which was previously in LocalAppData\NCSoft has been moved here. costumes - Saved costumes. No change from the traditional location. crash - Any saved crash reports or memory dumps end up here. data - May be used for file overrides for addons. While we will be adding an official addon architecture in the future, placing files under data is still the best current way to apply client modifications. logs - Contains system-wide log files. Inside is a subfolder for the launcher logs, as well as a folder for each profile. powercust - Saved power customization themes. No changes here. screenshots - Exactly what it says. settings - Contains subfolders for the launcher itself as well as each game profile. Inside the game profile folders, you will find json files with the graphics and client settings that were previously stored in the registry. The per-profile settings folder is also used by the various in-game commands for saving and loading settings, keybinds, etc. Interesting Technical Details The core of the launcher is written in C. It uses a cross-platform support library developed internally that the game itself has used for a little over a year -- it started life as part of the project to port the client & mapserver to 64-bit. That library in turn uses jemalloc, mbedtls, and zlib. The launcher adds IUP, libjpeg, libpng, and libcurl. The launcher also embeds a Lua interpreter, and the UI is written in Lua. This is partly to make minor updates to the launcher’s UI strictly a “data” change that doesn’t cause antivirus products to freak out, and partly a proof of concept and testbed for future projects involving the game client. The lua scripts are digitally signed and the launcher will not run unsigned code. The GUI is built using IUP, specifically IUP’s lua bindings. Enterprising users will discover a ‘launchercli’ binary inside of the bin directory. This is exactly what it sounds like. While slightly barebones, it allows for installing, updating, and launching the game from the command line.

Welcome to the public test of the new launcher for Homecoming: City of Heroes (or HC Launcher for short)! Internally codenamed ‘Project Hangover’, the new launcher is intended to function as an officially supported successor to take the place of programs like Tequila. This post is a high-level overview of the launcher. There is a separate post with a technical deep dive here. Looking for a quick start? Click here to download the launcher! Note: This needs to be installed to a new location (not your existing CoH install directory) If you are currently using Tequila it will offer to copy over most data This testing phase is focused on using the launcher under Windows. It is fully functional when running under wine on Linux, but installing wine varies from one flavor to another. Manga has an experimental build of Island Rum that can act as a bridge to the new launcher on Macs available here. HC Launcher has three primary design goals: Speed, Security, and Compatibility. Speed: The launcher is supposed to get you into the game, it shouldn’t take any longer than absolutely necessary to fulfill this function. Tequila and several other launchers are based on .NET, and as a result have noticeably slow startup times on some systems. HC Launcher is designed not only for fast startup and faster patching, but also to avoid long and unnecessary file verification, or delays due to Internet latency. Security: This has been a frequent criticism of Tequila and its use of unauthenticated manifest files. By putting in a manifest file, you are in effect trusting the author of that manifest with complete access to the user account you’re running it under, since it can download and run arbitrary code. It also lacks protection against man-in-the-middle attacks, making it impossible to know if a malicious third party has hijacked the manifest. HC Launcher addresses all of those issues by using strong cryptography to verify the integrity of itself and everything it downloads. More details can be found in the technical deep dive post. Compatibility: HC Launcher is written in C and uses many of the same libraries as the game itself. As a result, it should run in any environment that is capable of running the game, such as wine on Linux/Mac, without any additional work. Important note: HC Launcher uses a different file layout for the client install, reorganizing it to make it easier to manage. As a result, it cannot coexist with Tequila and must be installed to a different location. By default it attempts to locate existing files to avoid having to redownload them, as well as migrate user data. More information about the new layout is available in the technical deep dive post. The new game layout is completely self contained and is fully ‘portable’. Nothing is registered system-wide and the files can simply be moved to another location, or deleted if you want to remove it. Please post any feedback or bugs in this forum. Thank you!

Give it a try now?

@Blyzzard From your screenshot we can tell that it's an issue with the accented e in the path you have the game installed (C:\Users\Médiacenter\...). The game historically has very poor support for international character sets, but some like é that are in the system ANSI codepage are normally ok. There is new code in the works to fix this and work with any valid Unicode characters in paths, but it's not hooked up to everything yet and a lot of the old code is still there. The last patch extended it to more of the game, but there was a problem with the interface between the old and new, which broke paths that used to work. There are two options you can choose to get it working. Wait until Tuesday when the patch containing the fix goes out. Move your whole installation to something outside of your user folder, that doesn't have the é character in it. You can choose (2) if you want something working right now, or just wait and it will be resolved soon.

I committed a change earlier today, based mostly on crash reports, that should fix this, though I have not been able to reproduce the crash myself so it's not 100% certain. Most likely it will be bundled with some other stability enhancements for a release on Tuesday to coincide with the normal maintenance window.

I found your crash report. It looks like you have a corrupt powers.bin file. Make sure you are using the official manifest (http://patch.savecoh.com/manifest.xml) and reverify files. If that doesn't work, check your data folder and temporarily move anything out of the way to make sure it's not conflicting.

The workaround in this release is only targeted at the driver used for the Intel HD 3000. If the Q45 driver doesn't load as a result of the same issue (i.e. it works when you shim it with the VersionLie compatibility database), I can add it to our workaround. I just need the name of the OpenGL driver dll you use for the shim and it would take 30 seconds to add to the list.

How comfortable are you with things like copying files into the wine root that Island Rum originally set up, and editing the script that you've been running (or preferably a copy of it)?

@DoctorBobThat was a new patch going out that should fix the issue for some of the people having problems with it installed in a root directory. You happened to get it in the window after the manifest had been updated but before all the download files had synchronized.

On the retail servers, Paragon typically ran the Halloween event for around 2 weeks. Last year we ran it for an entire month so that people would have more of a chance to trick or treat or do banner events without feeling rushed to cram it all in. There were a number of complaints that it being night all the time started to get really old after 3-4 weeks of it. So this year we're splitting the difference and breaking it up, keeping the extended event schedule but not running eternal night for longer than it was on retail.

Those who are getting the 'unable to locate game data files', where do you have the game installed? So far everyone on Discord who has reported that error has had it directly in the root of a drive rather than in a subfolder, which is... not recommended.

The windows 7 issue was that the default UDP receive buffer size on Windows 7 is too small. Initially we hadn't gone down that road because there were scattered reports of the same problem on 8/10, but all the ones we were able to verify turned out to be on 7 and were solved by adjusting the buffer. The Intel issue is a lot more complicated/annoying and I'm planning on doing a dev blog or similar post about it. People may find it interesting since our fix involves doing something Microsoft engineers say never to do under any circumstances.

Try adding -gfxreset to launch parameters. If it works, don't forget to take it out after.

Access denied is probably overzealous antivirus not liking that it's a new file with no reputation yet.

Heck, we still have a very small number of players running XP of all things.

@OrionX Click the Options button in Tequila and check the Launch Parameters box. Make sure it's empty, it looks like you somehow got a '[' bracket in it.

@FoulVileTerror It's not exploitable, just horribly inefficient. The current format saves one file per character per map - for example Player4872Map16.vm. There could be hundreds of files there, especially if you have a lot of alts. What happens is that the first time the game client needs to display the minimap, it loads the visitedmaps data into memory. ALL of it. Every map, for every character that's ever logged in on your computer, regardless of which account they belong to. On top of that, any time the data changes, say by uncovering a map square, the game writes the map data to disk. To avoid extra code I suppose they used textparser for this -- a .vm file is just a .bin file -- so every time this happens the game opens a new file, writes all the data for the map to it, then closes the file (which flushes it to disk). It does this synchronously, in the main thread. You had better hope Windows has write cache available or it will stall everything. It repeats this every time you uncover another square. The new implementation does things very differently. It creates one .vm2 file per character using a simple but effective binary format that can contain multiple maps and be expanded as needed. When map data is needed, only that single vm2 file is loaded. Instead of writing the changes synchronously, a background thread checks every 60 seconds if any maps have been updated, and if so flushes those maps to disk. The background thread keeps the file handle open and just seeks to the correct place. Finally, the current files are stored using the dbid of the character, which changes when you transfer shards and causes you to lose the uncovered maps. It's also subject to potential collisions with the same ID across shards, but that's likely rare. The new files can use suid instead - a unique identifier that we added to the database back in 2019 which moves along with the character. The version in this patch I think still uses dbid because the client doesn't yet receive the character's suid, but all the framework to transition to suids is there. In the next patch that will be activated and after that date, transferring across shards will retain the map state.

This. The new launcher was a good opportunity to clean up the file layout, and the setup looks quite a bit different. Some modifications to the client were needed to support it. As part of that effort, it was made fully portable so that you can just pick up the whole thing and move it. Dependence on the windows registry was removed, and any files being saved outside of the CoH directory structure were moved inside of it, including the shader cache and fog of war (VisitedMaps) data. There is compatibility with the old way of doing things (activated by -legacylayout) since some people will no doubt prefer to keep using their existing Tequila/IR installs. The one exception is VisitedMaps. Examination of the way it was being handled revealed it was... not good. Rather than maintain that format for compatibility mode, a new format was created that is used for both modes. Data is automatically migrated to the new format so the transition should be seamless.

Some powers are nerfed AND buffed. TW is a great example of that. The overall damage output has some of the "you paid real money for this set" damage bonus removed (I can't help but imagine a weapon enhancement that is someone taping dollar bills to a big sword). However, there's also a mechanics change to Momentum that is a fairly big deal QoL-wise and hopefully will make the set less frustrating to play.

Sure, you can run a shard for 50-100 people or so on a $150/mo dedicated server. However, the game wasn't designed for that and running everything on one box is a bit of a house of cards -- I promise you from experience it will crumble once you put any kind of serious load on it. Our smaller shards have several times that number online on a slow night. As Jimmy said, we don't run one shard per server. Not only is that a bad idea for reliability but it's also an inefficient use of hardware and it would cost more, assuming we could even find a single server large enough to host something like Excelsior without crashing and burning. We run a clustered multi-shard setup with shared global services that's much closer to the way the game was run when it was in retail. That takes a higher class of hardware. There are multiple people on the team with enterprise-level operations experience, and I'm confident that our costs (on a per-player basis) are quite a bit lower than a professionally hosted MMO would be, despite being close to that level in terms of server reliability.

And now you know who usually writes the patch notes.

@WanderingAries Thanks, if you do end up installing it and find that it doesn't work on Windows 10 with an 'OpenGL not found' error, shoot me a PM and I'll send you the details.

Is there anyone who has successfully used the shim method to get the game running on an Intel HD 3000 or similar chip, and is willing to help me test a different method that may be easier for new players to use? @Adurias or @LightningSmurfperhaps?

Wow, it sure feels like it. The 2020 time dilation field is in full effect.

Not sure who you talked to but like a year ago when this came up I said that it's easy to do (if you want to do hardcoded hacks anyway), but that we didn't think it was a good idea for game balance and weren't planning to do it.