-
Posts
6415 -
Joined
-
Last visited
-
Days Won
35
Content Type
Profiles
Forums
Events
Store
Articles
Patch Notes
Posts posted by Glacier Peak
-
-
-
That's wild. I'd try revalidating your game files in the Homecoming Launcher or sending a Debug Report on the next crash.
-
-
4 minutes ago, srmalloy said:
Forgot one: Don't get hit.
-
3
-
-
38 minutes ago, Cipher said:
OVH has a number of newer or refreshed offerings which would allow us to reduce our costs and address some of the issues we've had over the past two or so years while retaining close to the same capacity, and one of my goals is to create and execute on a plan to migrate to some of these by mid-2026, though that timeline may shift in either direction since we're a fully volunteer operation with a lot of moving pieces these days.
Oh thank you for the reply. Glad that's something being considered.
-
-
Can't you screen snip it?
-
Breakfrees
Teammates
Temp powers
Kill the enemy that causes mez before they can cast the mez power on you
-
3
-
-
You're on the right track - by exploring the zone, players uncover the purpose and story behind it.
-
Not needed
-
2
-
1
-
-
I played KOTOR 1 religiously when it came out. I only did one play through of KOTOR 2. I never played the MMO. I watched the trailer but I didn't really get any idea of what the game was, so I guess it's just a teaser?
-
Who's going to be the main villain??
-
On 1/23/2024 at 7:38 AM, Telephone said:
Hello! I'm Telephone, and you may remember me from previous technical posts such as
As many of you are likely aware, performance on Excelsior during peak times has been less than optimal for some time, and the recent increased player count since the license announcement has not improved things.
We've spent a lot of time over the last couple of years profiling the shards and even more over the last couple of weeks in an attempt to resolve the performance issues, but we're now at a point where we need to actually upgrade our hardware, or, to be more specific, our SQL host hardware.
The SQL Queue
The issues we are encountering are caused by what the City of Heroes™ server software calls the SQL Queue. Everything players do is ultimately committed to a back-end database. Under normal operation, most of these operations are both parallelized and asynchronous, and they commit quickly (within nanoseconds), but there are some operations which are much larger and take more time to perform.
There are also certain operations which have stricter timing requirements to maintain database integrity; these operations often come with what is called a barrier. When a barrier occurs, the entire SQL queue must be drained before continuing, meaning our normal large pool of asynchronous operations has to stop and wait on the barrier.
One of the biggest barrier culprits (until a very recent fix by @Number Six) was the disbanding of a large league, which is why you may have noticed at the end of a Hamidon raid the shard often seemed to lag for some time when the league was disbanded.
While we were able to find a solution to this barrier issue, there are other operations where the barrier can't be removed without significant rearchitecting. When the shard is very busy and there are a lot of other large operations taking place, a barrier can cause the entire shard to lag for several seconds, and this often gets into a vicious cycle where some of those other large operations may have their own barrier, or there may be database conflicts and the entire SQL operation is rolled back, rebuilt by the server, and sent to the database again with another barrier.
The queue does eventually drain, but it could be a period of many seconds or even minutes until everything finally unwinds. If it gets particularly bad, the shard may enter what's called Overload Protection, where (among other load-shedding measures) new logins are temporarily forced to queue, even though the shard has not reached its player limit.
Throw More Hardware At It
Homecoming has been in operation for nearly five years now, and in that time the size of our databases and their indices has grown significantly. Our existing primary North America SQL hosts (of which we have two) are legacy OVH Advance-3 servers, with Xeon D-2141 CPUs (8 cores, 16 threads, 2.2-2.7 GHz), 64 GiB of RAM, and two NVMe drives (in a mirrored configuration). Excelsior's database alone is over 100 GB in size, and during peak time sees many thousands of transactions per second, so we have simply outgrown the hardware we have been running on.
We're planning to upgrade both of them to the newest iteration of the OVH Advance-3, which is a Ryzen 5900X (12 cores, 24 threads, 3.8-4.7 GHz), 128 GiB of RAM, and four NVMe drives (in a RAID10 configuration). The main benefits we expect are that the doubling of RAM will hold many more indices in memory, and moving to four drives instead of two will double our I/O capacity.
We're upgrading the one Excelsior (and Everlasting) are on first, and if that works well we will upgrade the other one (hosting Torchbearer, Indomitable, and global services) next month.
Power Underwhelming?
It's possible (but unlikely) that even this upgrade would not be enough to resolve the issues on Excelsior. The most likely cause of this would be insufficient RAM; the newest model of OVH Advance-3 has a maximum of 128 GiB of RAM, so we would need to go up to an Advance-4 to get more RAM (this would also increase the number of cores, but at a lower clock speed). This would be a somewhat significant cost increase, but if it becomes necessary we will explore it.
There's also the possibility that the issue can't be resolved by more hardware; some of the SQL Queue problems are fundamental to the system design. We haven't stopped looking at fixes from the software side and while some of the barrier operations must remain barriers, there are potentially other fixes we can do to reduce load and database contention.
TL;DR
We believe our SQL hosts are no longer up to the task of handling our shards as they have grown, and need to upgrade their hardware.
We're planning to spend approximately $600 this month and $600 next month in one-time charges on upgrading our primary North America SQL hosts. Our ongoing costs will increase by about $250-$300 per month ($125-$150 per month per database host; the amount is a little difficult to calculate due to taxes and SQL licensing costs).
Are there any plans to upgrade hardware further or is the quality of the hardware been sufficient over the past two years?
-
It's the journey not the destination.
-
2
-
-
Quote from Captain Powerhouse on Open Beta regarding the crashing T9 changes:
Note on the Crashing T9 power changes:
We have decided to roll these changes back. At least at this point there are no plans to revisit them in the future. That of course might always be subject to change.
This applies exclusively to the crashing T9 changes, so specifically:
- Energy Aura > Overload
- Invulnerability > Unstoppable
- Ninjitsu > Kuji-In Retsu
- Radiation Armor > Meltdown
- Shield Defense > One With the Shield
- Super Reflexes > Elude
- Willpower > Strength of Will
Due to the holiday break, not sure when the rollback will be hitting the open beta server, so if anyone still wants to play around with the current versions for any reason you might have a few days to do so.
Other changes to any of the sets those powers are in will likely continue testing and iterating and we will continue to gather feedback. Do note nothing is considered final until it reaches release candidate status.
-
1
-
2 hours ago, Snarky said:
in a recent patch some well meaning soul changed the Soul Transfer icon (a universal Rez looking thing) to a Heal icon... which the set already has.
this created a bit of confusion and inconvenience on a recent Zoe i did for the badges on a dusty old Brute
there is no way this is a good idea, not should it stand. thank you.
This might sound rhetorical, but were you dead or alive when you saw the icon?
-
Indom is on the board!
-
3
-
-
Is there a list of specific costume pieces that are locked to specific genders? And would that list also have pictures to see what they are?
-
12 minutes ago, Azael said:
The two toons I deleted bc nothing was coming out of them are now going to get good buffs....
The toon I decided to level up just for fun is now considered trash....
Time to get a break.
Or wait until changes are actually published to the live shards before making decisions?
-
-
What do you use T9s for anyways?
-
Good diversity of opinion here, glad to see that.
-
-
29 minutes ago, Zappalina said:
Hi,
With all of the options available for the Labyrinth of Fog - it would be nice to have a tutorial or something similar maybe a person standing there that we can click like in outbreak where there's an explanation of how to use enhancements. I'm not suggesting missions - just either a person or item there that we could click for more information. I'm seeing all of the options here in the picture and do not know what "font" is when asking I was directed to the forums which is fine I'll go look it up when I have time.
However I'd rather just have someone explain it in game and I am sure there are a lot of players out there who do not access the boards and may wonder what they've come across as they're exploring the game.
Suggestion: Have a person, creature or object, spirit item etc. explaining what the labyrinth of fog is - (Maybe Charon or Vanth and Charu (or Charun) ) and then that explains what we are accessing, what the glowing portals are, what the center black box is and what those zones are
-Maybe if you want to make it more interactive and true to the myths, we give him a coin or something to pay the price to the ones who guard the pathways to the underworld.
-
2
-
1
-
2
-



World's Collide (PvP event)
in The Arena (PvP)
Posted
No travel powers? Movement is like the best part of PvP.