Server down?

[Nferno]

I have to say, for $5K per month, I'd expect a little more responsivness from OVH. That's pretty shotty of them. Yikes. Makes me curious if they have a multitude of system problems at the moment beyond Homecoming which is inhibiting them from responding to any one customer quickly.

 

  

[ogmidas]

43 minutes ago, Zhym said:

 

When the servers are back online, I want to make a villain named OVH Tech Support, who never responds to any messages.  I'd level him up a bit then park him in Warburg where people can come by and beat the everloving crap out of him as therapy.  Consider it a public service. 

I just need to figure out the right AT for a "unresponsive tech support" villain.

 

LOL

[DoctorDitko]

It seems to be the internal networking between servers that is causing the issues. These can be tricky to track down. We had an entire cluster offlined because one node's InfiniBand card went bad, and it was spewing so much garbage to the IB switch that no real traffic could get through.

 

That said, OVH have a rep for good hardware, good connectivity, good value...

And crappy customer service. Think snooty French waiter (it's a French company).

[Luminara]

[jaywashere]

 

 

 

Telephone — Today at 12:33 AM

Update: We're currently waiting on ticket response from OVH. We've filed it and also called in to their support, but still have no ETA. Our current belief is that they will not respond for several more hours. We're looking at figuring out some alternate solutions, but expect that we will be down for at least another 6-12 hours. We apologize and we'll update you as soon as we know anything. To be 100% clear: Your characters and data are safe. Homecoming is in no danger and we will resume service ASAP.(edited)

 

Telephone — Today at 8:53 AM

Update: OVH assigned our ticket a little while ago. They've asked us to perform some testing which we are about to do.

 

Telephone — Today at 10:31 AM

Update: We've completed the testing OVH asked for and sent them all our results. We're now waiting on them to work on or escalate the issue.

 

Telephone — Today at 12:13 PM

Update: We're still waiting on a reply from OVH. In the meantime, we've got an alternate overlay networking technology we're working on setting up as a backup to vRack until the issue is fixed. We've just completed our first connection with it and are now in the process of rolling it out to our cluster.

 

Telephone — Today at 2:28 PM

Update: We're still waiting on a reply from OVH. In the meantime, we've also lost the ability to access one of our servers, so we've filed a second ticket for that server. We still have no ETA for when we will be up. We tried an alternate method of overlay networking in hopes that it would allow us to resume service, but unfortunately the first attempt was unsuccessful. We'll continue attempts to get this to work.

 

 

 

 

 

 

 

UPDATE ACCORDING TO DISCORD!

[GraspingVileTerror]

Well, not to undermine anyone who is chomping at the bit to get back in to the game, as I can absolutely appreciate that!  But I do want to remind folks to remember to take it easy and remain calm.  This isn't the sort of thing to get too worked up over, comparatively speaking.  It absolutely sucks when you allocate personal time for the game, and it just so happens that your plans get dashed against the rocky shores of Technical Difficulties.  But it's really beyond any of us to effect it.  I'd guess that worrying about that isn't going to make anyone feel any better.

 

I also really want to thank two types of heroes who are showing up in this thread (along with the Homecoming Team of volunteers, naturally).

The folks who are sharing the official update statements on the official forums on behalf of the Team.  Thank you!  It's appreciated immensely.  Please, continue to do so, as possible.

And the folks who are coming up with innovative ways to engage the community in creative and positive activities!  Special thanks to @KelvinKole for the offline costume contest!  Great idea!

[Blackbird71]

The downtime doesn't upset me.  Things happen, technology fails, and stuff has to be fixed.

What bothers me though is the apparent ~10 hours between HC submitting a ticket and OVH actually opening/assigning the ticket, which certainly seems like quite a gap for a company that boasts three different tech teams all providing 24/7 support.  Let's allow them to handle the immediate problem first, but I certainly hope that this response is addressed the next time CoH looks at their server arrangement and considers any changes or moves.

[damnimgood]

What we need...is a Karen. This would be fixed in under 15 minutes. Someone go grab a Karen off the shelf, unpack her,and put in the two double A batteries. Turn her on, (wait,what?) and let her go. I need my COH fix. I'm jonesing like a crackhead!!!

Edited June 22, 2021 by damnimgood

[Zhym]

1 hour ago, Marine X said:

1 hour ago, Zhym said:

AT for a "unresponsive tech support" villain

Petless Mastermind of course, his Pets even despise him, so they left.

Maybe Daemons/Time.  Except he'd need to use the Time powers to slow himself and his minions.

[Redbone1]

3 minutes ago, damnimgood said:

What we need...is a Karen. This would be fixed in under 15 minutes. Someone go grab a Karen off the shelf, unpack her,and put in the two double A batteries. Turn her on, (wait,what?) and let her go. I need my COH fix. I'm jonesing like a crackhead!!!

 

"I want to speak to your Arch-Villain, now!"

[ForceMajeure]

9 minutes ago, damnimgood said:

What we need...is a Karen. This would be fixed in under 15 minutes. Someone go grab a Karen off the shelf, unpack her,and put in the two double A batteries. Turn her on, (wait,what?) and let her go. I need my COH fix. I'm jonesing like a crackhead!!!

 

I know, or knew a guy in the UK that was together with an Elite piece of work, she told him off while he was on Teamspeak with all of us, and in that house, he did everything and payed for it. Sometime later he moved to Finland without telling her and started a new live with new hobbies and girls to hang out with. The woman's name in the UK, single mum with two kids? Well it was 'Karen'. I'm going to think about that moment till the day I die. 🤭 🙄 No one will ever be more "Karen" than her.

Edited June 22, 2021 by ForceMajeure

[RonJ73]

7 minutes ago, damnimgood said:

What we need...is a Karen. This would be fixed in under 15 minutes. Someone go grab a Karen off the shelf, unpack her,and put in the two double A batteries. Turn her on, (wait,what?) and let her go. I need my COH fix. I'm jonesing like a crackhead!!!

As a slightly inconvenienced gay man, I can take this role.  Who's manager do I need to call?

[Izuela]

This is because I didn't join that Terra Volta trial, isn't it?

[jaywashere]

UPDATE!!!!!!!!!!! ACCORDING TO DISCORD! please like this so that others can see it first! Thats how it works right?

 

Telephone — Today at 4:33 PM

Update: We've received an update from OVH. They are escalating our case to their networking department.

Edited June 22, 2021 by jaywashere

[RonJ73]

1 minute ago, jaywashere said:

UPDATE!!!!!!!!!!! ACCORDING TO DISCORD!

 

Telephone — Today at 4:33 PM

Update: We've received an update from OVH. They are escalating our case to their networking department.

Wow.  Only 16 hours after it was submitted.  Oy.

[Nerio72]

Any minute now they will call @Telephoneto ask if they have tried rebooting yet. And if its "plugged in".

[chi1701]

4 minutes ago, RonJ73 said:

Wow.  Only 16 hours after it was submitted.  Oy.

 

mean again, if you look at the link on discord, ticket was closed.

[migsbro]

Still down =(

Oh well it gave me a chance to check out the other private servers, rebirth and coxg, there were only like 10ppl online on each but I have to say the new powers/classes, pet/character customization is pretty sweet.

It does not beat the big population of Homecoming tho, I shall continue to wait.

[Psylenz0511]

9 minutes ago, Izuela said:

This is because I didn't join that Terra Volta trial, isn't it?

That Terra Volta Trial I was on is locked in limbo, heroes waiting for the servers to come back and resume the good fight just outside the Reactor building.

[Max Firepower]

5 minutes ago, jaywashere said:

UPDATE!!!!!!!!!!! ACCORDING TO DISCORD!

 

Telephone — Today at 4:33 PM

Update: We've received an update from OVH. They are escalating our case to their networking department.

I don’t work in IT, but is it likely that much happens at 10:30 pm in France, work-wise?

 

The people I know that do work in IT are support for various agencies/entities that demand immediate support, and contract/pay big money for it.  I am not disparaging anyone on HC for their time, effort, or focus.  I intended to ask if IT support commonly gets things done during night hours, I suppose.

[ForceMajeure]

https://www.youtube.com/watch?v=fQaavQNGsMY

 

I want a Refund!

[goatmonster]

I feel for the folks who missed their scheduled daily COH time. Thankfully I'm retired so I can just take a nap til the severs are back up.

And many thanks to the Homecoming team for all you do, it's very appreciated.

[damnimgood]

10 minutes ago, jaywashere said:

UPDATE!!!!!!!!!!! ACCORDING TO DISCORD! please like this so that others can see it first! Thats how it works right?

 

Telephone — Today at 4:33 PM

Update: We've received an update from OVH. They are escalating our case to their networking department.

Their first words are gonna be.."Did you try turning it off, and turning it on again?"

GUARANTEED.

[Oklahoman]

2 minutes ago, Max Firepower said:

I don’t work in IT, but is it likely that much happens at 10:30 pm in France, work-wise?

I used to work for a software company based in France, and a lot of the employees there go "on holiday" for extended periods of time this time of year. So, to add to your question, it's possible they are short staffed right now.

[Petraness]

I've only been on Homecoming for a week so I don't have any screenshots of my costume, but I would love to see a screenshot costume contest!  I love seeing other people's creativity!