Is work on page 6 ongoing or stalled?

[golstat2003]

On 7/12/2020 at 6:11 AM, chi1701 said:

Would be nice to see what your planning/adjusting.

No it wouldn't. Let them work. They can let us know what's ready to go when it's on test.

 

Less forum drama that way.

[kenlon]

23 hours ago, golstat2003 said:

No it wouldn't. Let them work. They can let us know what's ready to go when it's on test.

 

Yes it would. It would also be a bad idea, because people would not be able to deal when there were delays/changes, but then again human beings are the reason we can't have nice things. . .

[SwitchFade]

More importantly, Regen needs a good nerf. Issue 6, Regen nerf!

[golstat2003]

10 hours ago, kenlon said:

 

Yes it would. It would also be a bad idea, because people would not be able to deal when there were delays/changes, but then again human beings are the reason we can't have nice things. . .

Pretty much what I meant when I said it wouldn't be nice. Not nice here on the forums when plans inevitably change.

[Myrmidon]

“Keep it secret. Keep it safe.”

[Auroxis]

On 7/10/2020 at 8:41 PM, Jimmy said:

We're internally testing much of Page 6 at the moment. I don't want to give a timeframe because people's situations can change very quickly at the moment, but it's definitely not never.

How long have these tests been ongoing? On what criteria are these internal testers being selected by?

[SwitchFade]

47 minutes ago, Auroxis said:

How long have these tests been ongoing? On what criteria are these internal testers being selected by?

Can I be on the nerf Regen testing team?!

 

Weeeeeee *nerf*

 

😋😁

[ImpousVileTerror]

"Internal Testing" usually means only the Devs themselves and maybe the GMs.

"Closed Beta" is when it's a matter of inviting players.

 

 . . . usually.  Not sure if the Homecoming Team is using different definitions, though, @Auroxis.  They can kind of play fast-and-loose with a lot of stuff and make it up as they go along.

 

Futher; in most development, Closed Betas are usually conducted with players who either align to the developers' vision, or have demonstrated a capacity for critical review and analysis of the content that is being tested.  From my experience here, Homecoming is a bit of a mixed bag, skewing toward critical analysis within the scope of the presumed vision.

Granted, there's been no shortage of closed betas where P.R. turned them in to a sweepstakes to try and drum up hype, but I tend to lump that in to marketing more than the actual development cycle, even if helpful (even crucial) information is generated from the testing process.

 

Edited July 19, 2020 by ImpousVileTerror
Extra information.

[Auroxis]

As far as I'm aware, development and the internal testing server is still being led by Leandro and the SCORE team. My concern is that the same people who played on and financed the SCORE server are the ones being given exclusive access to the early development pipeline, not just the SCORE devs and the Homecoming team.

[ImpousVileTerror]

Hmm, yes.  The SCoRE thing.  That is a bit of a prickly pear . . . 

I have also heard that SCoRE is now essentially a test bed for Homecoming, but I've not seen any corroborating evidence.  Full transparency would obviously be ideal, and while there are some definite elements supporting secrecy as a protective measure in regard to liability and litigation . . . yeah.  There's a lot more being obscured than seems strictly necessary.  It might just be overly cautious behaviour, or there could be something nefarious; we just don't know.

 

For now, we have Homecoming, the good mixed with the bad.  Encouraging the policy makers to be more open and to promote healthy community engagement is probably our best avenue right now.  We could hope for another whistle-blower if there are things untoward going on, but at our level down here . . .  we're powerless to invoke any sort of meaningful change beyond supportive encouragement to do the right thing(s) when and where possible.

[JayboH]

2 hours ago, Auroxis said:

As far as I'm aware, development and the internal testing server is still being led by Leandro and the SCORE team. My concern is that the same people who played on and financed the SCORE server are the ones being given exclusive access to the early development pipeline, not just the SCORE devs and the Homecoming team.

Curious; why would that be a concern - how would that affect development in a concerning way

[ImpousVileTerror]

Simply; a potential lack of diversity in the testing pool.  One of the reasons that systemic testing was often conducted the way it was with for-profit products that I've been attached to in the past was that there was a drive toward making sure that development received a healthy variety in the points of view.  And when it comes to a game like City of Heroes, ESPECIALLY on Homecoming with the express mission statement . . . well, more than ever, a diverse testing pool will naturally provide a better reflection of critical reception.

There is a risk, particularly with less experienced development teams*, that when inviting close friends and family to play-test that the developers get a skewed perspective on the work they're doing.  They accidentally interpret the small, insular sample size as something demonstrating a collective and more objective point of view.  I've been a part of small projects where "but my best friend loved it" was expressed in genuine shock when an aspect of the game was harshly rejected from the players upon release.

 

I'd point to the Double Inf Feature's removal as a prime example of this . . . EXCEPT that (once again) obfuscation of the development here with Homecoming makes it very difficult to come to a precise conclusion on how that came to pass.

 

* I have no idea what level of experience Homecoming actually has, since it's something else obscured by the level of privacy and secrecy we have here.  There are clearly talented individuals on the Team, and I would not want to undervalue them!  I just know that I would feel a lot more comfortable knowing what level of experience they genuinely have.  Past work history.  Past successes to build off of.  Past failures to learn from.  That sort of thing.

[Auroxis]

12 minutes ago, JayboH said:

Curious; why would that be a concern - how would that affect development in a concerning way

For the same reason you don't hire family and friends to critique your work. Very narrow feedback resulting in development cycles often disconnected from the userbase.

 

Also, if (big If) Leandro is still recieving donations to run his SCORE server, he's essentially providing early access to Homecoming code for money, incentivizing people to continue playing there and donating to SCORE instead of leaving completely to Homecoming. This could result in code staying there for a while to maintain interest rather than being deployed to HC beta.

 

Anyway, I don't want to derail this topic into conspiracy theories. Jimmy can clear things up if he wants.

[ImpousVileTerror]

Good point, @Auroxis.

I didn't really want to touch on the "fairness" argument, because I know that a lot of people already don't view it as reasonable and it would be an uphill battle with little gain . . . 

BUT you do bring up the potential "payola" concern.  Even if it's not the conscious and express intent of anyone involved, a "paid early access server" raises some additional concerns.

[JayboH]

1 minute ago, Auroxis said:

For the same reason you don't hire family and friends to critique your work. Very narrow feedback resulting in development cycles often disconnected from the userbase.

 

Also, if (big If) Leandro is still recieving donations to run his SCORE server, he's essentially providing early access to Homecoming code for money, incentivizing people to continue playing there and donating to SCORE instead of leaving completely to Homecoming. This could result in code staying there for a while to maintain interest rather than being deployed to HC beta.

 

Anyway, I don't want to derail this topic into conspiracy theories. Jimmy can clear things up if he wants.

1.  There are/were thousands, so stating it is narrow isn't true.

2.  This is assuming he is/was using it for income instead of hosting costs, which would also affect taxes, and is completely ignoring the possibility that people don't want to say goodbye to their hard-earned characters in the years and years the server existed, with character ports from before the shutdown as well.  This is also assuming this would speed up your own access to it which is a big conclusion jump in of itself.

[SwitchFade]

Uh... This spiraled into crazy territory...

 

*Poof*

[Jimmy]

Simply put, the decisions about what gets tested where and when are entirely dictated by the needs of the development team.

 

Some of Page 6 (mainly the balance stuff) is being tested on Resurgence (aka Closed Beta), primarily because the last time early experimental powers changes were tested on our public test server there was a large amount of upset when changes got scrapped or altered (which is par for the course when it comes to live game balance), and this severely reduced the efficacy of the testing process.

 

I refer to this as 'internal testing' because this (unfinished stuff that's nowhere near ready for prime time) would traditionally be tested internally if we actually had, y'know, a paid QA team - but obviously we don't, so this is the next best thing. Many Homecoming players who have provided constructive feedback during the last few rounds of powers testing are also taking part in this test. Once we're more confident in the changes then they'll hit the public beta server for more rigorous testing.

1 hour ago, Auroxis said:

...he's essentially providing early access to Homecoming code for money, incentivizing people to continue playing there and donating to SCORE instead of leaving completely to Homecoming. This could result in code staying there for a while to maintain interest rather than being deployed to HC beta.

This is the kind of crazy conspiracy theorizing I'm used to seeing from other CoH communities, not this one 🙂

 

Players on that server donate to keep it running because they want to play there, and they've been doing that for far longer than Page 6 has been in testing for. I'm sure they'll continue to do so once Page 6 is on the public test server.

[summers]

Thank you for the update Jimmy 🙂 I am so grateful to be able to play this wonderful game again and the team of volunteers are the real heroes!

[Pzn]

15 hours ago, Jimmy said:

Simply put, the decisions about what gets tested where and when are entirely dictated by the needs of the development team.

 

Some of Page 6 (mainly the balance stuff) is being tested on Resurgence (aka Closed Beta), primarily because the last time early experimental powers changes were tested on our public test server there was a large amount of upset when changes got scrapped or altered (which is par for the course when it comes to live game balance), and this severely reduced the efficacy of the testing process.

 

I refer to this as 'internal testing' because this (unfinished stuff that's nowhere near ready for prime time) would traditionally be tested internally if we actually had, y'know, a paid QA team - but obviously we don't, so this is the next best thing. Many Homecoming players who have provided constructive feedback during the last few rounds of powers testing are also taking part in this test. Once we're more confident in the changes then they'll hit the public beta server for more rigorous testing.

This is the kind of crazy conspiracy theorizing I'm used to seeing from other CoH communities, not this one 🙂

 

Players on that server donate to keep it running because they want to play there, and they've been doing that for far longer than Page 6 has been in testing for. I'm sure they'll continue to do so once Page 6 is on the public test server.

The secret server is still running and it's getting updates before homecoming? 

[Coyote]

2 hours ago, josh1622 said:

The secret server is still running and it's getting updates before homecoming?

 

"Getting updates" and "being part of internal testing" aren't the same thing. Usually, it's assumed that updates are the FINISHED product of testing (internal, then closed beta, then beta), therefore being part of internal testing is not synonymous with getting updates first.

The reason is that there will undoubtedly be changes that are tested, and end up going live. And some that are tested and don't go live. So players testing those are "wasting" their time on powers, powersets, or changes, that go *poof*. That's part of testing, but focusing on "so they see the success stories first" isn't fair without considering that they also "waste their time on what gets discarded".

 

IMO, I want to play, not test. If I wanted to test, I think there's a good chance that I may be able to get access to the closed beta if I were to ask... but I am aware of the downsides of being in the testing environment, and I am not interested. It's not all roses and glory, there's a lot of boring frustrating time finding bugs, detailing bugs, playing with a character who's going to be deleted even if I like it, etc. Envying the testers because they see the code first is not necessarily the right viewpoint. They also get to find the potholes and land mines on the highway before we drive on the semi-smooth pavement later.

 

Although, they do get to be the first to nerf Regen. That's a nice perk.

[ImpousVileTerror]

Some folks get a thrill out of being an active part of the development cycle, though, so testing -is- more fun for them.  Not necessarily all the time, but being involved in the process significantly outweighs and kind of sense of loss when test characters are inevitably deleted.  There's of course no discounting the positive emotional benefit of your testing work being a key component to the benefit of the whole community.  Some of us find -that- to be the most important part of playing this game.

 

I feel that a clear and codified explanation of the differences between Theoretical, Experimental, Intensive, and Final Approval testing would be helpful for everyone at all levels of development here.  Especially given the ease at which openness can now be achieved.  If clearer guidelines on the hows and whys of testing methodology, and unambiguous thresholds like those four categories I just listed are established, then I believe the (from my perspective, having seen -much- uglier turn outs) relatively mild backlash of the Dark Melee experiment could be significantly mitigated.  Thus giving the Development Team here the much needed focus from the testers while also simultaneously achieving a less homogenized and more productive test group.

 

Standard volunteer timetable criticism applies, but I've seen decades of success in codifying standards for these sorts of projects.  I 100% believe that getting those standards clearly established and communicated now will save massive amounts of work hours in the future.  The sooner we have them, and display them publicly, the better off we'll all be.

[Jimmy]

1 hour ago, Coyote said:

IMO, I want to play, not test. If I wanted to test, I think there's a good chance that I may be able to get access to the closed beta if I were to ask... but I am aware of the downsides of being in the testing environment, and I am not interested. It's not all roses and glory, there's a lot of boring frustrating time finding bugs, detailing bugs, playing with a character who's going to be deleted even if I like it, etc. Envying the testers because they see the code first is not necessarily the right viewpoint. They also get to find the potholes and land mines on the highway before we drive on the semi-smooth pavement later.

Yep. This isn't an Ubisoft marketing ‘beta’. It’s highly unfinished work that needs serious testing.

However, public testing is also really important, and it's vital that everyone engages in the testing process when patches hit the public beta server - and I'll explain why!

 

If we went straight to public testing then there would be a wave of interest (and testing) immediately - which may seem useful - but in actual fact it would be mostly wasted as everyone would be encountering the really obvious showstopping issues. All of the smaller, harder-to-find (but still important) issues would be eclipsed by the obvious ones. Finding those smaller issues generally requires a much larger amount of time (and therefore players), which is why we don't want to waste that large wave testing wave on the obvious problems. Splitting the testing phases up allows us to 'save' that large testing wave to be used when it'll be the most effective.

 

This isn't a marketing exercise designed to generate interest or engagement - and as cynical as it may seem, it's not about making testers happy. It's about helping the development team produce the best work possible for the live servers.

 

I know it's been said a lot, but it bears repeating: This is a volunteer operation. We're professionals operating in a decidedly unprofessional environment with zero resources and zero in-house testers, so this is about as atypical as it gets - and therefore not something you can compare to most other testing experiences you may have had.

 

With that in mind, we're quite happy with how it's worked so far. To date only been two serious issues that have slipped through the testing net:

1. Brute taunt auras not working in farms - this was on Homecoming's test server for months, but the players it affected didn't engage in testing, so the issue wasn't found
2. A really bad mapserver crashed caused by character transfers - Resurgence has only one shard, and our Beta server also only had one shard at the time, so it simply wasn't possible for this to be found

#1 tells us how important it is that players engage in testing every step of the way, and how important it is for us to focus public testing at the time it will do the most good. #2 tells us we should have two beta shards set up so we can test character transfers (which we now do 🙂).

[Bionic_Flea]

But Jimmy, why can't I test and play and play and test and have cake and eat it too?

[Jimmy]

2 minutes ago, Bionic_Flea said:

But Jimmy, why can't I test and play and play and test and have cake and eat it too?

You can play and test, but I'm not giving you any cake.

[Coyote]

12 minutes ago, Jimmy said:

You can play and test, but I'm not giving you any cake.

 

How about a compromise? Let him play-test the cake 😄 

Just warn him that it may have bugs in it 🙂

Edited July 20, 2020 by Coyote