WanderingAries

This was my thought as I thought it was dif based TBH. I "think" I've seen as many as say 1:10-15 standard NPCs. I know one mish had like 5+, but it was laden with counsel. Found this, it has numbers.

Well when half the server is dry-humping Ms. Liberty...things start to tilt for sure. If they'd be kind enough to lose the pets and AoEs, then I'd bet that'd have an impact, but overall there's just so many people. AP often has 5 instances compared to like 2-3 for the other zones.

Yeah, I remember (FF?) back in OG where snipers were just as bad in general as Qs have always been for squids. You'd be flying a Little too close to a building and BAM, down to the city streets you go.

I wish this could be done with the base items in such a way that they could retain the same sorting as the editor.

Yeah...I'm pretty good at taxing people/things. O:) :p

So...mostly because I'm good at stressing anything out in life, I've pulled out 3 more drives to clone the data to and see if I can run more instances off of. I think I'm up to nearly 20 right now. All procs are set to real-time and the load is only 60Mb/instance. It's a good thing my RGB fans have a simple control switch to cycle the lights off. 😋

So, about that Error folder...it's putting some of the WARCs in there. Are they supposed to be or can we get rid of them? I'm at like 2Gb worth in the one folder alone. Also, What's everybody's folder down to? I'm at 608Gb left.

Look JUST below the text field in the reply box. ;)

Did I JUST now realize we can attach files? This is from my first folder: /second file is too big Errors.log

From the looks of it, there aren't a ton of errors, just lots of repeat errors. In the one file, the two examples above are repeated many times. So really, just two errors and the one is because of the max connection limit. Just checked the other log and it's basically the same two errors.

That just means you're trying too hard and need to let us catch up. ;) That and we're overloading PK's server. :p

This is from the Other folder, again...multiple times: 7/1/2019 9:44:30 PM | System.FormatException: Input string was not in a correct format. at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal) at System.Number.ParseInt32(String s, NumberStyles style, NumberFormatInfo info) at System.Int32.Parse(String s) at CoH_Forums_Parser.Program.FindUsersInThread(Int32 threadID, String content) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 412 at CoH_Forums_Parser.Program.AddThread(String URL, Nullable`1 date, StringBuilder content) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 398 at CoH_Forums_Parser.Program.ProcessContent(String URL, Nullable`1 date, StringBuilder content) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 280 at CoH_Forums_Parser.Program.Main(String[] args) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 182

I'll try, it's a couple hundred Kb Nope :p "The following error or errors occurred while posting this message: The message exceeds the maximum allowed length (20000 characters)."

I don't think you want the whole log (it's not short and probably won't fit), but this is what's at the end multiple times. 7/1/2019 9:36:57 PM | System.Exception: Failed to connect to server over three times in a row. Server may be down. Please try again later. ---> MySql.Data.MySqlClient.MySqlException: Authentication to host 'cityofplayers.com' for user 'wbenn_user' using method 'mysql_native_password' failed with message: User wbenn_user already has more than 'max_user_connections' active connections ---> MySql.Data.MySqlClient.MySqlException: User wbenn_user already has more than 'max_user_connections' active connections at MySql.Data.MySqlClient.MySqlStream.ReadPacket() at MySql.Data.MySqlClient.Authentication.MySqlAuthenticationPlugin.ReadPacket() --- End of inner exception stack trace --- at MySql.Data.MySqlClient.Authentication.MySqlAuthenticationPlugin.AuthenticationFailed(Exception ex) at MySql.Data.MySqlClient.Authentication.MySqlAuthenticationPlugin.ReadPacket() at MySql.Data.MySqlClient.Authentication.MySqlAuthenticationPlugin.Authenticate(Boolean reset) at MySql.Data.MySqlClient.NativeDriver.Authenticate(String authMethod, Boolean reset) at MySql.Data.MySqlClient.NativeDriver.Open() at MySql.Data.MySqlClient.Driver.Open() at MySql.Data.MySqlClient.Driver.Create(MySqlConnectionStringBuilder settings) at MySql.Data.MySqlClient.MySqlPool.GetPooledConnection() at MySql.Data.MySqlClient.MySqlPool.TryToGetDriver() at MySql.Data.MySqlClient.MySqlPool.GetConnection() at MySql.Data.MySqlClient.MySqlConnection.Open() at CoH_Forums_Parser.Program.TripleCheckOpen(MySqlConnection c) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 963 --- End of inner exception stack trace --- at CoH_Forums_Parser.Program.TripleCheckOpen(MySqlConnection c) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 974 at CoH_Forums_Parser.Program.UpToDateVersion(String ProgramName) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 906 at CoH_Forums_Parser.Program.Main(String[] args) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 89

That line about file loop error, That's what I was seeing when trying to startup the 15th instance. In your Else statement

Well that's an interesting error. I decided to up the number of instances per drive to 10, but had to stop at 4 on the 2nd drive because I was getting some sort of "end of loop" error. I retried a couple times and kept getting the same error.

So how is everyone so far ahead of me on the chart? Or are you guys a bunch of low-end-fruit cheaters? :p

Pretty sure there's a GPU thread or five on here talking about this. One is even a red letter thread I believe.

Did we not have this at one time REALLY early on?

Got home to find the sparser had been updated and it caused a version failure error, so both threads were shut down. I cloned the archive to a second drive and I'm running a thread per drive since they're mechanical.

Yeah, I'm really not sure what I was even looking for. I'm still not sure what the parser is doing per say, but it seems the 700+Gb has gone down a bit, so I'm guessing it's stripping stuff away, reporting whatever you're looking for, and then deleting the processed file. I tried running the app twice in the same folder, but that gave an error which likely sums up to it not wanting to run a 2nd instance. So instead I went and changed the apps Priority to Realtime. That added about 20% to the memory usage, a tad bit of CPU time, and sped up the scrolling a bit. Any general guess as to how long this will take to process now that there's like 6 of us and somebody's already munched a ton? Overall it looks like we've knocked out like <3k files of <19k.

IDK why, but I was expecting a UI of sorts instead of just a cmd window. So I just pointed it at the folder where all the files were and left it be. How long will That take? :p

I get an error when unpacking these four files. I had a fifth one that gave errors, but it unpacked fine when I did it individually. These continued with the error. Something about data beyond the package? boards.cityofheroes.com-threads-range-12708-20120905-012201.warc.gz boards.cityofheroes.com-threads-range-13957-20120905-104405.warc.gz boards.cityofheroes.com-threads-range-17737-20120905-012913.warc.gz boards.cityofheroes.com-threads-range-24525-20120905-105800.warc.gz I've isolated them into an error folder for now and think the rest can be deleted now that I've decompressed them in leu of running your script.

So the original direct DL finished sometime before work today. I then had WinZip unzip it all. I got home to find that I only needed to unzip a 2nd time? I'll reread the instructions, but I thought you said there was like 7 layers of unzipping. Unless that was just the Torrent to keep it thin? Also, getting errors as 7Zip unpacks the current group (2nd layer). I've an hour left to unpack this group and then I'll check things again.

The others have a point for "bang for your buck", but I've just been stacking that with the newer 2XP (thanks to @LadyDee for nudging me into it) to boost me through the first dozen powers or so.