Project Spelunker

[_NOPE_]

Oh yeah, we're in this for the long haul, thanks for all your efforts!

[WanderingAries]

So how is everyone so far ahead of me on the chart? Or are you guys a bunch of low-end-fruit cheaters? :p

[WanderingAries]

Well that's an interesting error. I decided to up the number of instances per drive to 10, but had to stop at 4 on the 2nd drive because I was getting some sort of "end of loop" error. I retried a couple times and kept getting the same error.

[Oubliette_Red]

He PK. I see the changes you did to the parser. I'm am now getting an error text file and associated .warc file moved to a sub-directory.

When I look in the error text files, I do not see a file mentioned in the error.  You original popup error (while not copy-able, listed the associated file right at the top. I have appended the numerical portion of the .warc file that was in the error sub-directory.

When it's only 1 file not a big issue, but one of my sub-folders had 3 files in it with 1 error file that has multiple entries and no indication as to which error belongs to which file.

 

 

Errors-24942-20120906-170948.log

Errors.log

[_NOPE_]

He PK. I see the changes you did to the parser. I'm am now getting an error text file and associated .warc file moved to a sub-directory.

When I look in the error text files, I do not see a file mentioned in the error.  You original popup error (while not copy-able, listed the associated file right at the top. I have appended the numerical portion of the .warc file that was in the error sub-directory.

When it's only 1 file not a big issue, but one of my sub-folders had 3 files in it with 1 error file that has multiple entries and no indication as to which error belongs to which file.

 

Well, that's just because I'm an idiot and was writing the filename to the console instead of the error file. This is what happens when you're trying to do too many things at once:

 

 

Fix published, but I didn't increment the version number for that minor bug. Get the new version if you want it.

[Oubliette_Red]

Fix published, but I didn't increment the version number for that minor bug. Get the new version if you want it.

 

[WanderingAries]

That line about file loop error, That's what I was seeing when trying to startup the 15th instance. In your Else statement

[_NOPE_]

That line about file loop error, That's what I was seeing when trying to startup the 15th instance. In your Else statement

Well, when you get the chance, can you attach to your post your whole error log? That'll help me troubleshoot.

[Oubliette_Red]

I grabbed the latest build and on one large file and I am see a new line popping up regularly in the console.

"Adding User <######> in thread <######>

 

[WanderingAries]

I don't think you want the whole log (it's not short and probably won't fit), but this is what's at the end multiple times.

 

7/1/2019 9:36:57 PM | System.Exception: Failed to connect to server over three times in a row. Server may be down. Please try again later. ---> MySql.Data.MySqlClient.MySqlException: Authentication to host 'cityofplayers.com' for user 'wbenn_user' using method 'mysql_native_password' failed with message: User wbenn_user already has more than 'max_user_connections' active connections ---> MySql.Data.MySqlClient.MySqlException: User wbenn_user already has more than 'max_user_connections' active connections
   at MySql.Data.MySqlClient.MySqlStream.ReadPacket()
   at MySql.Data.MySqlClient.Authentication.MySqlAuthenticationPlugin.ReadPacket()
   --- End of inner exception stack trace ---
   at MySql.Data.MySqlClient.Authentication.MySqlAuthenticationPlugin.AuthenticationFailed(Exception ex)
   at MySql.Data.MySqlClient.Authentication.MySqlAuthenticationPlugin.ReadPacket()
   at MySql.Data.MySqlClient.Authentication.MySqlAuthenticationPlugin.Authenticate(Boolean reset)
   at MySql.Data.MySqlClient.NativeDriver.Authenticate(String authMethod, Boolean reset)
   at MySql.Data.MySqlClient.NativeDriver.Open()
   at MySql.Data.MySqlClient.Driver.Open()
   at MySql.Data.MySqlClient.Driver.Create(MySqlConnectionStringBuilder settings)
   at MySql.Data.MySqlClient.MySqlPool.GetPooledConnection()
   at MySql.Data.MySqlClient.MySqlPool.TryToGetDriver()
   at MySql.Data.MySqlClient.MySqlPool.GetConnection()
   at MySql.Data.MySqlClient.MySqlConnection.Open()
   at CoH_Forums_Parser.Program.TripleCheckOpen(MySqlConnection c) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 963
   --- End of inner exception stack trace ---
   at CoH_Forums_Parser.Program.TripleCheckOpen(MySqlConnection c) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 974
   at CoH_Forums_Parser.Program.UpToDateVersion(String ProgramName) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 906
   at CoH_Forums_Parser.Program.Main(String[] args) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 89

[Oubliette_Red]

I don't think you want the whole log (it's not short and probably won't fit), but this is what's at the end multiple times.

 

Is the file too big to attach to your post?

[WanderingAries]

Is the file too big to attach to your post?

 

I'll try, it's a couple hundred Kb

 

Nope :p

 

"The following error or errors occurred while posting this message: The message exceeds the maximum allowed length (20000 characters)."

[Zep]

Just got home, downloaded the new parser -- it is showing a blank console with an occasional cursor blink in the upper left.

 

No errors, nothing -- I take that back as I typed no errors I saw some in the console and it closed.

 

Error exceeds the allowed 20k characters - shall I shove it in a drop box for you?

 

 

Here is the header:

 

7/1/2019 6:45:32 PM | System.Exception: Failed to connect to server over three times in a row. Server may be down. Please try again later. ---> MySql.Data.MySqlClient.MySqlException: Authentication to host 'cityofplayers.com' for user 'wbenn_user' using method 'mysql_native_password' failed with message: User wbenn_user already has more than 'max_user_connections' active connections ---> MySql.Data.MySqlClient.MySqlException: User wbenn_user already has more than 'max_user_connections' active connections

 

 

 

I see ppl are jealous of my work done :)

 

Error file is 525kb.

[WanderingAries]

This is from the Other folder, again...multiple times:

 

7/1/2019 9:44:30 PM | System.FormatException: Input string was not in a correct format.
   at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal)
   at System.Number.ParseInt32(String s, NumberStyles style, NumberFormatInfo info)
   at System.Int32.Parse(String s)
   at CoH_Forums_Parser.Program.FindUsersInThread(Int32 threadID, String content) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 412
   at CoH_Forums_Parser.Program.AddThread(String URL, Nullable`1 date, StringBuilder content) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 398
   at CoH_Forums_Parser.Program.ProcessContent(String URL, Nullable`1 date, StringBuilder content) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 280
   at CoH_Forums_Parser.Program.Main(String[] args) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 182

[WanderingAries]

User wbenn_user already has more than 'max_user_connections' active connections

 

That just means you're trying too hard and need to let us catch up. ;)

 

That and we're overloading PK's server. :p

[WanderingAries]

From the looks of it, there aren't a ton of errors, just lots of repeat errors. In the one file, the two examples above are repeated many times. So really, just two errors and the one is because of the max connection limit.

 

Just checked the other log and it's basically the same two errors.

[WanderingAries]

Did I JUST now realize we can attach files?

 

This is from my first folder:

 

/second file is too big

Errors.log

[Zep]

Did I JUST now realize we can attach files?

 

This is from my first folder.

 

Um... Cool

 

HOW DID YOU DO THAT!

 

 

------

 

As to other questions, I have all the data in one directory. I have had 18-20 copies running at a time though. :)

[WanderingAries]

Look JUST below the text field in the reply box. ;)

[WanderingAries]

So, about that Error folder...it's putting some of the WARCs in there. Are they supposed to be or can we get rid of them? I'm at like 2Gb worth in the one folder alone.

 

Also, What's everybody's folder down to? I'm at 608Gb left.

[Oubliette_Red]

 

Um... Cool

 

HOW DID YOU DO THAT!

 

 

------

 

As to other questions, I have all the data in one directory. I have had 18-20 copies running at a time though. :)

 

Either in the Reply or Preview option of posting, you'll see:

+ Attachments and other option

Click on it.

 

[WanderingAries]

So...mostly because I'm good at stressing anything out in life, I've pulled out 3 more drives to clone the data to and see if I can run more instances off of. I think I'm up to nearly 20 right now. All procs are set to real-time and the load is only 60Mb/instance.  It's a good thing my RGB fans have a simple control switch to cycle the lights off. 😋

[Zep]

So...mostly because I'm good at stressing anything out in life, I've pulled out 3 more drives to clone the data to and see if I can run more instances off of. I think I'm up to nearly 20 right now. All procs are set to real-time and the load is only 60Mb/instance.  It's a good thing my RGB fans have a simple control switch to cycle the lights off. &#128523;

 

So your the one. :)

 

There is a max number of total clients that can be run - not that it matters as long as the work gets done.

 

I am guessing we are at about 1/3 done.

[_NOPE_]

I grabbed the latest build and on one large file and I am see a new line popping up regularly in the console.

"Adding User <######> in thread <######>

 

Yes, that's the new table being populate that will allow us to connect threads to users.

[_NOPE_]

I don't think you want the whole log (it's not short and probably won't fit), but this is what's at the end multiple times.

 

7/1/2019 9:36:57 PM | System.Exception: Failed to connect to server over three times in a row. Server may be down. Please try again later. ---> MySql.Data.MySqlClient.MySqlException: Authentication to host 'cityofplayers.com' for user 'wbenn_user' using method 'mysql_native_password' failed with message: User wbenn_user already has more than 'max_user_connections' active connections ---> MySql.Data.MySqlClient.MySqlException: User wbenn_user already has more than 'max_user_connections' active connections
   at MySql.Data.MySqlClient.MySqlStream.ReadPacket()
   at MySql.Data.MySqlClient.Authentication.MySqlAuthenticationPlugin.ReadPacket()
   --- End of inner exception stack trace ---
   at MySql.Data.MySqlClient.Authentication.MySqlAuthenticationPlugin.AuthenticationFailed(Exception ex)
   at MySql.Data.MySqlClient.Authentication.MySqlAuthenticationPlugin.ReadPacket()
   at MySql.Data.MySqlClient.Authentication.MySqlAuthenticationPlugin.Authenticate(Boolean reset)
   at MySql.Data.MySqlClient.NativeDriver.Authenticate(String authMethod, Boolean reset)
   at MySql.Data.MySqlClient.NativeDriver.Open()
   at MySql.Data.MySqlClient.Driver.Open()
   at MySql.Data.MySqlClient.Driver.Create(MySqlConnectionStringBuilder settings)
   at MySql.Data.MySqlClient.MySqlPool.GetPooledConnection()
   at MySql.Data.MySqlClient.MySqlPool.TryToGetDriver()
   at MySql.Data.MySqlClient.MySqlPool.GetConnection()
   at MySql.Data.MySqlClient.MySqlConnection.Open()
   at CoH_Forums_Parser.Program.TripleCheckOpen(MySqlConnection c) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 963
   --- End of inner exception stack trace ---
   at CoH_Forums_Parser.Program.TripleCheckOpen(MySqlConnection c) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 974
   at CoH_Forums_Parser.Program.UpToDateVersion(String ProgramName) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 906
   at CoH_Forums_Parser.Program.Main(String[] args) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 89

 

That means y'all are overwhelming my server! That's actually a GOOD thing. If we keep that up, this list will be parsed in no time!