Project Spelunker

[_NOPE_]

7/1/2019 6:45:32 PM | System.Exception: Failed to connect to server over three times in a row. Server may be down. Please try again later. ---> MySql.Data.MySqlClient.MySqlException: Authentication to host 'cityofplayers.com' for user 'wbenn_user' using method 'mysql_native_password' failed with message: User wbenn_user already has more than 'max_user_connections' active connections ---> MySql.Data.MySqlClient.MySqlException: User wbenn_user already has more than 'max_user_connections' active connections

 

 

 

I see ppl are jealous of my work done :)

 

Yes, this is an awesome error to have.

[_NOPE_]

This is from the Other folder, again...multiple times:

 

7/1/2019 9:44:30 PM | System.FormatException: Input string was not in a correct format.
   at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal)
   at System.Number.ParseInt32(String s, NumberStyles style, NumberFormatInfo info)
   at System.Int32.Parse(String s)
   at CoH_Forums_Parser.Program.FindUsersInThread(Int32 threadID, String content) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 412
   at CoH_Forums_Parser.Program.AddThread(String URL, Nullable`1 date, StringBuilder content) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 398
   at CoH_Forums_Parser.Program.ProcessContent(String URL, Nullable`1 date, StringBuilder content) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 280
   at CoH_Forums_Parser.Program.Main(String[] args) in C:\Projects\CoH Forums Parser\CoH Forums Parser\Program.cs:line 182

 

I just released a fix for this. It looks like there's some "junk" in some of the memberIDs stored, standard internet junk. I'm now splitting that memberID by newlines and taking the first actual number that I can parse, which seems correct because the junk has all been alphanumeric rather than numeric. This one requires an updated Parser, please get it from the OP.

[_NOPE_]

So, about that Error folder...it's putting some of the WARCs in there. Are they supposed to be or can we get rid of them? I'm at like 2Gb worth in the one folder alone.

 

Also, What's everybody's folder down to? I'm at 608Gb left.

 

Feel free to move those back into your main directory if you've grabbed the newest copy of the Parser.

[_NOPE_]

And... my internet got wiped out due to an electrical storm. I may not be able to do any more code updates today, because while I can use my Cell Phone as a hotspot at home, I'm not going to leave it at home to keep my connection working while I'm at work to remote in... so I won't be alot of help until tonight....

[Healix]

Awww, PK...so sorry that happened to you! *grabs nearest lightning bolt, ties in a knot, stomps on it, places in an empty air-tight mayonnaise jar, throws in dumpster*

[justicebeliever]

Awww, PK...so sorry that happened to you! *grabs nearest lightning bolt, ties in a knot, stomps on it, places in an empty air-tight mayonnaise jar, throws in dumpster*

 

PK - I appreciate all the hard work you and the team are doing here (+1 Inf for PK, Zep, Red, Aries)

 

Healix - you are going to really freak out the folks at the dump when that jars shatters.  Either that or some poor guy goings to be killed by thugs, tossed in the dumpster, and wake up with lightning powers.

 

Edit:  +1 to Amatyr as well.

[Symbolis]

Have the parser up and running, now. Yay!

 

Attached my error log thus far. Hope it's useful!

 

Errors.log

[_NOPE_]

I'm talking with my host now about the timeout errors.

[Zep]

I'm talking with my host now about the timeout errors.

 

I have a 4 day weekend coming up. If it doesnt cost you anything to increase capacity I'll get 200+ copies going before bed and watch it during the day -- can load up my i7 laptop too and we can get this done :)

[_NOPE_]

It would cost to increase capacity. Alot. They said to give it some time and see if things clear up. If it doesn't after a bit, I'm supposed to try to change the nameserver in the connectionString to the IP address instead to remove another layer of abstraction. I'm doubtful... I got a funny feeling they're starting to throttle me to get me to pay for a higher level package...

[_NOPE_]

Anyone knows a good web host that provides:

 

• ASP.NET
  
• MySQL Databases > 200GB in size
  
• Isn't super expensive? (hoping for $50 a month or less
  

 

I can't seem to find all of these things. Either my search-fu is weak, or such a thing doesn't exist.

[_NOPE_]

Have the parser up and running, now. Yay!

 

Attached my error log thus far. Hope it's useful!

 

And you're on the board Symbolis, thanks for your contribution!

 

[Symbolis]

#CrushingIt

 

:D

[Oubliette_Red]

Anyone knows a good web host that provides:

 

• ASP.NET
  
• MySQL Databases > 200GB in size
  
• Isn't super expensive? (hoping for $50 a month or less
  

I can't seem to find all of these things. Either my search-fu is weak, or such a thing doesn't exist.

 

I am not a search guru when it comes to servers, since I've never needed the use of one. The only thing I've been able find so far that came close is https://www.tier.net/infrastructure

 

[Zep]

Anyone knows a good web host that provides:

 

• ASP.NET
  
• MySQL Databases > 200GB in size
  
• Isn't super expensive? (hoping for $50 a month or less
  

 

I can't seem to find all of these things. Either my search-fu is weak, or such a thing doesn't exist.

 

Question: Please understand the last time I did a lot of coding networking meant a 2400 baud modem.

 

Could you make an agent program that reports what all parsers do on a single machine? This would change us from how many total parsers are supported to how many total users.

 

Of course, I am guessing that coding multiple programs to talk to each other could be annoying.... could all the parsing leave text files that a master program reads, sends, and deletes?

 

Of course I am not doing the work and it is not like this is going to take forever as is. 

 

Thank you again for all the work you do on this - as is is fine, just some random thoughts.

[_NOPE_]

That's not really feasible. Actually, the text parsing itself is pretty quick. It's the checks to the database, making sure the data doesn't already exist, and then creating the data that takes most of the time. And that can't really be improved by any way that I can think of in a modular way.

[Oubliette_Red]

I'm getting the server timeout error now, whether I parse a single file or multiples.

Errors.log

[_NOPE_]

Yeah, I might just have to bite the bullet and pay for a dedicated server...

[_NOPE_]

Okay, so I've tested and ruled out the clients and their communication. The stored procedures and methods that I've created to get the job done are now failing ON THE SERVER. I think that this database has gotten bigger than their hardware can support. Or at least the hardware that they've given me permission to use.

 

At this point, I'm considering just hosting my own server... hmmmm... I do have a spare PC... but then it'd rely on my internet connection, which wouldn't necessarily be always up... what do you all think?

[_NOPE_]

I've officially stopped all of your parsers from looping again by artificially raising the version number to a non-existent number until I get this figured out. No point in y'all wasting time and resources trying to get something working that won't. It looks like we've grown bigger than my current host at their current price range.

[Oubliette_Red]

GoFundMe? I'm currently seeking employment, but I could chip in a few inf for the duration.

[_NOPE_]

Hmm... I'm looking hard at Patreon, it seems GoFundMe is kind of designed for a one time thing, and it'd be nice if I could get the community to help pay for a long-term server...

[WanderingAries]

It looks like we've grown bigger than my current host at their current price range.

 

Yeah...I'm pretty good at taxing people/things. O:) :p

[justicebeliever]

Hmm... I'm looking hard at Patreon, it seems GoFundMe is kind of designed for a one time thing, and it'd be nice if I could get the community to help pay for a long-term server...

 

Well...you are a big part of the HC community...everyone who has helped with this is part of HC...even HC mods/devs are helping out...Can we pitch a case to HC to have them host the forums and include the overhead in their monthly goals? 

 

A.) They have a scalable hosting model

B.) Funding can be consolidated

C.) It doesn't increase their legal exposure...and if it's a static forum tool, the support would be minimal and supplied by yourself...

 

Thoughts?  I mean it's HC's call, but since we are supporting them, I feel we have a voice...

[Oubliette_Red]

Hmm... I'm looking hard at Patreon, it seems GoFundMe is kind of designed for a one time thing, and it'd be nice if I could get the community to help pay for a long-term server...

 

Well...you are a big part of the HC community...everyone who has helped with this is part of HC...even HC mods/devs are helping out...Can we pitch a case to HC to have them host the forums and include the overhead in their monthly goals? 

 

A.) They have a scalable hosting model

B.) Funding can be consolidated

C.) It doesn't increase their legal exposure...and if it's a static forum tool, the support would be minimal and supplied by yourself...

 

Thoughts?  I mean it's HC's call, but since we are supporting them, I feel we have a voice...

 

As I've always said "If you don't ask, then the answer is no."