Project Spelunker

[_NOPE_]

And for today's update:

 

 

 

 

 

 

 

 

 

They never did... 😢 :

 

Discussion on the killing of Statesman:

 

 

 

 

And a post from one of my old (former) friends, memories.....

 

 

Edited August 2, 2021 by The Philotic Knight

[_NOPE_]

And it looks like an attempted hacker attack on the FTP server... IP address traces back to Mexico City... instaban!

 

 

[Ura Hero]

As a FYI, the last patches seem to have solved all of the errors I was getting.  It's plugging along quite well now.

[WanderingAries]

Another one of these from the i7, haven't checked the iMini yet today.

 

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-15354-20120904-213906.warc' ---> System.Exception: Failed to download file. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive.
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at System.Net.ConnectionPool.Destroy(PooledStream pooledStream)
   at System.Net.ConnectionPool.PutConnection(PooledStream pooledStream, Object owningObject, Int32 creationTimeout, Boolean canReuse)
   at System.Net.FtpWebRequest.FinishRequestStage(RequestStage stage)
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Abort(Exception e)
   at System.Net.CommandStream.CheckContinuePipeline()
   at System.Net.FtpDataStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)
   at System.Net.FtpDataStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at COH_WARC_Processor.FTPclient.Download(String sourceFilename, FileInfo targetFI, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 419
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Download(String sourceFilename, FileInfo targetFI, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 431
   at COH_WARC_Processor.FTPclient.Download(String sourceFilename, String localFilename, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 259
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 84
   --- End of inner exception stack trace ---

 

[TombTyrant]

So... the point of this is to... read the old forums?

[WanderingAries]

3 hours ago, TombTyrant said:

So... the point of this is to... read the old forums?

 

To make available the old forums for EASY navigation and archival reference. Links "should" work between the archive pages as they would on a live forum. Or so I believe that's how it will end up. The usefulness being that old methods, concepts, etc will be WAY more easily searched/rediscovered/shared with the current communities (not just HC).

Edited August 3, 2021 by WanderingAries

[WanderingAries]

I'm getting Really tempted to just wipe my Win10 drive and install Win11 on it to run these on the i7 bare hardware as most of the remaining errors are occurring on it (not the iMini's Win11 VMs). There 'is' an instance that has a last good date of early 8/1/21 though, so I'm not sure if that one is stuck or not.

 

As before, a reoccurring theme:

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-22075-20120904-162222.warc' ---> System.Exception: Failed to download file. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.FtpDataStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at COH_WARC_Processor.FTPclient.Download(String sourceFilename, FileInfo targetFI, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 414
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Download(String sourceFilename, FileInfo targetFI, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 431
   at COH_WARC_Processor.FTPclient.Download(String sourceFilename, String localFilename, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 259
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 84
   --- End of inner exception stack trace ---

 

[WanderingAries]

New from the iMini

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-26078-20120906-140957.warc' ---> System.Exception: Failed to download file. ---> System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.FtpDataStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at COH_WARC_Processor.FTPclient.Download(String sourceFilename, FileInfo targetFI, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 414
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Download(String sourceFilename, FileInfo targetFI, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 431
   at COH_WARC_Processor.FTPclient.Download(String sourceFilename, String localFilename, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 259
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 84
   --- End of inner exception stack trace ---

 

From the i7 (All basically the same)

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-28138-20120911-084842.warc' ---> System.Exception: Failed to upload file. ---> System.Net.WebException: The remote server returned an error: 227 Entering Passive Mode (208,95,210,183,39,204)
. ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 208.95.210.183:10188
   at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
   at System.Net.Sockets.Socket.Connect(EndPoint remoteEP)
   at System.Net.FtpControlStream.QueueOrCreateDataConection(PipelineEntry entry, ResponseDescription response, Boolean timeout, Stream& stream, Boolean& isSocketReady)
   at System.Net.FtpControlStream.PipelineCallback(PipelineEntry entry, ResponseDescription response, Boolean timeout, Stream& stream)
   at System.Net.CommandStream.PostReadCommandProcessing(Stream& stream)
   at System.Net.CommandStream.ContinueCommandPipeline()
   at System.Net.FtpWebRequest.TimedSubmitRequestHelper(Boolean async)
   at System.Net.FtpWebRequest.SubmitRequest(Boolean async)
   --- End of inner exception stack trace ---
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at System.Net.ConnectionPool.Destroy(PooledStream pooledStream)
   at System.Net.ConnectionPool.PutConnection(PooledStream pooledStream, Object owningObject, Int32 creationTimeout, Boolean canReuse)
   at System.Net.FtpWebRequest.FinishRequestStage(RequestStage stage)
   at System.Net.FtpWebRequest.GetRequestStream()
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 204
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 227
   at COH_WARC_Processor.FTPclient.Upload(String localFilename, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 108
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 136
   --- End of inner exception stack trace ---

 

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-19431-20120904-051104.warc' ---> System.Exception: Failed to upload file. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive.
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at System.Net.ConnectionPool.Destroy(PooledStream pooledStream)
   at System.Net.ConnectionPool.PutConnection(PooledStream pooledStream, Object owningObject, Int32 creationTimeout, Boolean canReuse)
   at System.Net.FtpWebRequest.FinishRequestStage(RequestStage stage)
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Abort(Exception e)
   at System.Net.CommandStream.CheckContinuePipeline()
   at System.Net.FtpDataStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)
   at System.Net.FtpDataStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 223
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 227
   at COH_WARC_Processor.FTPclient.Upload(String localFilename, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 108
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 136
   --- End of inner exception stack trace ---

 

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-28800-20120906-172735.warc' ---> System.Exception: Failed to upload file. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive.
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at System.Net.ConnectionPool.Destroy(PooledStream pooledStream)
   at System.Net.ConnectionPool.PutConnection(PooledStream pooledStream, Object owningObject, Int32 creationTimeout, Boolean canReuse)
   at System.Net.FtpWebRequest.FinishRequestStage(RequestStage stage)
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Abort(Exception e)
   at System.Net.CommandStream.CheckContinuePipeline()
   at System.Net.FtpDataStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)
   at System.Net.FtpDataStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 223
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 227
   at COH_WARC_Processor.FTPclient.Upload(String localFilename, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 108
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 136
   --- End of inner exception stack trace ---

 

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-25452-20120911-100442.warc' ---> System.Exception: Failed to upload file. ---> System.IO.IOException: Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.FtpDataStream.Write(Byte[] buffer, Int32 offset, Int32 size)
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 208
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 227
   at COH_WARC_Processor.FTPclient.Upload(String localFilename, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 108
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 136
   --- End of inner exception stack trace ---

 

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-20873-20120904-091324.warc' ---> System.Exception: Failed to upload file. ---> System.IO.IOException: Unable to write data to the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
   at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.FtpDataStream.Write(Byte[] buffer, Int32 offset, Int32 size)
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 208
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 227
   at COH_WARC_Processor.FTPclient.Upload(String localFilename, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 108
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 136
   --- End of inner exception stack trace ---

 

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-25631-20120905-131546.warc' ---> System.Exception: Failed to upload file. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive.
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at System.Net.ConnectionPool.Destroy(PooledStream pooledStream)
   at System.Net.ConnectionPool.PutConnection(PooledStream pooledStream, Object owningObject, Int32 creationTimeout, Boolean canReuse)
   at System.Net.FtpWebRequest.FinishRequestStage(RequestStage stage)
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Abort(Exception e)
   at System.Net.CommandStream.CheckContinuePipeline()
   at System.Net.FtpDataStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)
   at System.Net.FtpDataStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 223
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 227
   at COH_WARC_Processor.FTPclient.Upload(String localFilename, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 108
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 136
   --- End of inner exception stack trace ---

 

Edited August 5, 2021 by WanderingAries

[_NOPE_]

Any "failed to download" or "failed to upload" are just communications issues. Nothing we can do about that except try try again. After I get to work, I'll move all the files listed back to the input directory. 

[_NOPE_]

On 8/2/2021 at 10:54 PM, TombTyrant said:

So... the point of this is to... read the old forums?

The eventual "point" of this is to create a static copy of the forums that you can search, that (as Aries stated) you can navigate from page to page easily, and that the CoH community can use as a resource.

 

Now, as for "why", well if you really need to have "why" explained to you, I've got a whole monologue I could provide...

[WanderingAries]

Interesting development that I don't recall noticing in Win7 or Win10. Win11 is reporting my CPU boosting to a whopping 4.33 Ghz! I didn't realize that was the boost clock for this chipset as it reports as 3.50 Ghz under basically Any other circumstance. Now to see if running it on raw "unsupported" hardware is any different than the Win7 branch of this build. It broke my gadgets though, so I'm not happy about that as I now have to rely on task manager for data (not as detailed).

[WanderingAries]

We've crunched down below 10k now!

[TombTyrant]

12 hours ago, The Philotic Knight said:

The eventual "point" of this is to create a static copy of the forums that you can search, that (as Aries stated) you can navigate from page to page easily, and that the CoH community can use as a resource.

 

Now, as for "why", well if you really need to have "why" explained to you, I've got a whole monologue I could provide...

No. I already know why you'd do it. I just wanted to make sure I was understanding what you were working so hard on.

 

See if you can find that old tour of Sharkhead Isle I wrote for the forum newspaper.

[WanderingAries]

I decided to cut the VM instances in half as I didn't realize each one was trying to take 25-30%+ of the respective vCPU. Hopefully this will reduce the number of truly frozen instance and maybe go faster since they're not maxing each VM out this time. I still have to check on the i7 though as I didn't look at it from that perspective yet since it doesn't have the VM overhead. I did swap back to the Win7 drive though as it feels more stable than the Win11 hack I did.

 

Edit:

I'm gonna drop the i7 instances in half as well to give the CPU some breathing room. Taskmanager doesn't show the same way in Win7, so I'm not 100% sure how much each instance is taking.

Edited August 7, 2021 by WanderingAries

[WanderingAries]

The one lone error of the latest restart. SSDD

 

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-23423-20120905-164947.warc' ---> System.Exception: Failed to upload file. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive.
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at System.Net.ConnectionPool.Destroy(PooledStream pooledStream)
   at System.Net.ConnectionPool.PutConnection(PooledStream pooledStream, Object owningObject, Int32 creationTimeout, Boolean canReuse)
   at System.Net.FtpWebRequest.FinishRequestStage(RequestStage stage)
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Abort(Exception e)
   at System.Net.CommandStream.CheckContinuePipeline()
   at System.Net.FtpDataStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)
   at System.Net.FtpDataStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 223
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 227
   at COH_WARC_Processor.FTPclient.Upload(String localFilename, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 108
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 136
   --- End of inner exception stack trace ---

 

[WanderingAries]

If you haven't guessed, I just paste em because select all is muscle memory and faster than reading.  :p

 

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-12753-20120905-073400.warc' ---> System.Exception: Failed to upload file. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive.
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at System.Net.ConnectionPool.Destroy(PooledStream pooledStream)
   at System.Net.ConnectionPool.PutConnection(PooledStream pooledStream, Object owningObject, Int32 creationTimeout, Boolean canReuse)
   at System.Net.FtpWebRequest.FinishRequestStage(RequestStage stage)
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Abort(Exception e)
   at System.Net.CommandStream.CheckContinuePipeline()
   at System.Net.FtpDataStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)
   at System.Net.FtpDataStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 223
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 227
   at COH_WARC_Processor.FTPclient.Upload(String localFilename, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 108
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 136
   --- End of inner exception stack trace ---

 

[_NOPE_]

Over 30% done now, w00t! Keep it chugging, everyone!

 

[WanderingAries]

Next from the i7

 

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-19641-20120914-155328.warc' ---> System.Exception: Failed to upload file. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive.
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at System.Net.ConnectionPool.Destroy(PooledStream pooledStream)
   at System.Net.ConnectionPool.PutConnection(PooledStream pooledStream, Object owningObject, Int32 creationTimeout, Boolean canReuse)
   at System.Net.FtpWebRequest.FinishRequestStage(RequestStage stage)
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Abort(Exception e)
   at System.Net.CommandStream.CheckContinuePipeline()
   at System.Net.FtpDataStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)
   at System.Net.FtpDataStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 223
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 227
   at COH_WARC_Processor.FTPclient.Upload(String localFilename, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 108
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 136
   --- End of inner exception stack trace ---

 

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-27559-20120906-150821.warc' ---> System.Exception: Failed to upload file. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive.
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at System.Net.ConnectionPool.Destroy(PooledStream pooledStream)
   at System.Net.ConnectionPool.PutConnection(PooledStream pooledStream, Object owningObject, Int32 creationTimeout, Boolean canReuse)
   at System.Net.FtpWebRequest.FinishRequestStage(RequestStage stage)
   at System.Net.FtpWebRequest.SyncRequestCallback(Object obj)
   at System.Net.CommandStream.Abort(Exception e)
   at System.Net.CommandStream.CheckContinuePipeline()
   at System.Net.FtpDataStream.System.Net.ICloseEx.CloseEx(CloseExState closeState)
   at System.Net.FtpDataStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 223
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 227
   at COH_WARC_Processor.FTPclient.Upload(String localFilename, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 108
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 136
   --- End of inner exception stack trace ---

 

 

I'm currently pondering the subscription on Parallels as they just released v17 that's supposed to be compatible with Monterey and Win11 now (without hacks!), so we'll see if that impacts that workload at all. I'm kinda miffed as have already semi-recently paid for the update on the desktop version instead and it feels like a double dip. NeWay, time to spin-up the replacement instances.

[WanderingAries]

We're approaching 40% done it seems. I toyed with Parallels a bit and managed to convert to a single VM instance with full access to all the cores and RAM. Normally, this would be a bad idea, but RAM isn't an issue here and this isn't spiking all the cores either. I think the Mac will manage it better this way...or at least it will until the 14 day trial runs out on the v17 I've told it to run. God I wish I'd Sprung for the larger storage.

[WanderingAries]

Encore'

 

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-15807-20120905-055503.warc' ---> System.Exception: Failed to download file. ---> System.IO.IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.FtpDataStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at COH_WARC_Processor.FTPclient.Download(String sourceFilename, FileInfo targetFI, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 414
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Download(String sourceFilename, FileInfo targetFI, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 431
   at COH_WARC_Processor.FTPclient.Download(String sourceFilename, String localFilename, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 259
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 84
   --- End of inner exception stack trace ---

 

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-14357-20120905-032448.warc' ---> System.Exception: Failed to upload file. ---> System.Net.WebException: Unable to connect to the remote server
   at System.Net.FtpWebRequest.GetRequestStream()
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 204
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 227
   at COH_WARC_Processor.FTPclient.Upload(String localFilename, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 108
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 136
   --- End of inner exception stack trace ---

 

[WanderingAries]

Here's this morning's error from the i7. I did find out that the iMini's Win11 VM must have done an update because I had to log back into windows when I checked on it today, so who knows how long that was idle.

 

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-25166-20120907-055710.warc' ---> System.Exception: Failed to handle large WARC file ---> System.UnauthorizedAccessException: Access to the path 'C:\Users\Nathan\AppData\Local\Temp\CoH_Forums_Output\2\http_img696.imageshack.us_img696_2375_screenshot120113071824s.jpg' is denied.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.File.InternalWriteAllBytes(String path, Byte[] bytes, Boolean checkHost)
   at WHBennett.WarcExtractionRecord.StreamWARCFile(String sourceFileName, String outputDirectory, ToolStripStatusLabel& tssl) in C:\Projects\COH WARC Handler\WHB WARC Handler\WarcRecord.cs:line 209
   --- End of inner exception stack trace ---
   at WHBennett.WarcExtractionRecord.StreamWARCFile(String sourceFileName, String outputDirectory, ToolStripStatusLabel& tssl) in C:\Projects\COH WARC Handler\WHB WARC Handler\WarcRecord.cs:line 261
   at WHBennett.WarcExtractionRecord..ctor(String sourceFileName, String outputDirectory, ToolStripStatusLabel& tssl) in C:\Projects\COH WARC Handler\WHB WARC Handler\WarcRecord.cs:line 33
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 116
   --- End of inner exception stack trace ---

 

[WanderingAries]

Past the halfway mark now! Had to force restart the iMini like I've done in the past for some reason. Need to figure out what  causes it to get stuck asleep.

[WanderingAries]

Seems I get more errors on the Win7 overall. Here's what I found today before cycling the system (since it's been on for a few days).

 

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-24084-20120906-052656.warc' ---> System.Exception: Failed to upload file. ---> System.Net.WebException: Unable to connect to the remote server
   at System.Net.FtpWebRequest.GetRequestStream()
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 204
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 227
   at COH_WARC_Processor.FTPclient.Upload(String localFilename, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 108
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 136
   --- End of inner exception stack trace ---

 

There was an error in the application. Please copy and paste the following text in a message to the program author:
System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-12991-20120904-213257.warc' ---> System.Exception: Failed to upload file. ---> System.Net.WebException: Unable to connect to the remote server
   at System.Net.FtpWebRequest.GetRequestStream()
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 204
   --- End of inner exception stack trace ---
   at COH_WARC_Processor.FTPclient.Upload(FileInfo fi, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 227
   at COH_WARC_Processor.FTPclient.Upload(String localFilename, String targetFilename) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 108
   at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 136
   --- End of inner exception stack trace ---

 

As for the progress, we're at +57% and nearly down to 5k files.

[_NOPE_]

Thanks for that Aries, I've moved the files around as necessary. Current project status - nearly 60% done!

 

 

After these are all done, I'll go ahead and unzip them all into my current public facing web server, and make a "mini-app" to just go through the raw file list and index every file into a sitemap HTML file and submit to Google for processing. Then things will really start to show up and be searchable. I'll also see if I can zip up the whole thing into a single 7z archive at maximum compression and provide it to everyone as a "Phase 1" file. If there's enough interest, I can make it as a torrent. Then from there, anyone can do anything with these files that they want to on their own, as I move onto my own Phase 2 - removing duplicate documents, and processing the HTML files themselves to swap out all boards.cityofheroes.com or boards.cityofvillains.com references to the relative local path references to the files that can be found in the archive.

 

THAT will in fact take even LONGER than this phase, as it'll have to be on a file-by-file basis, actually scanning all links in each page, and then comparing them to what's out there on the file system (using my own SanitizeFileName method on both sides of the equation of course to make things match up properly). Not sure how I'd even begin to crowd source that thing, as it would create "race conditions" even more than the current process...

[WanderingAries]

Now approaching 65% and we're below 5k remaining