WanderingAries Posted June 11, 2021 Share Posted June 11, 2021 Decided to throw the M1 iMini into the ring as well. A little overhead involved because of parallels, but I'm running 4 for now until the Insider preview updates, then I'll push to 8. 1 OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 11, 2021 Author Share Posted June 11, 2021 Y'all are doing awesome, last check we already had over 20% done already! w00t! Now, I DID have to do a restart on the server this morning, to try to troublshoot some issues I've been having with setting up IIS all on my own, so you may want to check your processors for errors, and restart them if they have errors. That's due to my inexperience as a webmaster (I'm naturally a desktop app code monkey, just inching my way into the web world). I already have the appropriate subdomain setup and pointing to the right IP address, but when you try to go to the site, for some reason it's not picking up my default document, giving me a 404 error, which is really weird. Actively troubleshooting now: I'm out. Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 11, 2021 Author Share Posted June 11, 2021 Nevermind, I figured it out. Silly me, not realizing that my PUBLISHED web.config actually OVERWRITES whatever settings I try to set in IIS Configuration Manager. Go figure, and something good to remember going forward. I'll have to keep in mind that to change any "IIS" settings, I'll have to go google how to change them in the web.config file rather than being able to use a nice GUI. Basic page is now up, but for now, there's nothing to see: I'm going to build a mini-app to start extracting the zip files into the directory that that link goes to, so that you all will be able to see some of the progress so far yourselves. The final site won't look anything like this, trust me. This is just the temporary holding place while I work on the other phases to get this thing up and operational while phase 1 is still processing. What I'm picturing here is a basic front page that contains a search bar, as well as a list of links to the various "index" pages, to allow people to browse the painful way file-by-file if they want to. That list of links is really mostly for Google though, so that they have something to start from for their spider to start crawling. Most people will want to use the Search Feature. My final goal, ultimately, is to have it look JUST as it does on the Wayback engine, with the main subdomain by default loading a version of the boards front page that contains a special search bar at the top, but from there, people should theoretically be able to click around like they used to be able to way back in the day, with the realization that some of the links will go nowhere because external sites no longer exist, and that obviously the former "live" functionality is no longer usable, because, it's an archival state. If I get REALLY smart later, I'll be able to figure out how to remove those elements altogether, but for now, I'll just be happy if I can modify the HTML pages to point to each other instead of their original link destinations that no longer exist! I'm out. Link to comment Share on other sites More sharing options...
WanderingAries Posted June 11, 2021 Share Posted June 11, 2021 Did you sort out how to get the applet to restart the process when it errors out? OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 11, 2021 Author Share Posted June 11, 2021 1 minute ago, WanderingAries said: Did you sort out how to get the applet to restart the process when it errors out? If it errors out because it can't download the file, it should just loop and try to pickup the next file. If it errors out for any other reason, then I need to look at the errors and troubleshoot them. I'm out. Link to comment Share on other sites More sharing options...
WanderingAries Posted June 12, 2021 Share Posted June 12, 2021 7 hours ago, The Philotic Knight said: If it errors out because it can't download the file, it should just loop and try to pickup the next file. If it errors out for any other reason, then I need to look at the errors and troubleshoot them. I'll have to read the errors more closely, but whatever they are, they're pretty common and cause me to have to close them and start a new instance. OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 12, 2021 Author Share Posted June 12, 2021 Once you get an error, please copy and paste the entire text of the error to this thread, and I'll investigate. If you happen to have the filename of the file that you were processing as well, that would be good. I'm out. Link to comment Share on other sites More sharing options...
WanderingAries Posted June 12, 2021 Share Posted June 12, 2021 (edited) 1 hour ago, The Philotic Knight said: Once you get an error, please copy and paste the entire text of the error to this thread, and I'll investigate. If you happen to have the filename of the file that you were processing as well, that would be good. This is what I got already. The format is always the same for these errors, but IDK about the content: There was an error in the application. Please copy and paste the following text in a message to the program author: System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-22289-20120903-205756.warc' ---> System.IO.FileNotFoundException: Could not find file 'C:\Users\USERNAME\AppData\Local\Temp\CoH_Forums_Output\https_accounts.google__servicemailpassivetruermfalsecontinuehttps_mail.google.com_mail_ui3D226ik3Debd018fc9626view3Datt26th3D12b563ab717b036626attid3D0.126disp3Dinline26realattid3Df_gem5zqoe126zwscc1ltmpldefaultltmplcache2.com_ServiceLogin'. at Ionic.Zip.SharedUtilities.GetFileLength(String fileName) in C:\projects\dotnetzip-semverd\src\Zip.Shared\Shared.cs:line 43 at Ionic.Zip.ZipEntry.MaybeUnsetCompressionMethodForWriting(Int32 cycle) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 800 at Ionic.Zip.ZipEntry.WriteHeader(Stream s, Int32 cycle) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 1078 at Ionic.Zip.ZipEntry.Write(Stream s) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 2220 at Ionic.Zip.ZipFile.Save() in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipFile.Save.cs:line 168 at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 101 --- End of inner exception stack trace --- When it does this, it basically stops all work because it wants you to report the error. It'll happen intermittently, so it's easy to have a bunch of idle processes and not realize it. Also, this round seems to be less stressful on the cores as they're not maxing out on my 3770k with 8 threads, but my M1's pretty maxed with the 8 threads like it was before (remember, it is doing the work inside 2 Win10 VMs). Edited June 12, 2021 by WanderingAries OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
KauaiJim Posted June 12, 2021 Share Posted June 12, 2021 I too occasionally get these errors. Is it possible when running multiple files at the same time that a file could be common to more than one and on e it gets deleted by the first instance, the next instance is confused when it isn’t there any longer (a timing thing resulting from running multiple extractions at the same time)? Also fyi that I shut down during the peak electric hours as my local utility has me mistaken for a rich man. Lol Will be booting back up again shortly. 😀 Want to see my current list of characters? Want to know more about me than you ever wanted to know? Wish Granted! Check out the 'About Me' in my profile: KauaiJim - Homecoming (homecomingservers.com) Link to comment Share on other sites More sharing options...
WanderingAries Posted June 12, 2021 Share Posted June 12, 2021 1 minute ago, KauaiJim said: Also fyi that I shut down during the peak electric hours as my local utility has me mistaken for a rich man. Lol Well that explains why it looks like nobody else was doing anything at the time. Assuming I'm actually reading the bottom border of these windows correctly. 1 OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
WanderingAries Posted June 12, 2021 Share Posted June 12, 2021 (edited) Got another one. I haven't compared them though. There was an error in the application. Please copy and paste the following text in a message to the program author: System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-21841-20120905-200546.warc' ---> System.IO.FileNotFoundException: Could not find file 'C:\Users\USERNAME\AppData\Local\Temp\CoH_Forums_Output\https_accounts.google__servicemailpassivetruermfalsecontinuehttps_mail.google.com_mail_ui3D226ik3Df1581e923526view3Datt26th3D12f3b32ff1b1144226attid3D0.126disp3Dinline26zwscc1ltmpldefaultltmplcache2.com_ServiceLogin'. at Ionic.Zip.SharedUtilities.GetFileLength(String fileName) in C:\projects\dotnetzip-semverd\src\Zip.Shared\Shared.cs:line 43 at Ionic.Zip.ZipEntry.MaybeUnsetCompressionMethodForWriting(Int32 cycle) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 800 at Ionic.Zip.ZipEntry.WriteHeader(Stream s, Int32 cycle) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 1078 at Ionic.Zip.ZipEntry.Write(Stream s) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 2220 at Ionic.Zip.ZipFile.Save() in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipFile.Save.cs:line 168 at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 101 --- End of inner exception stack trace --- Funny story, I must have accidentally clicked verbose mode on one of them because I started seeing a constantly scrolling window and was like "WTF is going on?". :p Edited June 12, 2021 by WanderingAries OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
WanderingAries Posted June 12, 2021 Share Posted June 12, 2021 Encore? There was an error in the application. Please copy and paste the following text in a message to the program author: System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-11164-20120905-033118.warc' ---> System.IO.FileNotFoundException: Could not find file 'C:\Users\USERNAME\AppData\Local\Temp\CoH_Forums_Output\http_boards.cityofheroes.com_styles_CoH_misc_background_renamed_70.jpg'. at Ionic.Zip.SharedUtilities.GetFileLength(String fileName) in C:\projects\dotnetzip-semverd\src\Zip.Shared\Shared.cs:line 43 at Ionic.Zip.ZipEntry.MaybeUnsetCompressionMethodForWriting(Int32 cycle) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 800 at Ionic.Zip.ZipEntry.WriteHeader(Stream s, Int32 cycle) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 1078 at Ionic.Zip.ZipEntry.Write(Stream s) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 2220 at Ionic.Zip.ZipFile.Save() in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipFile.Save.cs:line 168 at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 101 --- End of inner exception stack trace --- OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
WanderingAries Posted June 12, 2021 Share Posted June 12, 2021 12 hours ago, The Philotic Knight said: If it errors out because it can't download the file, it should just loop and try to pickup the next file. If it errors out for any other reason, then I need to look at the errors and troubleshoot them. So it looks like this is the condition, but the loop is not occurring. OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
WanderingAries Posted June 12, 2021 Share Posted June 12, 2021 New error being spat out by the M1, Path/Filename length exceeded. Seems that the multiple instances may also be attempting to manipulate the same files as well. Could be a coincidence, IDK. There was an error in the application. Please copy and paste the following text in a message to the program author: System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-28218-20120906-144458.warc' ---> Ionic.Zip.ZipException: The path is too long, filename=C:\Users\USERNAME\AppData\Local\Temp\CoH_Forums_Output\http_titanindustries.guildportal.com_UI_SigGenHandler__mainTextTankletmtCol1FF0000mtCol2FF9900lti65tti10otherCol1FFFFFFotherCol2FFFFFFbgCol10000FFbgCol2000000sub1Player26writerofallthingsRPszsub2Use22Tanklet22tofindmeonglobalsr1r2r3r4pt10pl5fs116fs29mtGt0pGt1bgImgUrlthumbImgUrlhttp3a2f2fwww.guildportal.com2fCargo2fSystem2fLayout20062fSigThumbs2fWorldofWarcraft2f8.gifmtBorderCol000000fontRedCirclefontSmallArial.ashx ---> System.IO.PathTooLongException: The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters. at System.IO.LongPathHelper.Normalize(String path, UInt32 maxPathLength, Boolean checkInvalidCharacters, Boolean expandShortPaths) at System.IO.Path.NewNormalizePath(String path, Int32 maxPathLength, Boolean expandShortPaths) at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength, Boolean expandShortPaths) at System.IO.Path.GetFullPathInternal(String path) at System.IO.File.InternalGetLastWriteTimeUtc(String path, Boolean checkHost) at Ionic.Zip.ZipEntry.Create(String nameInArchive, ZipEntrySource source, Object arg1, Object arg2) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.cs:line 2400 --- End of inner exception stack trace --- at Ionic.Zip.ZipEntry.Create(String nameInArchive, ZipEntrySource source, Object arg1, Object arg2) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.cs:line 2419 at Ionic.Zip.ZipFile.AddFile(String fileName, String directoryPathInArchive) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipFile.AddUpdate.cs:line 405 at Ionic.Zip.ZipFile.UpdateFile(String fileName, String directoryPathInArchive) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipFile.AddUpdate.cs:line 902 at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 97 --- End of inner exception stack trace --- There was an error in the application. Please copy and paste the following text in a message to the program author: System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-28679-20120906-015155.warc' ---> Ionic.Zip.ZipException: The path is too long, filename=C:\Users\USERNAME\AppData\Local\Temp\CoH_Forums_Output\http_titanindustries.guildportal.com_UI_SigGenHandler__mainTextTankletmtCol1FF0000mtCol2FF9900lti65tti10otherCol1FFFFFFotherCol2FFFFFFbgCol10000FFbgCol2000000sub1Player26writerofallthingsRPszsub2Use22Tanklet22tofindmeonglobalsr1r2r3r4pt10pl5fs116fs29mtGt0pGt1bgImgUrlthumbImgUrlhttp3a2f2fwww.guildportal.com2fCargo2fSystem2fLayout20062fSigThumbs2fWorldofWarcraft2f8.gifmtBorderCol000000fontRedCirclefontSmallArial.ashx ---> System.IO.PathTooLongException: The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters. at System.IO.LongPathHelper.Normalize(String path, UInt32 maxPathLength, Boolean checkInvalidCharacters, Boolean expandShortPaths) at System.IO.Path.NewNormalizePath(String path, Int32 maxPathLength, Boolean expandShortPaths) at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength, Boolean expandShortPaths) at System.IO.Path.GetFullPathInternal(String path) at System.IO.File.InternalGetLastWriteTimeUtc(String path, Boolean checkHost) at Ionic.Zip.ZipEntry.Create(String nameInArchive, ZipEntrySource source, Object arg1, Object arg2) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.cs:line 2400 --- End of inner exception stack trace --- at Ionic.Zip.ZipEntry.Create(String nameInArchive, ZipEntrySource source, Object arg1, Object arg2) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.cs:line 2419 at Ionic.Zip.ZipFile.AddFile(String fileName, String directoryPathInArchive) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipFile.AddUpdate.cs:line 405 at Ionic.Zip.ZipFile.UpdateFile(String fileName, String directoryPathInArchive) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipFile.AddUpdate.cs:line 902 at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 97 --- End of inner exception stack trace --- There was an error in the application. Please copy and paste the following text in a message to the program author: System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-28809-20120906-200034.warc' ---> System.IO.FileNotFoundException: Could not find file 'C:\Users\USERNAME\AppData\Local\Temp\CoH_Forums_Output\https_accounts.google__servicemailpassivetruermfalsecontinuehttps_mail.google.com_mail_ui3D226ik3Df1581e923526view3Datt26th3D12f3b32ff1b1144226attid3D0.126disp3Dinline26zwscc1ltmpldefaultltmplcache2.com_ServiceLogin'. at Ionic.Zip.SharedUtilities.GetFileLength(String fileName) in C:\projects\dotnetzip-semverd\src\Zip.Shared\Shared.cs:line 43 at Ionic.Zip.ZipEntry.MaybeUnsetCompressionMethodForWriting(Int32 cycle) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 800 at Ionic.Zip.ZipEntry.WriteHeader(Stream s, Int32 cycle) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 1078 at Ionic.Zip.ZipEntry.Write(Stream s) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 2220 at Ionic.Zip.ZipFile.Save() in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipFile.Save.cs:line 168 at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 101 --- End of inner exception stack trace --- OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
WanderingAries Posted June 12, 2021 Share Posted June 12, 2021 From the other M1 instance. Same combination of long name and can't find file. There was an error in the application. Please copy and paste the following text in a message to the program author: System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-21980-20120914-181605.warc' ---> Ionic.Zip.ZipException: The path is too long, filename=C:\Users\USERNAME\AppData\Local\Temp\CoH_Forums_Output\http_www.google__qthighhighsocksum1hlensaNqscrl1nord1rlz1T4WZPH_enUS437US437tbmischtbnidF16UcnTK1hKhM_imgrefurlhttp_fashionclothing.info_3Fp3D145docidvsHYfeBt2OhfOMw535h298eiZNZlTr6YOsPsQKf_ICbCgzoom1iactrcdur502page1tbnh117tbnw210start0ndsp13ved1t_429r_3s_0tx95ty59biw1366bih618.com_imgres ---> System.IO.PathTooLongException: The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters. at System.IO.LongPathHelper.Normalize(String path, UInt32 maxPathLength, Boolean checkInvalidCharacters, Boolean expandShortPaths) at System.IO.Path.NewNormalizePath(String path, Int32 maxPathLength, Boolean expandShortPaths) at System.IO.Path.NormalizePath(String path, Boolean fullCheck, Int32 maxPathLength, Boolean expandShortPaths) at System.IO.Path.GetFullPathInternal(String path) at System.IO.File.InternalGetLastWriteTimeUtc(String path, Boolean checkHost) at Ionic.Zip.ZipEntry.Create(String nameInArchive, ZipEntrySource source, Object arg1, Object arg2) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.cs:line 2400 --- End of inner exception stack trace --- at Ionic.Zip.ZipEntry.Create(String nameInArchive, ZipEntrySource source, Object arg1, Object arg2) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.cs:line 2419 at Ionic.Zip.ZipFile.AddFile(String fileName, String directoryPathInArchive) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipFile.AddUpdate.cs:line 405 at Ionic.Zip.ZipFile.UpdateFile(String fileName, String directoryPathInArchive) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipFile.AddUpdate.cs:line 902 at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 97 --- End of inner exception stack trace --- There was an error in the application. Please copy and paste the following text in a message to the program author: System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-21949-20120905-084810.warc' ---> System.IO.FileNotFoundException: Could not find file 'C:\Users\USERNAME\AppData\Local\Temp\CoH_Forums_Output\https_accounts.google__servicemailpassivetruermfalsecontinuehttps_mail.google.com_mail_ui3D226ik3Df1581e923526view3Datt26th3D12f3b32ff1b1144226attid3D0.126disp3Dinline26zwscc1ltmpldefaultltmplcache2.com_ServiceLogin'. at Ionic.Zip.SharedUtilities.GetFileLength(String fileName) in C:\projects\dotnetzip-semverd\src\Zip.Shared\Shared.cs:line 43 at Ionic.Zip.ZipEntry.MaybeUnsetCompressionMethodForWriting(Int32 cycle) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 800 at Ionic.Zip.ZipEntry.WriteHeader(Stream s, Int32 cycle) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 1078 at Ionic.Zip.ZipEntry.Write(Stream s) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 2220 at Ionic.Zip.ZipFile.Save() in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipFile.Save.cs:line 168 at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 101 --- End of inner exception stack trace --- There was an error in the application. Please copy and paste the following text in a message to the program author: System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-21055-20120904-235946.warc' ---> System.IO.FileNotFoundException: Could not find file 'C:\Users\USERNAME\AppData\Local\Temp\CoH_Forums_Output\http_boards.cityofheroes.com_image__groupid25dateline1296635860typegroupthumb_renamed_17.html'. at Ionic.Zip.SharedUtilities.GetFileLength(String fileName) in C:\projects\dotnetzip-semverd\src\Zip.Shared\Shared.cs:line 43 at Ionic.Zip.ZipEntry.MaybeUnsetCompressionMethodForWriting(Int32 cycle) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 800 at Ionic.Zip.ZipEntry.WriteHeader(Stream s, Int32 cycle) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 1078 at Ionic.Zip.ZipEntry.Write(Stream s) in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipEntry.Write.cs:line 2220 at Ionic.Zip.ZipFile.Save() in C:\projects\dotnetzip-semverd\src\Zip.Shared\ZipFile.Save.cs:line 168 at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 101 --- End of inner exception stack trace --- OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
KauaiJim Posted June 12, 2021 Share Posted June 12, 2021 Yeah, as I mentioned before I suspect that this has to do with multiple instances writing / reading / deleting from the same temp subdirectory. Wonder if it is possible to have them write into unique subdirectories that get deleted when done? Want to see my current list of characters? Want to know more about me than you ever wanted to know? Wish Granted! Check out the 'About Me' in my profile: KauaiJim - Homecoming (homecomingservers.com) Link to comment Share on other sites More sharing options...
WanderingAries Posted June 12, 2021 Share Posted June 12, 2021 7 minutes ago, KauaiJim said: Yeah, as I mentioned before I suspect that this has to do with multiple instances writing / reading / deleting from the same temp subdirectory. Wonder if it is possible to have them write into unique subdirectories that get deleted when done? Until the issue of file length gets resolved, I don't think that it will matter. OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 12, 2021 Author Share Posted June 12, 2021 (edited) @KauaiJim @WanderingAries Thanks for the processing power, and for reporting these issues. This does not in fact appear to be an issue of conccurrency, at least not at this time (unless you've got some different error messages to report to me Jim?). The "file not found" issue actually appears to be RELATED to the "file size" issue discovered in Ionic's Zip functionality. And this would happen AFTER the WARC file was done being downloaded, thus why it stops the works, because it's an issue with processing the file (specifically, trying to add one of the output files into the final zip file). Look at your stacktrace here Aries: Line 2 of the stack trace shows that they can't find the file "xxxxxx" because line 3 of the stack trace shows that when Ionic's code is trying to get the length of the filename... the filename is too long. Windows filenames + paths accept a maximum length of 260 characters, and that filename/path 'C:\Users\USERNAME\AppData\Local\Temp\CoH_Forums_Output\https_accounts.google__servicemailpassivetruermfalsecontinuehttps_mail.google.com_mail_ui3D226ik3Debd018fc9626view3Datt26th3D12b563ab717b036626attid3D0.126disp3Dinline26realattid3Df_gem5zqoe126zwscc1ltmpldefaultltmplcache2.com_ServiceLogin' is 294 characters long! So it can't "find the file", because the filename itself is an invalid and unrecognizable filename for the OS. I don't know why I didn't think of that issue before this. I'll write into my "SanitizeFileName" procedure code at the end to truncate any paths tbat long down to the required 260 length limit (probably go 250 to be "safe"). Now THIS one, I'm concerned about: Because it's reporting the same error as the other, but the filename, 'C:\Users\USERNAME\AppData\Local\Temp\CoH_Forums_Output\http_boards.cityofheroes.com_styles_CoH_misc_background_renamed_70.jpg' is a mere 70 characters long. However, looking at the filename... notice the "_renamed_70"? That is in fact a bit of code that I had put into my SanitizeFileName method to try to help prevent concurrency issues.... by making sure each time I wrote an output file, if such a file of that name already exists, my code loops through all possible integers from 1 to infinity until it finds a filename that DOESN'T yet exist. However, if two processors at the same time happened to perform that check for the same base filename while processing the same output file at the same time... damn, I hadn't considered that. So, what I think I'm going to do here is in fact add an additional bit to my "temporary output" code to create a temp folder within the output folder using the same trick so that each instance makes a unique folder for itself to work in. That should take care of that issue... I hope. My family is going to a newly bought vacation cabin this weekend, sometime this morning. So I'll try to push this through and troubleshoot and debug/test the solution this morning before we leave. Keep you version 1.4 processors, and after I release 1.5, extract the new processor in another directory... just in case I fail in my troubleshooting and release a WORSE processor. Then you can fall back on the old "slightly faulty" one. Edited June 12, 2021 by The Philotic Knight I'm out. Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 12, 2021 Author Share Posted June 12, 2021 Okay, version 1.5 is now up on my server, and successfully processed the file that failed for you @WanderingAries. Thank you for your support. Just to warn you, I may not be able to do any more coding the rest of this weekend. I'm out. Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 12, 2021 Author Share Posted June 12, 2021 (edited) FYI, since we had 45 files stuck in "processing", I moved them all back to the input. This MAY have caused some issues for your currently running processes. If it did, just restart those instances, and it should continue from there. Maybe clear out your temp files first, but that's not necessary. I just thought I should "clean those up" before I left for the weekend. Edited June 12, 2021 by The Philotic Knight I'm out. Link to comment Share on other sites More sharing options...
WanderingAries Posted June 12, 2021 Share Posted June 12, 2021 5 hours ago, The Philotic Knight said: Because it's reporting the same error as the other, but the filename, 'C:\Users\USERNAME\AppData\Local\Temp\CoH_Forums_Output\http_boards.cityofheroes.com_styles_CoH_misc_background_renamed_70.jpg' is a mere 70 characters long. I forget the words to explain it, but it "did" seem to make sense to me when you look Just at the file name, but it could also be an instance of one error actually triggering an unrelated thing. I'd have to lookup the actual windows filename limitations, but do note it mentions 32 bit Integer (I read that somewhere in these errors). Again, this is just spitballing on old coding memories. OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 12, 2021 Author Share Posted June 12, 2021 @WanderingAries, as I mentioned before, maximum path limit is 260 characters, filename and path all included. There is no specific filename limit except where it makes the path linger than 260 characters: https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd I'm out. Link to comment Share on other sites More sharing options...
WanderingAries Posted June 12, 2021 Share Posted June 12, 2021 New (to me) error. Again, it stopped the instance. There was an error in the application. Please copy and paste the following text in a message to the program author: System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-24800-20120907-034834.warc' ---> System.ApplicationException: Target file already exists at COH_WARC_Processor.FTPclient.Download(String sourceFilename, FileInfo targetFI, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 273 at COH_WARC_Processor.FTPclient.Download(String sourceFilename, String localFilename, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 242 at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 81 --- End of inner exception stack trace --- OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 12, 2021 Author Share Posted June 12, 2021 39 minutes ago, WanderingAries said: New (to me) error. Again, it stopped the instance. There was an error in the application. Please copy and paste the following text in a message to the program author: System.Exception: Error while processing file 'boards.cityofheroes.com-threads-range-24800-20120907-034834.warc' ---> System.ApplicationException: Target file already exists at COH_WARC_Processor.FTPclient.Download(String sourceFilename, FileInfo targetFI, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 273 at COH_WARC_Processor.FTPclient.Download(String sourceFilename, String localFilename, Boolean PermitOverwrite) in C:\Projects\COH WARC Handler\WHB WARC Processor\FTP Client.cs:line 242 at COH_WARC_Processor.MainForm.BtnProcess_Click(Object sender, EventArgs e) in C:\Projects\COH WARC Handler\WHB WARC Processor\MainForm.cs:line 81 --- End of inner exception stack trace --- With it being caused by FTP Download, there's only one explanation- it's one if those 45 files that got moved from processing to input. Sorry. I suggest opening your temp folder and deleting any WARC files there that aren't currently being processed. I'm out. Link to comment Share on other sites More sharing options...
WanderingAries Posted June 12, 2021 Share Posted June 12, 2021 1 minute ago, The Philotic Knight said: With it being caused by FTP Download, there's only one explanation- it's one if those 45 files that got moved from processing to input. Sorry. I suggest opening your temp folder and deleting any WARC files there that aren't currently being processed. Ok, The temp folder in question is the C:\Users\USERNAME\AppData\Local\Temp right? OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now