Oubliette_Red Posted June 27, 2019 Share Posted June 27, 2019 FYI, I'm also claiming boards.cityofheroes.com-threads-range-15775-20120907-054529, which I think is the biggest file. Have all of the 1m+ files been parsed already? I actually tried to run a parse of the full folder to remove already parsed files but none of those were deleted. Dislike certain sounds? Silence/Modify specific sounds. Looking for modified whole powerset sfx? Check out Michiyo's modder or Solerverse's thread. Got a punny character? You should share it. Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 27, 2019 Author Share Posted June 27, 2019 I don't know, I haven't really been keeping track, TBH. With SO MANY files... I don't think it's really worth it to classify them really. I just figured if you're hitting the small ones, I'd jump on the biggest one(s). I'm out. Link to comment Share on other sites More sharing options...
Oubliette_Red Posted June 27, 2019 Share Posted June 27, 2019 Big parse error this time around. Dislike certain sounds? Silence/Modify specific sounds. Looking for modified whole powerset sfx? Check out Michiyo's modder or Solerverse's thread. Got a punny character? You should share it. Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 27, 2019 Author Share Posted June 27, 2019 That one's literally just a timeout. Try again. I'll modify the code to make that more obvious and less Cryptic. 1 I'm out. Link to comment Share on other sites More sharing options...
justicebeliever Posted June 27, 2019 Share Posted June 27, 2019 That one's literally just a timeout. Try again. I'll modify the code to make that more obvious and less Cryptic. Maybe an error message that's a little more NCSoft on the eyes? I'm no Paragon of virtue when it comes to this though... 1 "The opposite of a fact is falsehood, but the opposite of one profound truth may very well be another profound truth." - Niels Bohr Global Handle: @JusticeBeliever ... Home servers on Live: Guardian ... Playing on: Everlasting Link to comment Share on other sites More sharing options...
Oubliette_Red Posted June 27, 2019 Share Posted June 27, 2019 That one's literally just a timeout. Try again. I'll modify the code to make that more obvious and less Cryptic. Maybe an error message that's a little more NCSoft on the eyes? I'm no Paragon of virtue when it comes to this though... Nice pun, although the error messages would have been generated by Paragon Studios and were not always helpful. I recalled talking to one of the Devs way back in the day and he relayed a story regarding their move to Jira for bug tracking and that it auto-generated tickets for player-generated bugs using the reporting system. Bug ticket #1 was by a player, who'd description of their issue was "Shit's broke, fix it." Funny as hell but not at all helpful. Dislike certain sounds? Silence/Modify specific sounds. Looking for modified whole powerset sfx? Check out Michiyo's modder or Solerverse's thread. Got a punny character? You should share it. Link to comment Share on other sites More sharing options...
justicebeliever Posted June 27, 2019 Share Posted June 27, 2019 That one's literally just a timeout. Try again. I'll modify the code to make that more obvious and less Cryptic. Maybe an error message that's a little more NCSoft on the eyes? I'm no Paragon of virtue when it comes to this though... Nice pun, although the error messages would have been generated by Paragon Studios and were not always helpful. I recalled talking to one of the Devs way back in the day and he relayed a story regarding their move to Jira for bug tracking and that it auto-generated tickets for player-generated bugs using the reporting system. Bug ticket #1 was by a player, who'd description of their issue was "Shit's broke, fix it." Funny as hell but not at all helpful. We had a big launch one time that I was running UAT for, and I put in a bug that was labeled as Showstopper Bug...everyone freaked out, but no one bothered to look at the attachment - picture of a giant cockroach dancing on a stage...I was told later that it was funny as hell, but not helpful... "The opposite of a fact is falsehood, but the opposite of one profound truth may very well be another profound truth." - Niels Bohr Global Handle: @JusticeBeliever ... Home servers on Live: Guardian ... Playing on: Everlasting Link to comment Share on other sites More sharing options...
Amatyr Posted June 27, 2019 Share Posted June 27, 2019 That one's literally just a timeout. Try again. I'll modify the code to make that more obvious and less Cryptic. I just got that timeout as well. Is this maybe a problem with larger files, or just bad luck? Excelsior Global Channel - for your server wide chat and forming TFs, Trials, Radios, Farms, whatever you want to do - /chan_join Excelsior today! Link to comment Share on other sites More sharing options...
Oubliette_Red Posted June 27, 2019 Share Posted June 27, 2019 That one's literally just a timeout. Try again. I'll modify the code to make that more obvious and less Cryptic. I just got that timeout as well. Is this maybe a problem with larger files, or just bad luck? I was running 4 separate chunks of 20k files, could just be a hiccup anywhere along the internet. I am also currently parsing boards.cityofheroes.com-threads-range-12058-20120912-121713 which is a 3.4MB file so I believe it will take a while. Dislike certain sounds? Silence/Modify specific sounds. Looking for modified whole powerset sfx? Check out Michiyo's modder or Solerverse's thread. Got a punny character? You should share it. Link to comment Share on other sites More sharing options...
Oubliette_Red Posted June 27, 2019 Share Posted June 27, 2019 Parsing error Dislike certain sounds? Silence/Modify specific sounds. Looking for modified whole powerset sfx? Check out Michiyo's modder or Solerverse's thread. Got a punny character? You should share it. Link to comment Share on other sites More sharing options...
Amatyr Posted June 28, 2019 Share Posted June 28, 2019 Parse error Excelsior Global Channel - for your server wide chat and forming TFs, Trials, Radios, Farms, whatever you want to do - /chan_join Excelsior today! Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 28, 2019 Author Share Posted June 28, 2019 I also added code to try to help prevent the timeout - by attempting each connection attempt three times before failing out with a more descriptive message. I'm out. Link to comment Share on other sites More sharing options...
Zep Posted June 28, 2019 Share Posted June 28, 2019 I've started the direct download. Supposedly 2 hours to go. I was also going to ask about how to run multiple threads and not reprocess work. ** Asus TUF x670E Gaming, Ryzen 7950x, AIO Corsair H150i Elite, TridentZ 192GB DDR5 6400, Sapphire 7900XTX, 48" 4K Samsung 3d & 56" 4k UHD, NVME Sabrent Rocket 2TB, MP600 Pro 8tb, MP700 2 TB. HDD Seagate 12TB ** ** Corsair Voyager a1600 ** Link to comment Share on other sites More sharing options...
Oubliette_Red Posted June 28, 2019 Share Posted June 28, 2019 I've started the direct download. Supposedly 2 hours to go. I was also going to ask about how to run multiple threads and not reprocess work. You can run multiple instances of the parser .exe. As mentioned above, once you get the files extracted, you can create multiple folders and drag multiple .warc files into each of them. Then each time you launch the parser, just point to a different folder. Once all of my folders have been parsed, I will usually run the parser on the main folder to auto-delete any files someone else has parsed in the meantime. I've been running chunks of similar sized files, but PK may call on folks to proceed by file name, that way people can call out a section of files they are parsing at any given time, to minimize overlap. Dislike certain sounds? Silence/Modify specific sounds. Looking for modified whole powerset sfx? Check out Michiyo's modder or Solerverse's thread. Got a punny character? You should share it. Link to comment Share on other sites More sharing options...
Oubliette_Red Posted June 28, 2019 Share Posted June 28, 2019 Finally finished the 3.4m file I started at 3:30... whew. I am now parsing the following: boards.cityofheroes.com-threads-range-27851-20120906-050401 boards.cityofheroes.com-threads-range-11881-20120905-014249 boards.cityofheroes.com-threads-range-12360-20120905-044715 boards.cityofheroes.com-threads-range-11936-20120904-182407 boards.cityofheroes.com-threads-range-16457-20120907-063526 boards.cityofheroes.com-threads-range-11231-20120911-085820 boards.cityofheroes.com-threads-range-12767-20120906-235325 boards.cityofheroes.com-threads-range-11242-20120904-020055 Dislike certain sounds? Silence/Modify specific sounds. Looking for modified whole powerset sfx? Check out Michiyo's modder or Solerverse's thread. Got a punny character? You should share it. Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 28, 2019 Author Share Posted June 28, 2019 Nice, thanks for all your work. For what it's worth, I've got the first archive decompressed, now I'm working on decompressing the second layer of archives. Then if there's a third layer (I don't remember), I'll decompress that. Once it's all decompressed into one folder, I'll zip it back up with 7z at the highest compression and upload it somewhere, and see about how to make a tracker... I'm out. Link to comment Share on other sites More sharing options...
Zep Posted June 28, 2019 Share Posted June 28, 2019 I have the direct DL. 9 minutes to copy from C to I - wow. After work tomorrow I can get busy. ** Asus TUF x670E Gaming, Ryzen 7950x, AIO Corsair H150i Elite, TridentZ 192GB DDR5 6400, Sapphire 7900XTX, 48" 4K Samsung 3d & 56" 4k UHD, NVME Sabrent Rocket 2TB, MP600 Pro 8tb, MP700 2 TB. HDD Seagate 12TB ** ** Corsair Voyager a1600 ** Link to comment Share on other sites More sharing options...
Amatyr Posted June 28, 2019 Share Posted June 28, 2019 I finally have a full copy of the files! Also, a new parser error: Excelsior Global Channel - for your server wide chat and forming TFs, Trials, Radios, Farms, whatever you want to do - /chan_join Excelsior today! Link to comment Share on other sites More sharing options...
Murcielago Posted June 28, 2019 Share Posted June 28, 2019 Outstanding, have an upvote! Wait wrong website. Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 28, 2019 Author Share Posted June 28, 2019 I finally have a full copy of the files! Also, a new parser error: That one was actually already fixed, you just have to get the new version and try that file again. I'm out. Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 28, 2019 Author Share Posted June 28, 2019 Update on the archive - got everything unzipped, now rezipping: In other news... looks like I'm about to be pushed to the bottom of the list. Thanks Amatyr! I'm out. Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 28, 2019 Author Share Posted June 28, 2019 In other news, since I've got the whole list again, I was able to update the list of "All Files" to finally be accurate! I'm out. Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 28, 2019 Author Share Posted June 28, 2019 Right now, if I've got my math right, I estimate the final file size will end up being a little over 102GB. I've got 743 files compressed, and the file size right now is 4026 MB. There's 18,869 total files, so.... [pre] 743 18869 ____ = _____ 4026 X 743X = 18869*4026 743X = 75,966,594 X = 75,966,594 / 743 X = 102,243.06 MB, or 102GB [/pre] Let me know if my math/logic seems wrong, but it looks like it'll be less than half the size of the original archive, when I'm all done zipping it up. ^.^ Now, it MIGHT take longer to unzip, probably... but at least there'll be less chance/opportunity for errors during transfer over the internet for future Contributors! I'm out. Link to comment Share on other sites More sharing options...
City Council Widower Posted June 28, 2019 City Council Share Posted June 28, 2019 Right now, if I've got my math right, I estimate the final file size will end up being a little over 102GB. I've got 743 files compressed, and the file size right now is 4026 MB. There's 18,869 total files, so.... [pre] 743 18869 ____ = _____ 4026 X 743X = 18869*4026 743X = 75,966,594 X = 75,966,594 / 743 X = 102,243.06 MB, or 102GB [/pre] Let me know if my math/logic seems wrong, but it looks like it'll be less than half the size of the original archive, when I'm all done zipping it up. ^.^ Now, it MIGHT take longer to unzip, probably... but at least there'll be less chance/opportunity for errors during transfer over the internet for future Contributors! Ah, wunderbar. I was thinking of pitching in myself, but I don't have 220 GB sitting around, so this will be most useful to me. "We need Widower. He's a drop of sanity in a bowl of chaos - very important." - Cipher Are you also a drop of sanity in a bowl of chaos? Consider applying to be a Game Master! Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 28, 2019 Author Share Posted June 28, 2019 I think you misunderstand. That's the COMPRESSED size. After you get it downloaded, you'd still have to uncompress it on your hard drive, in all its 735 GB glory. So, if you don't have around a terabyte to spare to have both the archive, and to decompress the archive... you sadly won't be able to help... I mean, I suppose I could upload the individual files somewhere perhaps and make them available to download, but then I'd have to setup some sort of code or something on a webpage that deleted the file from the server automatically and removed the link once it was parsed... and that right now I think is beyond my web coding ability, as I'm primarly a desktop programmer, just starting to step my foot into the world of the modern web... I'm out. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now