City Council Widower Posted June 28, 2019 City Council Share Posted June 28, 2019 I think you misunderstand. That's the COMPRESSED size. Ah, you wound me, friend PK. I was thinking - why decompress the whole archive when I could decompress but a part of it and parse that, then rinse and repeat 'til there are no more files? A lot more work for me, yes, but it should get it done. (Unless, of course, my right honorable friend Cipher gets to it first! ;D) "We need Widower. He's a drop of sanity in a bowl of chaos - very important." - Cipher Are you also a drop of sanity in a bowl of chaos? Consider applying to be a Game Master! Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 28, 2019 Author Share Posted June 28, 2019 Well, you COULD do that... but then you'd have to actually pay attention to the file names, and only uncompress the files that haven't been parsed already by using the Viewer as a cross reference... and those files are named... painfully.... I mean, if you want to go ahead and do that work, I'm all for whatever help I can get... but I'm telling you right now, it'd make my eyes BLEED to compare these file names against each other to find unparsed files: Right now, the Parser program handles all this for you, by deleting all of the "previously parsed" files from your system, and then grabbing a random file that's left, on every loop. I'm out. Link to comment Share on other sites More sharing options...
City Council Widower Posted June 28, 2019 City Council Share Posted June 28, 2019 Friend PK, there is nothing more pleasurable in the world than eye-bleed-inducing nonsense! "We need Widower. He's a drop of sanity in a bowl of chaos - very important." - Cipher Are you also a drop of sanity in a bowl of chaos? Consider applying to be a Game Master! Link to comment Share on other sites More sharing options...
Zep Posted June 28, 2019 Share Posted June 28, 2019 Good thing I have a 4TB Spinner that is almost empty -- not sure I have room on the NVME. ** Asus TUF x670E Gaming, Ryzen 7950x, AIO Corsair H150i Elite, TridentZ 192GB DDR5 6400, Sapphire 7900XTX, 48" 4K Samsung 3d & 56" 4k UHD, NVME Sabrent Rocket 2TB, MP600 Pro 8tb, MP700 2 TB. HDD Seagate 12TB ** ** Corsair Voyager a1600 ** Link to comment Share on other sites More sharing options...
Zep Posted June 28, 2019 Share Posted June 28, 2019 Any reason I could not make a whole bunch of copies of the executable and alias them to the same data folder? Or would they all try and work on the same file at the same time? Or will it all be done by the time I get home from work today? :) ** Asus TUF x670E Gaming, Ryzen 7950x, AIO Corsair H150i Elite, TridentZ 192GB DDR5 6400, Sapphire 7900XTX, 48" 4K Samsung 3d & 56" 4k UHD, NVME Sabrent Rocket 2TB, MP600 Pro 8tb, MP700 2 TB. HDD Seagate 12TB ** ** Corsair Voyager a1600 ** Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 28, 2019 Author Share Posted June 28, 2019 Good thing I have a 4GB Spinner that is almost empty -- not sure I have room on the NVME. I do hope you mean 4TB, instead of 4GB. Any reason I could not make a whole bunch of copies of the executable and alias them to the same data folder? Or would they all try and work on the same file at the same time? Or will it all be done by the time I get home from work today? :) You don't even have to make copies of the executable. Just run the same executable as many times as you want, and point them to the same directory. The way the program works, on EVERY program loop, it: Checks for already parsed files in the database, and deletes them from the local folder Puts the list of all remaining files into an array and grabs a RANDOM file from that array to process After it's done processing the file, it deletes the local file, and repeats the loop again, refreshing its own internal lists every time. I'm out. Link to comment Share on other sites More sharing options...
WanderingAries Posted June 28, 2019 Share Posted June 28, 2019 A bit late in the game (having a hard time getting back to threads I've read), but what I was Trying to get at was something like having you put up say 1-10Gb chunks for people to munch on that would speed up the initial DL and make it easier for those who appear to be using SSDs. Personally, I'm sitting on a good ~6Tb so long as I don't go back to my AV project (been avoiding) in the meantime. I hadn't started anything because I first read this when it was like half a page long. :p OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
City Council Widower Posted June 28, 2019 City Council Share Posted June 28, 2019 A bit late in the game (having a hard time getting back to threads I've read), but what I was Trying to get at was something like having you put up say 1-10Gb chunks for people to munch on that would speed up the initial DL and make it easier for those who appear to be using SSDs. Personally, I'm sitting on a good ~6Tb so long as I don't go back to my AV project (been avoiding) in the meantime. I hadn't started anything because I first read this when it was like half a page long. :p It's a good idea, and I personally would take advantage, but I suspect it'd be difficult to manage - you wouldn't really want two people grabbing the same chunk. That would be... awkward. "We need Widower. He's a drop of sanity in a bowl of chaos - very important." - Cipher Are you also a drop of sanity in a bowl of chaos? Consider applying to be a Game Master! Link to comment Share on other sites More sharing options...
justicebeliever Posted June 28, 2019 Share Posted June 28, 2019 It's a good idea, and I personally would take advantage, but I suspect it'd be difficult to manage - you wouldn't really want two people grabbing the same chunk. That would be... awkward. No one likes awkward chunks... "The opposite of a fact is falsehood, but the opposite of one profound truth may very well be another profound truth." - Niels Bohr Global Handle: @JusticeBeliever ... Home servers on Live: Guardian ... Playing on: Everlasting Link to comment Share on other sites More sharing options...
WanderingAries Posted June 28, 2019 Share Posted June 28, 2019 It's a good idea, and I personally would take advantage, but I suspect it'd be difficult to manage - you wouldn't really want two people grabbing the same chunk. That would be... awkward. Yes it would require some vetting. Perhaps instead of everyone grabbing, have OP Assign chunks as people ask to help. This would require the download folder be organized in such a way that there's a minimal amount of confusion as to who should be grabbing what. Again, a bit late to the party, so I'd hate to undo some of the work already begun. OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 28, 2019 Author Share Posted June 28, 2019 I kinda really wanted it to be setup in such a way that would allow anyone at any time to grab all the files, start the parser, and start parsing. No fuss, no muss. I'm out. Link to comment Share on other sites More sharing options...
WanderingAries Posted June 28, 2019 Share Posted June 28, 2019 Oh wow, I don't torrent so started up the TAR file for download on my first post (in here) today. I'm at 4% running <2MB/s, 11.3Gb file size right now. O.O OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 28, 2019 Author Share Posted June 28, 2019 Yeah, it's gonna take a while. That's why I'm trying to compress the whole archive into as small of a space as possible, and then re-upload it. I'll host it on CityOfPlayers.com, and I'll also host it via torrent, for as long as it takes to get this thing done. ETA, about 16 hours until the files are compressed. I'm out. Link to comment Share on other sites More sharing options...
Zep Posted June 29, 2019 Share Posted June 29, 2019 Yes, TB not GB Were gonna see how many threads this baby can handle. ** Asus TUF x670E Gaming, Ryzen 7950x, AIO Corsair H150i Elite, TridentZ 192GB DDR5 6400, Sapphire 7900XTX, 48" 4K Samsung 3d & 56" 4k UHD, NVME Sabrent Rocket 2TB, MP600 Pro 8tb, MP700 2 TB. HDD Seagate 12TB ** ** Corsair Voyager a1600 ** Link to comment Share on other sites More sharing options...
WanderingAries Posted June 29, 2019 Share Posted June 29, 2019 14% :p OG Server: Pinnacle <||> Current Primary Server: Torchbearer || Also found on the others if desired Installing CoX: Windows || MacOS || MacOS for M1 <||> Migrating Data from an Older Installation Clubs: Mid's Hero Designer || PC Builders || HC Wiki || Jerk Hackers Old Forums <||> Titan Network <||> Heroica! (by @Shenanigunner) Link to comment Share on other sites More sharing options...
Zep Posted June 29, 2019 Share Posted June 29, 2019 Wow Win Zip sucks monkey balls -- like really salty smelly ones. Should have listened and used 7z right off. Estimating 90 min to uncompressed all the files. ** Asus TUF x670E Gaming, Ryzen 7950x, AIO Corsair H150i Elite, TridentZ 192GB DDR5 6400, Sapphire 7900XTX, 48" 4K Samsung 3d & 56" 4k UHD, NVME Sabrent Rocket 2TB, MP600 Pro 8tb, MP700 2 TB. HDD Seagate 12TB ** ** Corsair Voyager a1600 ** Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 29, 2019 Author Share Posted June 29, 2019 I've got some bad news for you, that's just the first round of unzipping. I'm out. Link to comment Share on other sites More sharing options...
Zep Posted June 29, 2019 Share Posted June 29, 2019 Win zip did the first ok. Maybe 20-25 min. It just wasnt willing to do more then 1 at a time with the second wave. I was not going to queue them up 1 by 1 time. Second wave took 77 min. Before bed I'll get it going. ** Asus TUF x670E Gaming, Ryzen 7950x, AIO Corsair H150i Elite, TridentZ 192GB DDR5 6400, Sapphire 7900XTX, 48" 4K Samsung 3d & 56" 4k UHD, NVME Sabrent Rocket 2TB, MP600 Pro 8tb, MP700 2 TB. HDD Seagate 12TB ** ** Corsair Voyager a1600 ** Link to comment Share on other sites More sharing options...
Zep Posted June 29, 2019 Share Posted June 29, 2019 In case any of you end up with all the files to be parced in separate directories too. https://itstillworks.com/make-one-folder-out-multiple-folders-26192.html Never deleted 18k+ empty folders before and that is saying something for me. ** Asus TUF x670E Gaming, Ryzen 7950x, AIO Corsair H150i Elite, TridentZ 192GB DDR5 6400, Sapphire 7900XTX, 48" 4K Samsung 3d & 56" 4k UHD, NVME Sabrent Rocket 2TB, MP600 Pro 8tb, MP700 2 TB. HDD Seagate 12TB ** ** Corsair Voyager a1600 ** Link to comment Share on other sites More sharing options...
Zep Posted June 29, 2019 Share Posted June 29, 2019 Looking at cpu use, disk use, net bandwidth, I could run over 200 of the app at the same time. Could you make a local config file so I could set my name and data directory so I could just start clicking away? ** Asus TUF x670E Gaming, Ryzen 7950x, AIO Corsair H150i Elite, TridentZ 192GB DDR5 6400, Sapphire 7900XTX, 48" 4K Samsung 3d & 56" 4k UHD, NVME Sabrent Rocket 2TB, MP600 Pro 8tb, MP700 2 TB. HDD Seagate 12TB ** ** Corsair Voyager a1600 ** Link to comment Share on other sites More sharing options...
Zep Posted June 29, 2019 Share Posted June 29, 2019 Got an error about the server being busy 3 times - hope we arnt over loading you :) Another error I can email you - dont have an image host set up to post it -- suppose we could buddy on discord and do it that way. ---- Morning and 19 errors Can these be saves to a log? ** Asus TUF x670E Gaming, Ryzen 7950x, AIO Corsair H150i Elite, TridentZ 192GB DDR5 6400, Sapphire 7900XTX, 48" 4K Samsung 3d & 56" 4k UHD, NVME Sabrent Rocket 2TB, MP600 Pro 8tb, MP700 2 TB. HDD Seagate 12TB ** ** Corsair Voyager a1600 ** Link to comment Share on other sites More sharing options...
Hyperstrike Posted June 29, 2019 Share Posted June 29, 2019 Okay, even with Gig Internet, the torrent isn't stable enough to download in less than a day... Downloading the TAR? A couple hours. If you want to be godlike, pick anything. If you want to be GOD, pick a TANK! Link to comment Share on other sites More sharing options...
Amatyr Posted June 29, 2019 Share Posted June 29, 2019 I'm just throwing the images onto imgur. Last night's only error: Excelsior Global Channel - for your server wide chat and forming TFs, Trials, Radios, Farms, whatever you want to do - /chan_join Excelsior today! Link to comment Share on other sites More sharing options...
hp Posted June 29, 2019 Share Posted June 29, 2019 16 Link to comment Share on other sites More sharing options...
_NOPE_ Posted June 29, 2019 Author Share Posted June 29, 2019 I'm just throwing the images onto imgur. Last night's only error: That's perfect, thank you. As I have time, I'll make new versions to fix these. I'm out. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now