VidiotMaps for Issue 24 and Beyond

[PsiBug]

Thank you for your replies. Oh well, it's not a show stopper for me anyway. 

 

My drivers appear to be current. Maybe I'll install the nVidia DDS plugins for photoshop and test out a few things. might just be a compression type or some save option that my computer can't stomach for one reason or another.

 

 

[Icecomet]

23 hours ago, AboveTheChemist said:

Do the other two maps included in the vidiotmap_updates_feb_2021 (Kallisti Wharf and Warburg) exhibit the same black/blank map issue (or perhaps other issues), or do those maps appear normal?

 

Hi ATC, I think we covered this, but for my personal situation it is ONLY Siren's Call that has the problem.  Khalisti and Warburg and every other zone in game works absolutely perfectly (and Siren's Call worked before it was rotated for some unknown reason AND it worked for a short time after December 5th when Michiyo provided a quick rotation of the existing vidiotmaps badge.  But since about February it simply stopped working.  Nothing on my client PC has changed (aside for the crappy forced windows 10 updates).  😧

[AboveTheChemist]

On 4/6/2021 at 8:05 PM, Icecomet said:

 

Hi ATC, I think we covered this, but for my personal situation it is ONLY Siren's Call that has the problem.  Khalisti and Warburg... works absolutely perfectly

Just to be clear, are you using my updates to the Kallisti and Warburg maps from Feb (the ones I now available in this post)? If so, that shoots some holes in my working theory, but I suppose it brings us ever closer to a potential resolution. If not, do you mind trying them and reporting back? And make a backup of your existing textures should mine prove problematic!

Edited June 21, 2021 by AboveTheChemist
remove old links, re-point to new update post

[Icecomet]

28 minutes ago, AboveTheChemist said:

Just to be clear, are you using my updates to the Kallisti and Warburg maps from Feb (the ones I posted in this post)? If so, that shoots some holes in my working theory, but I suppose it brings us ever closer to a potential resolution. If not, do you mind trying them and reporting back? And make a backup of your existing textures should mine prove problematic!

 

Ok, I just tried all 3 of them again and as reported earlier by the other person.  Khalisti Wharf is BLACK, Warburg and Siren's are "scrambled" like the previous screenshots I provided you!

 

I was sure I tested your files previously, but I suppose maybe not, I did so many things in that timeframe it's hard to recollect what all I did and did not try map wise.

 

Thanks!

 

[_NOPE_]

Silly question- have you all tried just wiping your data directory and then installing the maps via CoH Modder? Kind of curious how the files in that specific version of the mod would behave on these "problem PCs".

[PsiBug]

I deleted the whole texture library folder and ran the modder to install the I26 maps V6. The siren's call map is scrambled. Warburg and Kallisti maps are displaying correctly but are not the most recent (feb 2021) versions, I think.

 

After copying all three newer maps manually from vidiotmap_updates_feb_2021 to their correct locations, Sirens map is still scrambled, and Kallisti and Warburg are just black (with trainers, contacts, and other markers visible).

 

(One thing I noticed is that the maps that do not work for me have GIMP DDSZ in their header information. A working file I checked did not have that.)

[_NOPE_]

3 hours ago, PsiBug said:

I deleted the whole texture library folder and ran the modder to install the I26 maps V6. The siren's call map is scrambled. Warburg and Kallisti maps are displaying correctly but are not the most recent (feb 2021) versions, I think.

 

After copying all three newer maps manually from vidiotmap_updates_feb_2021 to their correct locations, Sirens map is still scrambled, and Kallisti and Warburg are just black (with trainers, contacts, and other markers visible).

 

(One thing I noticed is that the maps that do not work for me have GIMP DDSZ in their header information. A working file I checked did not have that.)

Sillier question- did you try deleting the ENTRIE data directory, rather than just the texture directory?  It's possible other mod files outside of the textures may be affecting things. We need to rule everything but the original game files out. 

 

Second question - have you had both the new launcher and the old? Or just ever one or the other?

[Icecomet]

The lengthy steps I conducted through 2 solid days of doing nothing but troubleshooting this are as follows:

 

1. I have deleted my entire CoH installation out and installed the homecoming launcher.

2. I wiped out the fog of war map cache in C:\Users\%username%\AppData\Local\NCSoft\CoX\VisitedMaps

3. I installed the vidiotmaps installation via cohmodder (map was scrambled).

4. I installted the optimal path patch via cohmodder (map was still scrambled).

5. Removed the SC overlay and the map displays correctly.

6. AboveTheChemist sent me many various revisions saved in varying formats all were still scrambled and a few were just black.

 

Thanks!

 

[AboveTheChemist]

7 hours ago, PsiBug said:

One thing I noticed is that the maps that do not work for me have GIMP DDSZ in their header information. A working file I checked did not have that.

I do use GIMP to convert my image files to DDS before replacing the header info to turn them back into .texture files. I suspect that the VidiotMap folks may have used other software to either convert the image files to DDS, or to manipulate the DDS files directly, before turning them back into .texture files. I don't otherwise have much (if any) need for image manipulation software, so GIMP was the best option for me. But it is entirely possible that the issue is related to the software.

[PsiBug]

5 hours ago, The Philotic Knight said:

Sillier question- did you try deleting the ENTRIE data directory, rather than just the texture directory?  It's possible other mod files outside of the textures may be affecting things. We need to rule everything but the original game files out. 

 

Second question - have you had both the new launcher and the old? Or just ever one or the other?

Hi, (Breakthrough in green text below)

I am using modder version 1.68. I don't believe that I have used a different one in the past.

I ran the modder again today and uninstalled vidiotmaps I26 (V6).

Then I deleted the whole data folder

Then I ran CoH and confirmed the original default maps were working. Then I exited CoH.

Then I used the modder to install vidiotmaps I26 (V6) again.

Then I ran CoH. The Siren's call map  (Juggy's version from 10-7-2020 I think) was scrambled . Warburg (and presumably all other maps) displays correctly.

 

Then......... I changed my Menu>Options>Graphics and Audio>WORLD TEXTURE QUALITY from MEDIUM to HIGH. and the Siren's call map displayed correctly

 

@Icecomet, are you able to get the same result?

 

edit:

Confirmed that the 3 new maps from "vidiotmap_updates_feb_2021" also display properly for me if world textures are set to HIGH or VERY HIGH.

All maps now working at HIGH World Texture setting.

 

-Hope that helps,

PsiBug

 

Edited April 7, 2021 by PsiBug

[Icecomet]

33 minutes ago, PsiBug said:

Hi, (Breakthrough in green text below)

I am using modder version 1.68. I don't believe that I have used a different one in the past.

I ran the modder again today and uninstalled vidiotmaps I26 (V6).

Then I deleted the whole data folder

Then I ran CoH and confirmed the original default maps were working. Then I exited CoH.

Then I used the modder to install vidiotmaps I26 (V6) again.

Then I ran CoH. The Siren's call map  (Juggy's version from 10-7-2020 I think) was scrambled . Warburg (and presumably all other maps) displays correctly.

 

Then......... I changed my Menu>Options>Graphics and Audio>WORLD TEXTURE QUALITY from MEDIUM to HIGH. and the Siren's call map displayed correctly

 

@Icecomet, are you able to get the same result?

 

edit:

Confirmed that the 3 new maps from "vidiotmap_updates_feb_2021" also display properly for me if world textures are set to HIGH or VERY HIGH.

All maps now working at HIGH World Texture setting.

 

-Hope that helps,

PsiBug

 

 

Hello, yes, I can confirm that this corrects the issue.  Now the larger question is WHAT did the devs do to THIS zone when they rotated it that causes this issue?  SOMETHING had to be done that makes this zone special that requires a world texture detail of HIGH in order for that to happen.

 

@Jimmy or @Cipher, are you aware of any underlying reason why AFTER Siren's Call was rotated that a world texture setting of HIGH is required to make something like vidiotmaps work correctly in game?  Every other zone works 100% perfect on lower settings (as did Siren's Call before the rotation).  We'd appreciate any insights you might be able to share so we can put this topic to bed!

 

Thanks in advance.

 

Edited April 7, 2021 by Icecomet

[Pouncy]

Hi. I been following this. I have tampered with the same issues after finding my Siren's Call map was spun.

 

The maps we are having gradient issues with are the ones that where just updated by an individual who did confess was new at this. So lets look first at what he might have missed. That the high quality version works and that the others don't seems to indicate that the problem lies in the compression or resolution of the maps just updated.  Perhaps they were created at too high a resolution for the lower settings to use and thus they grain.

SO finding out what was used on the original ones that do work might help solve this.  It is even possible that there are multi versions drawn from and selected related to the resolution.

[_NOPE_]

4 hours ago, Pouncy said:

Hi. I been following this. I have tampered with the same issues after finding my Siren's Call map was spun.

 

The maps we are having gradient issues with are the ones that where just updated by an individual who did confess was new at this. So lets look first at what he might have missed. That the high quality version works and that the others don't seems to indicate that the problem lies in the compression or resolution of the maps just updated.  Perhaps they were created at too high a resolution for the lower settings to use and thus they grain.

SO finding out what was used on the original ones that do work might help solve this.  It is even possible that there are multi versions drawn from and selected related to the resolution.

That's a really good idea. I'll look into the original texture files when I get the time. I didn't think of that. That might have some answers.

[AboveTheChemist]

6 hours ago, Pouncy said:

The maps we are having gradient issues with are the ones that where just updated by an individual who did confess was new at this. So lets look first at what he might have missed. That the high quality version works and that the others don't seems to indicate that the problem lies in the compression or resolution of the maps just updated.  Perhaps they were created at too high a resolution for the lower settings to use and thus they grain.

Assuming that I am the individual to whom you refer, I didn't alter the resolution (in relation to the original in-game map) of any of the maps I updated. I also didn't compress any of the maps I updated, because I found that any time I used the available compression methods, the image quality was degraded compared to the original. I am not sure if the originals were compressed or not, but my best results were achieved when I used no compression.

 

From what I can tell, at least some of the original maps use RGBA format, and I initially used that format as well, but I found that some of my modified maps appeared at least partially 'scrambled' when I used RGBA, so I switched to RGB and had consistently good results, so that is what I used.

 

As noted previously in the thread, in attempting to solve the issue I sent Icecomet a number of test files saved in different formats (including RGBA), both compressed and uncompressed, and none worked. I'll also note that, for the Siren's Call map at least, I believe Icecomet also tried several other variations by other authors that were created since that zone was rotated in late Nov 2020, and none of them worked. So I don't think the problems were isolated to just maps that I created.

 

As you noted, I am indeed new at this, so if there is some part of the process I can tweak to make the maps more universally usable, I am happy to do so. In addition to the modified maps I have included here, I have published another (much larger) set of modified maps, and have a couple of other projects using modified maps in the works, so I'd like to make sure those are error-free for anyone that wants to use them.

[PsiBug]

 

1 hour ago, AboveTheChemist said:

In addition to the modified maps I have included here, I have published another (much larger) set of modified maps, and have a couple of other projects using modified maps in the works, so I'd like to make sure those are error-free for anyone that wants to use them.

 I installed the Warburg map from the optimized badging paths set, and I can confirm that it works correctly on High, but is scrambled at Medium World Texture Quality.

@AboveTheChemist, assuming that your world texture quality setting is currently at High or Very High, have you tested Medium or Low World Texture Quality setting in CoH to test the maps in question? I'm curious if the issue occurs on ALL computers running low quality settings in CoH.

Regarding the possibility that the cause of the trouble is within the images themselves, a screenshot of your GIMP export settings menu might be a good place to start. Are you including mipmaps? I believe they are tied two the game's ability to display .texture files in multiple resolutions.

 

Apologies if I am asking any questions that have already been answered.

 

 

Edited April 9, 2021 by PsiBug

[PsiBug]

Hi,

I was able to modify one of the maps from the OPTIMAL BADGING PATHS map pack to display correctly at all quality settings by editing the DDS image as well as the text portion of the .texture file

1) I took the Warburg map (map_V_PvP_04_01.texture) from @AboveTheAlchemist 's optimized badge path pack and confirmed that it had the same behavior (displays correctly on my computer at "high" or "very high" World Texture Quality, but displays incorrectly at "medium", "low" or "very low" world texture quality)

2) I used the detexturizer tool from this page: http://www.cityofplayers.com/pks-tools/

which I found via this thread: https://forums.homecomingservers.com/topic/9304-quality-of-life-texture-mods-and-other-things/page/4/

I used that tool to convert the .texture file to a .dds file and a .xml file.

 

3) I opened the Warburg DDS file in GIMP and re-exported it with the following settings:

 

4) I compared the .xml file to two other .xml files from textures that worked 100% correctly for me, and I noticed two (or three) numerical values that were identical in the two working files but different in the Warburg map file. I copied the following numerical values to the warburg xml file and resaved it (original warburg file is on the left, "100% working xml file" is on the right.

 

 

5) Then I used the detexturizer tool again to recombine the xml and dds files to a .texture file and copied the file back to its correct CoH folder. The updated file works at all World Texture Quality settings (EDIT- unfortunately, not working 100% correctly. My edited map changes resolution when the world texture quality setting is changed, but it SHOULD remain at full resolution, so more testing is needed).

 

notes:

a) Probably there is a simpler way to make the changes if you have programs that export directly to a .texture file.

b) Possibly only one of the two edits I made to the XML file were necessary, I have not tested them separately.

c) Updating only one of the two files (DDS or XML) did not not result in a working .texture file (assuming I kept track of my testing properly)

d) The file size of the original .texture file was 769kb and the new .texture file is 1025kb (possibly due to the inclusion of mipmap data)

 

Hope that helps!

 

 

Edited April 10, 2021 by PsiBug

[AboveTheChemist]

1 hour ago, PsiBug said:

Are you including mipmaps? I believe they are tied two the game's ability to display .texture files in multiple resolutions.

My world texture quality is set to very high, but I have not tested any of my maps at any lower settings. I have not been including mipmaps during the export process (because they didn't previously seem necessary), but I am happy to export a test map using them to see if that is part of the issue.

[Pouncy]

2 hours ago, AboveTheChemist said:

As you noted, I am indeed new at this, so if there is some part of the process I can tweak to make the maps more universally usable, I am happy to do so. In addition to the modified maps I have included here, I have published another (much larger) set of modified maps, and have a couple of other projects using modified maps in the works, so I'd like to make sure those are error-free for anyone that wants to use them.

 

You have all my respect.  Being new is not an insult. And I love that you did not take it that way.  I admire you and encourage you in this endeavor and want to not let my appreciation go unnoticed. 

I am hoping as a community we can find what the game actually does with the image.  I see light at the tunnel.

[AboveTheChemist]

Thanks for the kind words, Pouncy. It's been a big learning process but these projects would not have come to fruition if I didn't at least try!

 

29 minutes ago, PsiBug said:

The .texture file contains a .dds image and some data in the form of a text or xml file. It appears that the warburg file I am testing has numerical values in two fields that need to be different in order for the maps to display properly at all world texture quality settings. Copying the two numerical values from a .texture that works at all settings to the warburg .texture, caused the warburg .texture to work at all quality settings for me.

Interesting. The header data that I use in my modified maps has been transplanted right from the original VidiotMap textures, so it should be good. Unless those two numerical values you found are actually in the dds part of the data and not the header data. If so, perhaps those values are tied to the mipmaps? Or, perhaps I should use header data from the original in-game maps?

[AboveTheChemist]

3 hours ago, PsiBug said:

<Super Helpful Info>

Thanks for all the investigative work, PsiBug! I had a couple of observations and a question.

 

I checked on my system and as you can probably guess, my maps were blank and/or scrambled at any world texture setting other than 'High' or 'Very High'. You asked previously about my GIMP export settings, and they are identical to what you showed EXCEPT the mipmaps settings (I didn't use mipmaps on mine).

 

My questions is:

 

Did any of the '100% working' XML files you examined come from the original VidiotMap files that Blondeshell posted?

 

I used the original VidiotMap textures as my 'donor' textures for the header data and I wonder if I would be better off using the original map textures that come with the game data. Or, if the VidiotMap header data is fine, it's possible my re-texturizing script (I wrote a simple one in Python) is somehow garbling the data, and I should switch to using PK's detexturizer. I'll run some test maps later on my end to see what I can get working. I may also try using the RGBA format. I had some issues with it previously (although I don't remember which maps gave me trouble) but those issues might have been more due to the lack of mipmaps and/or the faulty header values, rather than the RGBA format.

 

Thanks again!

[PsiBug]

16 hours ago, AboveTheChemist said:

Did any of the '100% working' XML files you examined come from the original VidiotMap files that Blondeshell posted?

Hi,

For my 100% working samples (EDIT- unfortunately, not working 100% correctly. My edited map changes resolution when the world texture quality setting is changed, but it SHOULD remain at full resolution, so more testing is needed), I used map "map_v_city_06_01.texture" from September 7, 2011 and "map_V_PvP_02_01.texture" from July 1, 2019 . I am not sure who edited them. The newer maps that do not work fully for me are all more recent than June 2020 I believe.

 

Happy testing!

Edited April 10, 2021 by PsiBug

[_NOPE_]

@AboveTheChemist for whatevewhat's worth, the DeTexturizer is SUPER easy to use, it works basically like WinZip, bit with a simpler interface. 

[AboveTheChemist]

8 hours ago, PsiBug said:

Happy testing!

I wouldn't describe it as happy, but it was thorough. Unfortunately, despite all the iterations I tried, my best result wasn't quite a full success.

 

I did a couple of control tests, one with the stock in-game map (i.e., no modified map in my data folder) and one with the original VidiotMap, and confirmed that both those maps display 100% clearly at all world texture quality options. I tested both Warburg and King's Row with the same result.

 

The best results I was able to achieve with my modified Warburg and King's Row maps was after I followed your instructions as laid out above. This method produced maps that were visible (i.e., not blank or 'scrambled') at all world texture quality options, but the maps were low-resolution at lower world texture quality settings and would appear at increasing resolution as the world texture quality was increased. The maps were 100% clear at the highest world texture quality settings.

 

I did learn that my re-texturizing script was not causing any of the issues, but it did not permit me to edit the header information so I ended up using the detexturizer for that purpose. I also gained some insight as to what some of the numbers in the header mean, not that that helped me achieve the goal of a 100% working map.

 

PsiBug, if you are willing, I propose an exchange of our 'best' maps, to see if my maps appear any differently on a different system, and to see if I can gain any further insight from your map. If you are agreeable to that, I can PM you, or you can feel free to PM me as well.

 

Thanks!

[PsiBug]

Oh drat, I assumed since my edited map was visible at all quality settings that it was were working as intended, but I see what you mean. My "working" map is not behaving correctly after all.  I assumed the old maps and my edited maps were working the same. However, the old maps (and original default game maps) seem to remain the same resolution regardless of the world texture quality setting, whereas the map I edited changes resolution when the world texture quality is changed. Additional research is definitely needed.

 

We probably don't need to exchange map files. I think we have the same observations now. Thanks for catching that!

 

So it seems we need to:

1) go back to no mipmaps and somehow instruct the game to load the same map at all world texture quality settings, or

2) change the mipmaps so each mipmap is full size and looks the same, but that does not seem to be allowed (in gimp at least). I can edit the mipmaps but if I resize them GIMP won't let me resave the edited mipmaps.

3) I might try photoshop or another program if we think Gimp might be introducing or omitting some data. On that note, I opened the kings row map in GIMP and tried to re-save it using the "overwrite <filename>" feature and GIMP failed to save the file. Maybe that means GIMP does not understand some of the information in the source file?

 

 

 

 

 

Edited April 10, 2021 by PsiBug

[AboveTheChemist]

9 hours ago, PsiBug said:

We probably don't need to swap maps, I think we have the same observations now.

I agree. I feel better now that at least we are on the same page, even if the problem isn't fully solved. I'll probably take a few minutes to collate my notes/thoughts and post a brief summary of my observations shortly, if that would be of help.

 

Edit: My first inclination is to try mipmap settings other than 'Default' but I'm still compiling my thoughts, then I'll start testing different mipmap settings.

 

Further edit (1:30 PM eastern): I tried all the different mipmap filters (using Default wrap mode), as well as all the different wrap mode (using Default filter), with no luck. I tinkered with some other stuff, as well as reading up a little on what mipmaps are and do, and I started to think that maybe mipmaps was the wrong road to follow. Having just read your edit above it seems like you have also come to that conclusion, so I am going to keep tinkering with stuff other than mipmaps.

 

Last edit for a bit (4:15 eastern): I tried exporting using all the formats (aside from RGB and RGBA) that seemed logical with no luck. I definitely think mipmaps are not the way to go, and I am beginning to think that maybe it's something with GIMP.

 

I compared the header info between the stock game maps and VidiotMaps for 9 different maps (and I plan to check some more). Of the 9 I checked, 7 were identical. The two that were not were Siren's Call (which has been edited by the HC staff, the UnknownInt value was 0 instead of 65730) and Warburg (which makes sense because the VidiotMap for Warburg is 640x640 whereas the stock Warburg is 512x512).

 

I think the image dimensions and file size matter more than I realized, and I think they are clues to the format used to export each image. I think that some maps need to be exported as RGBA, and others as RGB, and not just all RGB as I previously thought. For instance, the file size for Warburg (769k) corresponds to a 512x512 image with 3 channels (RGB), and the file size for King's Row (2049k) corresponds to a 512x1024 image with 4 channels (RGBA). That doesn't solve the issue of the problematic maps at lower world texture qualities, but I think it is an important part of the process.

 

I think I have exhausted all the reasonable export options available in GIMP, and I am not sure what else to try at this point. I don't have immediate access to any other software (like Photoshop) that is capable of editing DDS files, but I'd be willing to make that investment if I knew another software product would work. I may see if I can track down the dev that worked on the Siren's Call update, to see if I can get any insight from them.

Edited April 10, 2021 by AboveTheChemist