VidiotMaps for Issue 24 and Beyond

[AboveTheChemist]

Regarding the numbers in the XML file, the ImageFileLength value seems to be the file size of the corresponding DDS file, in bytes (see screenshot). At least, for the handful of DDS/XML file pairs I have checked, which has included both VidiotMap and stock game maps, these numbers have always matched. I noticed that for the handful of my modified maps that I exported with mipmaps (except Warburg), the file size is 1 byte off from what I would expect. For example, both the stock and VidiotMap map for King's Row (the stock version is shown in the screenshot) have a file size (and ImageFileLength value) of 2097280 bytes. My exported DDS for King's Row has a file size of 2097281 bytes. I tried setting the ImageFileLength value in the XML for my modified file to both values, but the results were the same.

 

 

The numbers for the various versions of the Warburg maps were all over the place, but the Warburg map is a bit of an odd duck so I am not going to worry too much about it at the moment.

 

I also tried altering the UnknownInt value independently, but that did not seem to have any effect. HeaderLength varies by map as well, and it seems to be tied the length of the FileRelativePathAndName value, but I don't fully know how the HeaderLength is determined. I generally left that number alone.

 

My process to date has included conversion from texture to DDS to BMP, then the BMP was modified and reconverted to DDS and texture. I have used several pieces of software to do this, including Paint3D for most of the image editing. I did a trial run using just the detexturizer to convert to DDS, and then editing/re-exporting the DDS using GIMP alone, but that seemed to make no difference at all in the file size or quality of the final product. So I feel reasonably comfortable that my overall process is sound, although I won't rule out that there may be some small tweak in the DDS export process (like using a different mipmap setting) that will solve the issue.

 

I did tinker with the RGBA export and while I initially didn't have much luck, after testing some other stuff, and especially after it seemed apparent that mipmaps are probably not the answer, I think exporting some maps in RGBA may be the correct format. (this part was edited)

 

I tested some other odds and ends but I think that covers the most relevant bits.

Edited April 10, 2021 by AboveTheChemist
update about RGBA format

[PsiBug]

Hi!

Good news:

I think I have pinpointed a working method to edit maps that behave exactly like Blondeshell's maps. The file size for my test file remained the same as the source file and the process was pretty uncomplicated. You were right about not using mipmaps

Bad news: I ended up going to Photoshop CS5 and the NVidia DDS plugin to achieve these results. The save options I chose in photoshop were "8.8.8.8 ARGB 32bpp" which I believe means an RGB file with an alpha channel and 8 bits per channel for a total of 32bits per pixel.

 

My process was to:

0) install the DDS plugin which I downloaded from nVidias website. There is a version for photoshop CC and a legacy version for photoshop 5 through CS6 (which I used)

1) use the detexturizer (make sure the XML option is ticked) to separate the .texture source file into a .dds and a .xml

2) open the .dds file in photoshop, using the default import options ("load default size" I think). Edit as desired, flatten image (flattening may not be necessary)

3) "Save As..." (select dds as file format) input a file name and select folder location, click Save button.

4) (a menu should open with a variety of options) select no mipmaps, then from the top dropdown menu choose 8.8.8.8 ARGB 32bpp, then click the Save button

5) Use Detexturizer to recompile the .texture file. You can edit the .xml file first if the file size of the .dds has changed, but that does not appear to be necessary.

 

Using the above steps, I edited the three files attached below (which were previously giving me trouble at very low, low, and medium world texture quality), for anyone who wants them.

 

(NOTE: close the game before replacing any game files or the game will crash.)

-PsiBug

 

map_city_05_03.texture map_V_PvP_03_01.texture map_V_PvP_04_01.texture

Edited May 5, 2021 by PsiBug
additional info and attachments and grammar

[AboveTheChemist]

On 4/11/2021 at 10:34 AM, PsiBug said:

Good news

 

Thanks for looking into the Photoshop options, PsiBug! And my sincere thanks in general for all your help in working the bugs out of the map creation process. I will purchase a copy of Photoshop (or maybe not, see edit below) and use that going forward with my map projects.

 

With your permission, and with appropriate credit attribution, I would like to replace the problematic files I originally included in this post with the updated files you posted above.

 

EDIT: I have since generated corrected maps of my own and made them available in the linked post above.

 

Thanks again, I owe you a beverage of your choice should we ever meet in person!

 

Edit: I wonder if the NVIDIA Texture Tools Exporter standalone would work? Current Photoshop seems to only be available via subscription only, and both it and older standalone Photoshop software (via eBay) have somewhat prohibitive pricetags.

Edited April 22, 2021 by AboveTheChemist

[PsiBug]

Thanks! Yeah feel free to use or share them however you wish!

I will have my beverage preferences sorted out just in case.

Maybe the nVidia standalone tools will work. I tried them but they are command line (DOS prompt) utilities and I didn't get very far with them.

Yeah, I stopped upgrading photoshop at version CS5 because of the incredible prices.  I think you can use CC as a free trial for 30 days (each time a new version is released I assume) if that interests you.

And maybe we can still figure out some way to match the photoshop settings in GIMP. or something like PAINT.

 

(disclaimer: these are just links I have found. As always use your own discretion regarding any downloads or link clicking)

PAINT (free program) link: https://www.getpaint.net/

link to forum thread about DDS plugin included in PAINT (I think A8R8G8B8 mentioned in this thread may be equivalent to the file format I used): https://forums.getpaint.net/topic/111731-dds-filetype-plus-04-10-2021/

 

=PsiBug

 

<EDIT> I installed PAINT.net and tried the two A8R8B8G8 options (linear and sRGB I think) and they seem to provide the same result as GIMP (only displayed properly at high or very high world texture quality settings) unfortunately. Possibly there is still some option in one of those two programs that works, but I am doubtful.

 

 

Edited April 12, 2021 by PsiBug

[Pouncy]

AWESOME!  Thank you whole community. I was going to list people then it got too long. Everyone who did agonizing testing, leg work, research, and anything else.  Fantastic.

 

 

ALSO ARRRRRRG! Close your game FIRST!!!! Dam 8((

 

 

Hay Psi just above the links for the file maybe add WARNING: close game before installing.

Edited April 11, 2021 by Pouncy

[PsiBug]

hahaha, yeah I wish we could swap maps with the game running!

[AboveTheChemist]

3 hours ago, PsiBug said:

I installed PAINT.net and tried the two A8R8B8G8 options (linear and sRGB I think) and they seem to provide the same result as GIMP (only displayed properly at high or very high world texture quality settings) unfortunately. Possibly there is still some option in one of those two programs that works, but I am doubtful.

I got the NVIDIA standalone texture tool loaded and I tinkered with some exports but I have not been able to get anything to work yet that isn't garbled at the lower world texture quality settings. I plan to do a little more testing but my hopes are not high. Speaking of PAINT.net, I found a different thread that mentioned it would handle the A8R8G8B8 format so it was next on my list of things to try. Since it is free I probably will give it a go just to exhaust all options before turning to PhotoShop.

 

That thread you linked seems to be active and the OP seems quite knowledgeable so if I don't have any luck on my own I might see if anyone in that thread can give me any pointers in getting a non-Photoshop option to work.

Edited April 12, 2021 by AboveTheChemist

[PsiBug]

 

3 hours ago, PsiBug said:

<EDIT> I installed PAINT.net and tried the two A8R8B8G8 options (linear and sRGB I think) and they seem to provide the same result as GIMP (only displayed properly at high or very high world texture quality settings) unfortunately. Possibly there is still some option in one of those two programs that works, but I am doubtful.

but you may have better luck than I did.

 

[Pouncy]

Is one of those above the "fixed" Siren's Call map?  I finally got around to testing the map and its still rotated wanky.

 

I see 2 of the mare PVP so I assume Warburg and Siren's call.

 

[PsiBug]

8 minutes ago, Pouncy said:

Is one of those above the "fixed" Siren's Call map?  I finally got around to testing the map and its still rotated wanky.

 

I see 2 of the mare PVP so I assume Warburg and Siren's call.

 

It seems to be rotated correctly for me. Possibly you copied the new file to the wrong location? the Kallisti map (map_city_05_03.texture) goes in the MAP/static folder, and the other two pvp maps go in the V_MAP/static folder.

 

[AboveTheChemist]

On 4/11/2021 at 10:13 PM, Pouncy said:

Is one of those above the "fixed" Siren's Call map?  I finally got around to testing the map and its still rotated wanky.

It is, you can see details about the updated maps in this updated post. I have updated that post with the corrected maps that PsiBug posted.

 

EDIT: I have since generated corrected maps of my own and made them available in the linked post above.

Edited June 21, 2021 by AboveTheChemist
February, not April, updated linked post with my own corrected maps

[Pouncy]

Yep yep its fixed.  I did just what you said i did. I dragged it into the right folder and went having fun in Siren's Call.  Those hot zones are so fun.

Super thanks again for fixing these maps.

 

I am on Torchbearer if you ever wana say hi.

Edited April 12, 2021 by Pouncy

[tripthicket]

Here's likely a silly question, but reading through the last 2 or 3 pages of posts is only leaving me more confused. Is the download link in the original post still the one to use? Or is there a more up-to-date version available elsewhere?

[AboveTheChemist]

On 4/15/2021 at 1:47 AM, tripthicket said:

Here's likely a silly question, but reading through the last 2 or 3 pages of posts is only leaving me more confused. Is the download link in the original post still the one to use? Or is there a more up-to-date version available elsewhere?

Not a silly question at all. The download link in the original post is still the main one to use, although a few of those maps are slightly out of date. The original poster hasn't been around in 6 months or so, so a few other people and I have attempted to update the out-of-date maps as best we could. There was a slight flaw in these maps, and the last few pages have been a flurry of activity trying to identify and fix the flaw. A fix was found, and if you'd like the updated maps, they are available at this link.

 

In related news, I am 90% certain I have found the root cause of the flaw and have a fix that avoids having to use Photoshop. I still need to test a batch of maps and then I'll make a post with more details assuming the testing goes well (knock on wood!). If anyone would like to volunteer to test a couple of the maps on their machine (I don't have a second gaming setup), I would be forever grateful. I need sleep first, though, so it'll have to wait until tomorrow.

 

Edited June 21, 2021 by AboveTheChemist
remove old links, re-point to new update post

[_NOPE_]

@AboveTheChemist and if you want to update the main Map Pack in Modder with the updated maps once you confirm they work, go nuts. Or I can do it if you want.

[AboveTheChemist]

Map testing of 16 maps was a complete success, but I wouldn't mind having someone test a few maps on their machine before I consider the issue resolved. If you are willing to do so, please let me know. In the meantime, here's the short version of the problem.

 

DDS files contain a header plus the actual map image. Within that DDS file header is a flag that specifies the mipmap count (i.e., the number of mipmaps). I compared the DDS headers for several 'good' maps with those of several 'bad' maps and noticed that the 'good' maps all had that flag set to 0 (no mipmaps), while the flag for the 'bad' maps indicated 1 mipmap. I changed that flag in the 'bad' map DDS header from 1 to 0 and that seemed to correct the issue.

 

Here's a slightly more detailed version of how I arrived at that conclusion:

 

Spoiler

I ended up trying several different DDS conversion tools, including the NVIDIA Texture Tools Exporter (standalone), ImageMagick, DirectXTex Texconv, and of course GIMP, none of which worked. I also explored a solution via Python modules, as well as a solution via Paint.NET, but those inquiries led me to ImageMagick and DirectXTex Texconv, respectively. During the course of the fairly extensive Google trail that led me to those particular tools, I found some links to the Microsoft Programming Guide for DDS (which was cryptic to my relatively untrained eye, but got me thinking in the right direction). I also read and re-read the Paint.NET forum post that PsiBug linked, thinking that I might post there looking for help. I wasn't confident that I could explain the problem clearly enough in order to get the help I needed, so I dug a little deeper.

 

I downloaded a hex editor and took a look under the hood of some of the DDS files. Between that and the DDS programming guide, I got a clearer picture of what and where the DDS header was. It occurred to me that I had the tools (namely, the Python script I use to disassemble and re-assemble texture files) to essentially splice a 'good' DDS header onto a 'bad' DDS map file, and that worked! At that point it seemed logical that the issue must be somewhere within the 128-byte DDS header, which isn't a big place for a problem to hide. That approach felt a bit like surgery with a meat cleaver, so I kept digging hoping to find a less drastic solution.

 

Around that time I found this somewhat random but infinitely helpful wiki post that gave me a much clearer sense of what I was seeing in the hex editor, and I assembled as many examples of 'good' and 'bad' DDS map files as I could find to compare the two. I noted that, even among the 'good' map files, there was a lot of variation in the contents of the DDS header, so clearly some parts of the DDS header could vary without negatively impacting the in-game texture. One of the first commonalities I noted, though, was that the flag that specified the mipmap count was set to 0 mipmaps in the 'good' files, and to 1 mipmap in the 'bad' files.

 

In the 'bad' files, I changed that mipmap count flag from '1' to '0' and that seemed to solve the problem. And it made sense based on the results that were seen in-game with the 'bad' map files. At higher world texture quality settings, the game used the primary image in the file, which displayed without issue. However, the game was being told that there was 1 mipmap available, so when lower texture quality settings were used, it went looking for a mipmap that was not there and as a result, displayed the 'scrambled' image. With that mipmap count flag set to 0, it stopped looking for mipmaps, and instead displayed the primary image at all world texture quality settings. At least, that makes sense based on the limited knowledge I've gained during the process.

 

The primary modification to my process involved adding a line to my previously mentioned Python script which changed the mipmap count flag in the DDS header from 1 to 0. I also decided that, of the various DDS conversion tools I tried, the NVIDIA Texture Tools Exporter standalone worked the best for my workflow, so I will be using it going forward. I plan to eventually document my process (perhaps as a wiki article on the HC wiki) for anyone else that would like to create custom in-game maps. I tested my new process by generating 15 maps (9 in Paragon city, 3 in PvP zones, and 3 in the Rogue Isles) and checking each world texture quality setting in each zone. Each one displayed perfectly. I also made sure to check against a 'bad' map as a control, to verify I was still seeing the 'scrambled' image at lower world texture quality settings.

 

For all the time that was spent by myself and others trying to identify and correct this problem, it seems somewhat funny that the solution could be as simple as a single line in a script, in which a single character in a text string is changed from 1 to 0.

Edited April 16, 2021 by AboveTheChemist
typo

[_NOPE_]

Now, why would a simple boolean value EVER cause any issues???

 

[Nerio72]

Nice job sticking to it @AboveTheChemist
I would have gave up and started day drinking! Well done!

[PsiBug]

Excellent troubleshooting! Might be a good note for the GIMP forum too. It has to be an oversight.

[AboveTheChemist]

The most recent updates are available at this link: Click here for updated maps

 

Now that I think I have corrected the issues with my map production process, I updated the Perez Park map, which I noticed had some minor issues.

Edited June 21, 2021 by AboveTheChemist
remove old links, re-point to new update post

[AboveTheChemist]

On 4/15/2021 at 8:16 AM, The Philotic Knight said:

if you want to update the main Map Pack in Modder with the updated maps once you confirm they work, go nuts. Or I can do it if you want.

EDIT: I appear to have corrected my earlier problems with updating the map pack via CoH Modder. The latest version of VidiotMaps now in CoH Modder now contains the maps that I have recently updated.

Edited May 4, 2021 by AboveTheChemist
solved my update issues, clarification

[Pouncy]

OH, what was wrong with Perez Park?

And, is it somewhere I can download without that toolmabob?  I prefer to slip it in place by hand.

[AboveTheChemist]

On 4/26/2021 at 2:39 PM, Pouncy said:

OH, what was wrong with Perez Park?

And, is it somewhere I can download without that toolmabob?  I prefer to slip it in place by hand.

The most recent updates are available at this link: Click here for updated maps

Edited June 21, 2021 by AboveTheChemist
removing old link, re-point to new update post

[Pouncy]

Thank you.  what are those big golden round things spaced along the Northern border and along the Southern lake in Perez Park?

Edited April 26, 2021 by Pouncy

[AboveTheChemist]

16 hours ago, Pouncy said:

Thank you.  what are those big golden round things spaced along the Northern border and along the Southern lake in Perez Park?

Those are the locations where the Kraken can spawn. They are the icons for the badge that is earned from defeating the Kraken. Any zone that can spawn a giant monster will have that giant monster's spawn locations marked with the defeat badge icon for that giant monster. I have not verified the accuracy of each one, as I didn't think there was a need. I simply used the same locations as in the existing Perez Park map.