#1 2022-03-19 06:10:31

triple_agent
Member

Case sensitivity

'Linux' is a case sensitive environment, contrary to a more popular 'Windows'. Many developers and mappers in the 'Quake' community, who natively develop on 'Windows', sometimes create a little bit of confusion for the 'Linux' users, when certain files or directories in their maps or mods, are named with uppercase letters, while the game, actually expects lowercase letters to appear for that matter. That is at least how I understand it, why does the problem occur. On 'Windows', again, it is not any problem, since 'A' = 'a' there - but on Linux, only 'A' = 'A' and only 'a' = 'a', while 'A' ≠ 'a'.

The problem shows up as soon as with the "pak0.pak" file in the original 'Quake' data structure, as it comes named as "PAK0.PAK". In order for the game to work, 'Linux' users must rename that file accordingly to all-lowercase letters. Little thing, which may create a disproportionately big obstacle; not a very decent puzzle for the start.

While it is feasible to use scripts or functions to massively turn any kinds of spellings to all-lowercase spellings - even if this way out mostly appears sufficient for 'Quake', I believe it does not entirely solve the issue. The issue, is: what does the program actually expect and what does it get? The root cause of the problem, in my opinion, is a thing of precision, which in order to be solved, demands consistency from developers. The most simple way to spare some of us the case sensitivity headache, is to simply avoid using any uppercase letters in file or directory naming altogether - just to avoid potential mistakes of this nature. The question is, why not?

For the already existing issues - save for data contained in ".pak" archives - one of the easy deals that has worked well for me thusfar, is a script:

#!/bin/bash
#print usage
if [ -z $1 ];then
echo "Usage :$(basename $0) parent-directory"
exit 1
fi

#process all subdirectories and files in parent directory
all="$(find $1 -depth)"

for name in ${all}; do
#set new name in lower case for files and directories
new_name="$(dirname "${name}")/$(basename "${name}" | tr '[A-Z]' '[a-z]')"
#check if new name already exists
if [ "${name}" != "${new_name}" ]; then
[ ! -e "${new_name}" ] && mv -T "${name}" "${new_name}"; echo "${name} was renamed to ${new_name}" || echo "${name} wasn't renamed!"
fi

done
exit 0

Reference link - I did not find the other mentioned 'rename' program as easy to use as the script given, if processed as follows - on 'Ubuntu':

1. Open terminal [CTRL+ALT+T], then create a new file through command: "touch rename.sh"
2. Open the newly created "rename.sh" file, then copy-paste the code given earlier and save
3. Right-click on the newly created "rename.sh" file, enter "Properties" in the pop-up menu and enable the option: "Allow executing file as program"
4. In order to apply, drag the "rename.sh" file icon and drop it in the terminal window, then drag the target folder and drop it in the same terminal window, so that both entities, would manifest as two pathways. In the end, hit enter to execute. All files in the target folder will be renamed, without a copy; the folder name will also be subject.

Last edited by triple_agent (2022-03-19 07:41:34)

#2 2022-03-20 02:29:09

triple_agent
Member

Re: Case sensitivity

One example of the problem discussed, occurs in a map Oxyblack Fortress, codenamed: "ad_heresp1". On case sensitive systems, engine such as 'Quakespasm', encounters issues with reading data under pathway - within the map directory structure: "sound/MarkieV1/". Pathways the engine actually seeks, are thus: "sound/Markiev1/" and "sound/markiev1/", which both differ from the original name.

In this example, the engine seeks pathways with particular configurations of uppercase and lowercase letters, therefore massive conversion to all-lowercase, provides only partial solution.

The authors, apparently made two or three mistakes for this one unfortunate folder. I like to have data complete when playing a game, even if the data could be technically considered not mandatory for the game function; I must be sort of completionist, I guess. To solve the issue, one could create two instances of the original folder, by copying that folder once and renaming both present entities accordingly; one could alternatively create two appropriate, symbolic links to the already existing folder, which is a more efficient approach, in my opinion.

Not all errors nonetheless are described by the engine in much detail - which then could be saved as a textfile, with the commandline function of "condump" - therefore sometimes it is hard to figure out, running only one operating system or only one engine, whether the issue is particular or universal of nature; although 'Quakespasm', should not sound bad in terms of compatibility.

For example, with Dwell - Episode 1, codenamed: "dwellv1p2.zip", there is an issue only mentioned as such - leaving little info about further error background: "Mod_LoadTexinfo: 9 texture(s) missing from BSP file".

Last edited by triple_agent (2022-03-20 06:06:36)

#3 2022-03-20 03:32:14

triple_agent
Member

Re: Case sensitivity

Someone could ask: why do you use 'Linux', if you want to play games - does it not make things quirky, at least from time to time?

'Linux' world, has come a long way to become more user friendly and by 'Linux', I mean 'Ubuntu' - the most representative and popular variant. I am not a long time 'Ubuntu' user and I know little of 'Linux' how-to, but what I do know, is that one must be willing to learn and search things out in the Internet, if need be, in order to make the most of it. Majority of cases, there is always way out, but sometimes it takes patience to sort out. Some actually find it fun to delve into details; some though prefer to just have things work.

'Linux' or 'Ubuntu' nowadays is more user friendly and has better infrastructure than it used to, but probably, if you are a seasoned 'Windows' insider, it will not scratch your itch. I have personally moved to 'Linux', because of the marketing push in the 'Microsoft' dominated landscape. 'Linux', came out to be a place more quiet and I found it paradoxically handy that some games, simply did not work on it - there was less risk of silly involvement, including unwise money and time spenditure. By the way, I do not use 'WINE', much at least.

One could say that I came to 'Linux' because of economical reasons, although the currency I talk about, is other than monetary. I felt like the 'Microsoft' and 'Steam' big-world, is simply too polluted for me to enjoy anything at all and I mean it. That is why I moved to 'Linux'.

Why do I talk about this, particularly in given thread? The answer is, because of the visage. I do not like the observation that 'Linux', so often is associated with practical problems running casual stuff. Every time someone talks about problems either on 'Linux' or with 'Linux', it effectively works as a disadvertisement - but probably, there is truth to it; even if the thing is not all unusual in the PC realm. Casual users should not really go 'Linux', unless they are willing to simply give up on some things for the sake of ideal or proving something, which - facing it - not a lot of people nowadays are willing to do, do they? That is for the home users.

On the contrary, 'Linux' is very popular among engineers and industry-grade developers, who like to use open-source software and delve in productivity culture, that 'Linux' ecosystem is so abundant with. It is not by chance that 'Linux' mostly attracts technologically inclined users and adventurers of sort, I guess.

Truth be told, if gaming is the purpose, gaming consoles - such as 'Playstation' or 'XBOX' - are the way to go. Myself, I never had any gaming console though, since I am not particularly willing to spend much money on gaming either. Recently, the cloud gaming has become a feasible option as well - an option very attractive for low-end hardware users, so that is something; if the Internet connection, provides.

Last edited by triple_agent (2022-03-20 06:10:13)

#4 2022-03-20 15:28:28

Spirit
Administrator

Re: Case sensitivity

I'd recommend checking if your engine of choice has an option to ignore the file name case. reQuiem had that and it was awesome!

#5 2022-03-21 08:01:38

gormless.player
Member

Re: Case sensitivity

There's a much easier way of renaming Linux files to ensure they are all lower case. Install the convmv utility, available in most distributions as far as I know. By default, it will preview the changes before actually executing them, for example.

convmv -r --lower *

Which will show all the renames to lowercase recursively.

convmv -r --notest --lower *

To actually perform the renames.

#6 2022-03-21 15:55:21

triple_agent
Member

Re: Case sensitivity

gormless.player wrote:

There's a much easier way of renaming Linux files to ensure they are all lower case. Install the convmv utility, available in most distributions as far as I know. By default, it will preview the changes before actually executing them, for example.

convmv -r --lower *

Which will show all the renames to lowercase recursively.

convmv -r --notest --lower *

To actually perform the renames.

Great tip, thanks! Will maybe try it out sometime.

For the script given earlier, I have tested it against few different conditions and observed that it does not overwrite existing entities, which means that it will not convert "A" to "a", if there is already an "a" entity present. Test run as follows - I have created a folder with:
file "a" of text content "1",
file "A" of text content "2",
file "B" of text content "3".

While the terminal output informed of successful conversion of both "A" and "B" - without mention of "a" - having then looked into remaining file values, it came out that after script application, "a" maintained original "1", "A" was not converted at all and maintained original "2", while "B" was converted into "b" with the same content of "3".

Test outcome, is that all original file content was secured; nothing was overwritten. Script seems to work safely, but the feedback it gives is faulty - at least on 'Ubuntu' version "20.04.4 LTS" - as the entity "A" was not converted, despite the feedback claiming so. Besides, the script does not inform of files which were skipped in the process, while the code apparently says there is definition for it:

if [ "${name}" != "${new_name}" ]; then
(...) ; echo "${name} was renamed to ${new_name}" || echo "${name} wasn't renamed!"

I hope my reasoning in this case is correct.

Once again, renaming all names to lowercase sometimes does work, but as the example with 'Oxyblack Fortress' shows, it is not a way out. The way out, much like 'Spirit' pointed out, would be to make the engine ignore case sensitivity - or to make developers stop doing mistakes, which takes a good habit to embrace.

EDIT:

Spirit wrote:

I'd recommend checking if your engine of choice has an option to ignore the file name case. reQuiem had that and it was awesome!

My engine of choice - like my 'Linux' variant of choice - is not much a case of personal preference; it is simply what is being most popular in the general public or in the particular community that I try to communicate with. I tried few different 'Linux' variants in the past and found that what is being most popular, oft deserves the title - probably the same with 'Quake' engines.

The 'reQuiem' engine - sounds a bit like the name of that game, 'Requiem: the Avenging Angel'. It was a first person perspective shooter. Anyway, I wonder why was the case sensitivity option not historically picked up by other, emergent engines, particularly 'Quakespasm'?

Last edited by triple_agent (2022-03-21 17:21:59)

#7 2022-03-22 02:46:24

triple_agent
Member

Re: Case sensitivity

@'gormless.player', when it comes to the "star" or the asterisk symbol on 'Linux' - speaking of the commandline - to my knowledge, it sometimes can wreak a lot of havoc, since literally meaning "all"; if applied without proper knowledge, one can accidentally affect all files in own data structure, instead of just within a selected directory - especially if going under recursive mode. Simple stuff can be very powerful on 'Linux'.

Last edited by triple_agent (2022-03-22 05:38:18)

#8 2022-03-22 19:48:34

Johnny Law
Member

Re: Case sensitivity

Some miscellaneous comments on this since it's something I was looking at recently:

* Filename case only matters for "loose" files and subdirectories outside of pak files. Paths within pak files are handled case-insensitively.
* Quake expects lowercase for certain extensions and certain special directory names and filenames; for other resources though (like bsp files, textures, sounds, etc. as well as subdirectories of "usual" directories) it matters how the name is "cased" in any references to it from within bsp files or config files. So just lowercasing everything won't always fix things.
* Yeah this can be annoying on Linux! And macOS too. Anything that's not Windows.
* FTE has a case-insensitive file treatment on Linux. I didn't know/remember that reQuiem also had that feature, pretty neat.

It would be great if other engines implemented case-insensitive file loading behavior on non-Windows platforms. I can't think of any situation where this would actually break a map/mod that worked on Windows.

For engines that are NOT case insensitive on Linux, yeah sometimes you can work around the issues by renaming files. Sometimes that is problematic though because a file might be referenced in different ways -- for example if you have a file "start.bsp" that is the hub for a collection of jam maps, and some of the changelevel triggers in the other maps call it "start" while others call it "START". In that case no single filename will work for everyone, so it's usually better to create symbolic links as "alternate" names that a file can be referenced through.

FWIW, part of the feature set in my little Linux desktop integration project for Quake-launching includes the auto-fixing of common filename case errors, by lowercasing certain things that must be lowercased and (optionally) auto-creating symlinks to account for common capitalization variants.

#9 2022-03-23 01:40:48

triple_agent
Member

Re: Case sensitivity

Johnny Law wrote:

Filename case only matters for "loose" files and subdirectories outside of pak files. Paths within pak files are handled case-insensitively.

It seems like the third way out would be to always pack everything into a ".pak" archive, since regardless how the data names are cased or called, they will always be treated "case agnostic"?

#10 2022-03-23 02:01:10

Johnny Law
Member

Re: Case sensitivity

I may be misremembering about pak files unfortunately. That's the piece of info I was remembering/passing-on rather than actually testing, and now I'm looking into it and the pak file loading seems to be case sensitive too. Bleh. If that wasn't the case then yeah pak-ifying things would be a possible solution.

#11 2022-03-23 02:32:24

triple_agent
Member

Re: Case sensitivity

Johnny Law wrote:

I may be misremembering about pak files unfortunately. That's the piece of info I was remembering/passing-on rather than actually testing, and now I'm looking into it and the pak file loading seems to be case sensitive too. Bleh. If that wasn't the case then yeah pak-ifying things would be a possible solution.

It would be good to figure that out.

One way to test it, is to check whether 'Windows' users get the following error in "dwellv1p2.zip": "Mod_LoadTexinfo: 9 texture(s) missing from BSP file". I guess someone would have noticed otherwise, if the issue was universal.

Johnny Law wrote:

It matters how the name is "cased" in any references to it from within bsp files or config files. So just lowercasing everything won't always fix things.

Which is why I humbly proposed to avoid uppercase letters in any naming regardless.

Last edited by triple_agent (2022-03-23 02:32:34)

#12 2022-03-23 16:43:15

triple_agent
Member

Re: Case sensitivity

Johnny Law wrote:

if you have a file "start.bsp" that is the hub for a collection of jam maps, and some of the changelevel triggers in the other maps call it "start" while others call it "START". In that case no single filename will work for everyone, so it's usually better to create symbolic links as "alternate" names that a file can be referenced through.

True, but for that matter, you need to know how the pathway goes - you need to know the spelling. Majority of times, the engine does give enough of a problem description, but sometimes the information can be obscure. The other question is, how far one must go to fix the problem; if we talk about files in the open, it is probably much easier than trying to pry-open some archive or digging in the configurations of some ".bsp".

Johnny Law wrote:

It would be great if other engines implemented case-insensitive file loading behavior on non-Windows platforms. I can't think of any situation where this would actually break a map/mod that worked on Windows.

Since apparently every developer around here works on 'Windows' - and 'Windows' is mostly case-agnostic, save for passwords - it is nearly guaranteed that the case-agnostic engine mode, would not become the reason for any map or mod shooting error, either on 'Ubuntu' or other case-sensitive systems. In other words, case-agnostic, seems the default condition among general public and nowadays it is very improbable that someone would need to relay on case-sensitive nuances in order to organize data, especially due to exhaustion of possibilities.

In the home PC realm, 'Windows' has won again - it appears much more user-friendly and much more reasonable, while 'Linux' again comes out as a quirky stuff for nerds who like to have problems where other people are fine.

Frustration put aside, I believe the problem, can be solved easily - question is, whether there is will to it? I think the case-agnostic mode in popular 'Quake' engines was abandoned, because nobody really cared.

Last edited by triple_agent (2022-03-24 03:09:25)

#13 2022-03-25 14:57:46

Johnny Law
Member

Re: Case sensitivity

Ah man, my dumb brain.

So the thing with pak files is that access to their contents is case sensitive on ALL platforms. Therefore these case errors don't tend to creep in when it comes to stuff in pak files, since Windows users would see them too.

Anyway yeah, for loose files the best solution of course is for engines to fix it.

The idea of having a mapping standard for only using lowercase filenames has been around for a long time, and every now and then someone will mess up and someone else will remember to remind them... but a lot of folks just forget or don't know.

#14 2022-03-25 15:16:08

triple_agent
Member

Re: Case sensitivity

Johnny Law wrote:

So the thing with pak files is that access to their contents is case sensitive on ALL platforms.

It must be the reason why one would rarely see errors regarding the contents of ".pak" archives - people notice and make corrections. If they do not notice - they do not make corrections.

Johnny Law wrote:

The idea of having a mapping standard for only using lowercase filenames has been around for a long time

It seems reasonable, if you ask me.

#15 2022-03-28 16:02:17

triple_agent
Member

Re: Case sensitivity

Somewhere I have seen a mention that the 'Windows' operating system, even though practically case-agnostic by default, is capable of running in case-sensitive mode. This made me wonder, whether 'Linux', should have a likewise possibility to be switched into case-agnostic mode, on the contrary? It comes out to be true - 'Linux' can run in case-agnostic mode; it can eventually apply case-agnostic mode just for selected folders, basing on what I understood. The entire procedure of getting there, seems nonetheless somewhat convoluted for a layman and I would pity if anyone of such was forced to go that heavily custom route, possibly risking to bork the system.

Reference to an article discussing the problem.

Last edited by triple_agent (2022-03-29 01:11:48)

#16 2023-05-10 10:51:30

houston
Member

Re: Case sensitivity

triple_agent wrote:

while the game, actually expects lowercase letters to appear for that matter. That is at least how I understand it, why does the problem occur.

Actually, I remember checking my CD at some point and confirming that all of the filenames are originally all higher-case. If your source port requires them to be lower-case (as in Quakespasm, but I can forgive it since that port is so awesome in all other kinds of different ways), then that's a bug on their end.

Board footer