Background
In Germany there was a famous commercial that starts with the sentence “The history of menstruation is a history full of misunderstandings” and the same can be said about QDOS file headers. So what’s it all about?
On today’s PC file systems every file has some meta data associated with it. The main one is the name, obviously. But then there are also the file dates (creation, modification, last access) and a few flags like “file is hidden”, “file is a system file” and so on. Additionally files now have detailed access control lists that define which user can do what with it.
But apart from that little is known about the file itself, even the type information is only encoded more or less by convention in the filename by the “file extension” at the end of the names, like “.txt” or “.exe”.
When QDOS was created they wanted to be a bit more clever and they made room for additional information about the file, mainly the file type and some type dependent data. With the introduction of TK2 even more fields were added, like the file version and backup date.
In the end this meant for the QL that an executable can only be started with EXEC if it has the correct file type encoded in the header and the so called “data space” is also correctly set. “Data space” is basically the amount of RAM the executable needs for its variables and unfortunately it’s not simply included in the file itself but is part of the meta data associated with the file. And this is where things get hairy.
As we all know simpler operating and file systems have won the race and they do not have space for the additional data the QL needs, so a conglomerate of different solutions have been developed to cope with this. Let’s start with ZIP files.
ZIP files
There were many archive formats in the past of the QL, but only ZIP files have survived as they have become some sort of international industry standard. In the late 90s Jonathan Hudson ported the famous InfoZIP package to the QL which is still in use today. Fortunately the ZIP format has defined a so-called “extra field” to save OS specific data, which Jonathan made a good use of and saved the QL header within this field so all all meta data survives being zipped… but, and this is the catch, only as long as it is unzipped on a QL! PC based unzip utilities don’t know what to do with the extra information and in fact have no place to put them anyway on PC based file system, so all meta data is lost.
Emulator style headers
Emulator authors like myself were always thinking about ways to also somehow preserve the header information on PC file systems, but each method has certain advantages and drawbacks. QemuLator was the first to come up with an inline-header style format, meaning that the actual file contents is prepended by a small data block that starts with the the text "]!QDOS File Header", followed by the header data and finally the actual file contents:
I must note here in the strongest term possible that this is a feature exclusive to emulators. This type of file header is recognized only by QemuLator, QPC2 v5 and SMSQmulator so everything works as expected within them, but when zipping the file on a PC, the header information is not magically transferred to the “extra field” mentioned above. This means that when such a file is unzipped on a QL, not only will the header information be missing, the file will also start with garbage from QDOS’ point of view!
So how can one create QL compatible ZIP files in this case? Simply by not zipping the files on the PC but within the emulation. For the emulated system the data block is transparently removed, it doesn’t even know that it’s there, and all information is placed in the fields it expects them to be in, so the ZIP utility can get them as always and again place them in the “extra field” where they belong.
XTcc
Some users may also have heard of the “XTcc” trailer, what’s up with that? I think this was invented by Jonathan Hudson who was developing his software by cross-compiling it on a Unix machine, so he needed a way to preserve the “data space” value of the executables some way. For this he introduced special code into the Unix ZIP executable (not enabled by default, must be specially compiled with the “QLZIP” flag enabled) that scans for the XTcc trailer at the end of files and, if found, creates the standard QL compatible “extra field” information in the ZIP file. The XTC68 C compiler emits the XTcc trailer, so all he had to do was ZIP the files on his Linux box and get a completely QL compatible ZIP in the process.
In conclusion
I hope this clears up some of the confusion around QDOS file headers. As for QPC I fought with myself for many years about how to deal with them, I even had a fully working solution based on NTFS alternative data streams. This is a little known Windows feature where a file can not only have one contents but several at the same time, acting more like a directory in this case. I simply hid the QL header there, invisible to the user, a solution that has many advantages over and is much simpler to implement than the inline header explained above, but of course also has several drawbacks. In the end I opted not to introduce yet another incompatible solution to the world but instead play nice with the existing QemuLator solution so that at least these two systems can talk to each other seamlessly.
Anyway, if there are further questions simply ask them in the comments so that I can amend the article if needed.