Author Archives: Marcel

Cueshell – The definitive file manager for the QL

24 years ago Albin Hessler released his file manager Cueshell into the world. It was and still is one of the most advanced programs for the Pointer Environment ever written, featuring a Multiple Document Interface (smaller directory windows within the larger application window) and drag&drop file management. Managing large directory trees became incredibly easy and I still use it very frequently. It was truly way ahead of its time and this might have been its problem, as commercially I think it wasn’t as successful as hoped. Some people might remember that we did a special QPC+Cueshell promotion bundle in the early days to help adoption, but I don’t know how well those did.

Some 13 year ago I made a feebly attempt to update it to the new WMAN2 system palette, but because visually it was so advanced and non-standard I never truly managed to do a version that would do it justice, so I still use the 4-colour version. It has aged quite well however:

A few days ago I asked Albin if he could provide me with the latest manuals etc so I could release it for free and he readily agreed. The manual, too, turned out to be very beautiful, with many screen shots and illustrations. So today I present to you the latest version Cueshell v2.14, I hope you take some time to check it out because it really deserves it.

Cueshell 2.14
Cueshell manuals

QL-SD news

(Summary at the end if you’re not interested in the details)

Some people know that I’ve been working for quite some time on making the QL-SD work with the late Stuart Honeyball’s masterpieces, the GoldCard and the SuperGoldCard. And I seem to have succeeded, but the way to success took a lot more time than I anticipated. A mix of simultaneous problems in the hardware connections, bus signals, driver software and CPLD chip made for quite a difficult debugging experience. Small changes in the CPLD code that should not have any logical impact made the thing not work anymore at all, etc. Hardware is weird! Good thing I don’t have hair anymore anyway.

But now that the problems appear to be solved there is no supply of new QL-SDs to spread the joy! Dave mentioned he’s working on an external version, but I quite like the internal ones and quite frankly after all this work I didn’t mind the idea of generating at least some small revenue for my time for a change. So two weeks ago I decided to have a go myself and designed a new replacement board that is a bit easier to manufacture than the originals but other than that very similar. I ordered a batch of 50 gold plated PCBs on the good faith that they will simply work and yesterday DHL express delivered the precious cargo:

I soldered one prototype and, surprise surprise, it worked flawlessly from the get-go! Pure adrenaline 😉 The second board, made with lead-free solder this time, was a bit more stubborn but eventually succumbed to my superior solder skills (just kidding). Here’s a sneak peek:

Connoisseurs will notice that I’ve used bigger traces and components on the PCB to make it easier to produce, the space is there after all. What I haven’t finalized yet is the design of the MDV daughter board, which is really trivial but I wanted to wait if my main board worked before I invested even more money! Also I had the idea to provide a dual-card board and even made a working prototype out of my existing QL-SD adapter:

QL-SD dual card adapterQL-SD dual card adapter

What I didn’t anticipate were the small rails holding down the MDVs in the top of the case, so the whole thing almost but doesn’t quite fit. I’m thinking about lowering the main PCB to be flush with the screw-posts but I’m not sure how mechanically sound this would be and if it’s really worth the trouble. Tell me if you’re interested! I also tried a completely different approach, but that was more a thing done in jest, using an existing ventilation hole in the QL case (that sticker was actually put onto my QL about 28 years ago 🙂 ):

TL;DR

  • QL-SD works great now with GoldCard INGOT 5 including Tetroid clone (I think INGOT 6 already worked before) and SuperGoldCard. The original hardware only needs an update of the CPLD code for this. I will probably provide an update service for a fee, tell me if you’re interested.
  • I have submitted my fixes and changes to the driver to Wolfgang (thanks again Wolfgang, without his driver I wouldn’t have started my quest!), he said he will check them out soon.
  • I started producing a new batch of QL-SDs, though hand soldering them is a bit tedious and I will have to see if I can speed up the process. Price has yet to be determined.
  • I created a prototype dual-card adapter, but mechanically it’s not sound yet and even if these problems are overcome it might be difficult to produce. Anybody interested in this? It is 100% compatible with the original QL-SD, of course.

So, tell me if you’re interested or have any other comment.

Debugging Minerva

Martyn Hill recently discovered a bug in later versions of Minerva, but couldn’t quite pin it down. I thought it might be nice to write up a little tutorial about how I try to tackle these kind of problems.

The most important thing of course is being able to reproduce the problem, which was the hard part here as I haven’t had any networked QLs for 20 years. I didn’t remember any of the commands and devices used and somehow one ULA died along the way, which didn’t exactly help matters. But in the end I got it going and yeah, I could get it to crash:

The “LBYTES” never returns here. The first thing to do in these cases is loading QMON and simply activating it

This way we will ideally break into the debugger if anything fishy happens. So, try the repro again and bingo:

Excuse the quality, at this point I didn’t know I will be writing about this, I made the picture just in case I need any data from it later in the process! Two things will look odd for people experienced in QMON, the “__000000” and the “MULS.L”. The first prefix “__” is an extension that displays addresses relative to the job base, which makes debugging jobs so much easier. In this case the JOB is SuperBasic, which has a start address of 0, so we’re actually at the absolute address 0 here. “MULS.L” is a 68020 instruction and now I can be glad I did this on a BBQL because this way the crash happened as early as possible. I think both the job-relative and the 68020 features were added to QMON by me some 15 years ago and were probably never released, but it doesn’t matter for this exercise.

The next step is to look at the stack, hence the “d2 (a7)”. There we see “00000D28”, which could potentially be a ROM address. In this case I have assembled Minerva myself so I have the corresponding _MAP file ready, which is invaluable when debugging assembler code:

This looks legit, so we open the source file “io_serio_asm” and also have a look into DISA to check which instruction exactly is there

Bingo, there is a jump instruction directly in front of it, so we’re at the right place. We look for it in the source code

It jumps to a4 and, referencing the picture above, a4 is 0. We can also see the contents of the saved registers on the stack “0000000F” for D4, “00000000” for d5, “000b6568” for a1 and again “00000000” for a4. The next long word is “00000d52” which again looks like an address, so where are we coming from? Again, looking it up in DISA and then searching the instruction sequence in the source we end up at the label “headr1”

Ah, so we were calling io_pend, let’s look at this code

Lau commented out the “moveq” because he thinks d0 is 0 here anyway. We have a look at “vectest”

where we see that D0 is used as an index but should be the same going out as going into the function. No we have a look at the QMON screenshot and see that D0 is actually “00000058”! So the assumption that it would be “0” doesn’t hold and this explains our crash. We include the “moveq” in the source code, reassemble and burn the EPROM and yes, it doesn’t crash anymore. Happy days 🙂

The whole process maybe took 10 minutes, it was actually a pretty straight forward. For sure it took a lot longer to write it up 😉 I know Martin has been working at this for quite a bit, so I thought I’d share my method in the hopes that more people can find and fix these kind of bugs in the future.

For everybody who is not at home assembling Minerva I have added an English and German version with the fix here:

Minerva 1.98 with I/O fix (English/German)

Clonetastic: QL-SD clone working with GoldCard clone

Every since I got my QL-SD interface I was annoyed with the driver support, especially on the PC side. I didn’t manage to fill the SD card without destroying the image one way or another. Then came Wolfgang Lenerz awesome QLWA compatible driver and now it works like a charm.

But only on my original QL, the QL-SD always had problems with the (Super)GoldCards of the world and especially with my Tetroid GoldCard clone, where it didn’t work at all. The general assumption was that the QL lines are too noisy and the chip employed on the QL-SD too fast so that all sort of trouble happens. To check for this I started building a QL-SD clone (thanks to Peter Graf for open-sourcing the hardware!) using a slightly older Xilinx XC9572XL architecture. But before I could finish that I finally did some measurements myself and found something interesting concerning my GoldCard:

When the ROM area is accessed the CPU puts the address on the bus and the motherboard generates the ROMOE (ROM Output Enable) signal. This is fed to the ROM chip, which in turn outputs the data for the address on the data bus. When the data read is finished, ROMOE is disabled again, the ROM stops driving the data bus and then the next address is put onto the address bus. Pretty simple. But now let’s look at this on the GoldCard, upper signal is ROMOE, lower signal is A0:

ROMOE/A0 signals

ROMOE/A0 signals

One can clearly see that the address line A0 begins to change even though ROMOE at this time is still high! This doesn’t harm much when accessing a ROM, it would just output some garbage which will be ignored anyway, but with QL-SD it’s a bit more complicated. This illegal state can trigger actions in the interface that were not requested by the software, especially when the driver code is executed from the ROM, too, as then obviously there are many more accesses to the ROM area. This also explains why some people apparently had more luck getting it to work when the driver was purely executed from RAM. Anyway, this can be fixed fairly easily. Problem is, I don’t have a Lattice programmer to change QL-SD, though one is on its way from China right now. But I did have my clone, so I worked with that:

Looks pretty wild, right? But what can I say, despite the long lines and everything, once my changes to the Verilog code were uploaded to the chip the interface began to work, without a single fault so far, even when the driver is executed from the ROM. With the original Verilog code my clone didn’t work at all, so it’s definitely not just the different chip.

Next step is waiting for my Lattice programmer to arrive (apparently today it was put onto an Airplane) and to reprogram the original hardware. I’m fairly confident that this will work, too, at least for the GoldCard. So far I still was not able to acquire a SuperGoldCard :-(, so I cannot tell in how far this would help there, too. Stay tuned!

Sandy SuperQBoard – now in HD!

A few years ago I got my old QL system out of storage and into working condition, but after 20 years of using QPC I completely forgot how much of a pain it really was. I never owned any of the Gold Cards, in fact all my BBQL hardware (640KB internal memory expansion, a 192kb battery powered static RAM-disc card (“MOS-Card”) and a Sandy SuperQBoard clone) were all hand-built by my late father, together with his work colleague Jochen Hassler (he jokingly called the Sandy clone “Herbert” card in the source and documentation, after my father).

Two major points I didn’t remember was how painfully slow the QL was and also that my setup could only read and write DD discs. Regarding the later point I found schematics of a HD capable version years ago, also designed by Jochen, and I went and checked them out. Though it’s not much, it took me a while to find all the differences in the schematics and understand what they did (remember, I’m not a hardware person!). I started to design some “upgrade board”, but many months passed without doing much of it and eventually I was able to buy one of the new Tetroid Gold Card clones (finally! Not the SGC I hoped to get one day, but at least progress 😉 ), so both of my problems were solved in principle (if only my QL-SD worked with it, but that is a topic for another time).

But the idea of a hardware upgrade for my old and trusty Sandy card still intrigued me and after Christmas I said “what the hell” and just tried it. This was the prototype:

And believe it or not, despite all the noise and the inherent problems with this solution it still worked 🙂 Not from the get go, there were some connection problems, but eventually. So I went back to my PCB design. Originally it was supposed to be a little board that can be fit somewhere on the Sandy board with lots wires to different parts of the board, but after realizing that almost all signals I need are already available on the WD1772 pins I designed it as an intermediate board between the controller board and the chip. In fact there is only a single additional line needed, the signal that ultimately selects between DD and HD, everything else is neatly handled by the design. So two days before new years eve I sent it to China for manufacturing and only 4(!!!) days later I got this back (shoddy soldering all mine):

Of course in my enthusiasm I forgot to include the decoupling capacitors for the two chips, but with the short lines, a huge ground plane on the bottom and the WD1772 capacitor so close-by it apparently doesn’t make any difference. So after all this time and effort this is what the result looks like and what can I say, it works like a charm:

Interestingly the same board could potentially be used to upgrade a TrumpCard, in this case however changes to the PAL and an upgraded TrumpCard driver would be needed, too. Unfortunately I currently don’t have any TrumpCard sources, even though considering that both use a WD1772 chip the difference to the Sandy driver should not be that great. The Sandy HD driver itself is a prototype that was also never released into the wild, I think, and it is unfortunately too big to still fit into 16kb (as it includes the ATR device), so combining it with TK2 is somewhat of a challenge. But then, nobody who every combined these two (except TT) had the sources before 🙂 So, let’s see what the future might bring…

Anyway, this was my little quest, thanks for reading and a happy new year!

QMenu v8.03

Thanks to Jochen Merz I can now release a very new version of QMenu (menu_rext), along with the original manual: QMenu download page

Earlier releases done by me had a few annoying bugs which I was now able to track down. The problems actually were not located in the QMenu source code itself, but Jochen had to slightly adapt his copy of the Tony Tebby libraries, changes which were sadly missing in my copy of those libraries… a pointer was not saved and thus all sort of havoc ensued. This was for example why the “file extension filter” in the file select box wasn’t working in any binary assembled by me.

Furthermore I tried to address the incompatibilities with QDT. It took me quite a long time to realize that the problem is actually pretty simple: QMenu v7.68 added a “move” button to the LIST dialog and of course this increased the size of the dialog somewhat. No big deal, right?

Well, QDT, not having any main window, employs a special EXE to invoke QMenu and the window outline this EXE provides only has enough space for the smaller pre-v7.68 menu… in v8.03 I have now included the old menu as a fallback option for this special case when the newer menu doesn’t fit. This way QMenu seems to be compatible to QDT again, but if you know of any more problems please contact me or write on the ql-users list.

Updated QPC2 for Mac OS X

Peta Jäger sent me an update for his QPC2 for Mac bundle. It comes in at a hefty 544MB and includes the latest Black Phoenix distribution, QMovie and QBase. Get it from the QPC downloads page.

QPC2 4.0.5 for MAC

QPC2 4.0.5 for MAC

Category: QPC

SBasic Bug Boogie

This will be a highly technical post, but as the last one got some positive feedback, why not 😉 Recently on the QL-users mailing list there was a bug discussed that if you load new Basic extensions using LRESPR within a basic PROCedure or FuNction SBasic would sometimes crash.

Wolfgang did some initial research and found the place where things eventually go wrong but wondered if it’s really worth the effort to try and fix it. And rationally speaking it’s absolutely not: the bug is relatively obscure and the SBasic interpreter code in SMSQ/E is so complex that it takes me huge efforts to barely understand parts of it. But being stubborn I went ahead regardless, starting with the findings Wolfgang already provided.

During the search I found references to bug fixes in the code I made 10 years ago about which I have no recollection whatsoever. The fixes are so deep in the innards of the beast that I apparently once understood what it does, but not anymore. So this time I want to record a few thoughts so maybe I don’t have to relearn everything from scratch next time.

Parser

The SBasic main loop starts at sbm_loop (main_asm). It provides the command line interface and it’s the place where I implemented the command line history. Entering a command using the keyboard or loading a program from disc is basically the same: the strings are read from the channel and fed into the parser stage (parse_asm). It does some basic error checking and translates the commands into numerical “parser tokens”. So 5 spaces for example become $8005 and “IF” becomes $8103. The tokens are defined in the file “parser_keys”.

Compiler

When the program is RUN the action starts at sb_execute (execute_asm). First it calls the compiler at sb_compile (compile_asm) which translates the “parser tokens” into “compiler tokens”. Compiler tokens are similar but not entirely like the parser tokens: “END” and “FOR”, for example, are two tokens in the “parser tokens” space, but when preparing for execution combined into a single “END FOR” token in the compiler token space. These tokens are found in the comp_keys file.

In the next stage the compiler tokens are compiled into operations (compop_asm). This creates the actual stream of operations that is later executed by the interpreter. Pretty much all control structures like GO TO/GO SUB/PROCedure, FuNctions, SELect, FOR/REPeat loops and even IF THEN ELSE are compiled into (conditional) GOTO jumps. At this stage these jumps are still addressed by line- and statement number.

From the operation stream a statement table is built (cmpstt_asm). This table translates line/statement numbers into absolute addresses within the stream.

During the next stage (cmpadd_asm) and using the just built statement table, each line/statement number in the operation stream is replaced by the absolute address within the stream.

Next a table of the DATA statement locations is created, which concludes the compile phase.

Interpreter

After compilation the interpreter (inter_asm) is invoked. It sets up the data structures and uses a jump table to jump to the actual code blocks that executes the operation tokens in a tight loop (sb_iloop). The code snippets that correspond to the operations have the prefix bo_, so the “+” operation for example is executed by bo_add. This is all done by a fairly complex macro, so if you search the source for bo_add you will only find the code itself but no location from where it is called.

Complex structures like PROCedure calls are split into many different operations, like bo_spcall to setup the call, bo_dospr to actually do it and a lot of operations to set up the parameters (bo_formp amongst others). bo_return handles RETurn statements and also things like END DEFines.

The bug

SBasic crashed in the bo_return code because it a) wanted to clean up the PROCedure parameters and b) wanted to return to the location following the PROCedure call in the operation stream. So what happened? The LRESPRed extension calls sb.inipr (inipr_asm) to link in its new commands. This just results in an extension of the name table, which in itself is not a problem and which made me really wonder where things go wrong. It took me a long time to see that these three little lines cause all the ruckus:

The LRESPR function that is loading and executing the extension is itself called in sb_icall, which is called from bo_docpr. This checks the sb_cont flag and returns to sb_istop if not set anymore. As the name suggests this stops the interpretation of the program and returns all the way up to the main SBasic loop. The loop checks which action is to be done, which is sb.nact (no action) and then it COMPILES THE PROGRAM AGAIN! This was a bit surprising at first but if you think about it it’s clear that this needs to be done because we’ve just loaded an extension with new SBasic procedures and functions and the programmer might like to start using those now and they weren’t known when we last compiled the program! But now we’re in the middle of a procedure, how can we just continue here? Enter the “return stack”.

Return stack

As the name implies, the stack holds all data necessary to facilitate returns from PROCedures, FuNctions and GO SUBs. Mainly it holds the return addresses in the operation stream and more data pertaining to the parameters. These are absolute addresses… which is usually not a problem… unless we recompile the program during execution and it so happens that the whole instruction stream data block moves! 😮

First fix

What do we always do in SBasic if a block can move? Correct, we just make all pointers relative to the base. Actually understanding the code to a degree to find all those pointers is a wholly different matter, though. And in the end it was all in vain, this did fix most of the crashes but not all, because not only the block as a whole can move but also the operation boundaries within the block, so even relative addresses were not good enough. And as I’m not happy when something only works 50, 80 or even 99% of the time this meant going back to the drawing board.

Final fix

The final code is curiously both more complex and less invasive at the same time: before the program is re-compiled after the LRESPR the return stack entries are now translated from absolute addresses into Line/Statement numbers (using the previously mentioned statement table). The Line numbers are invariant to the compilation process, so after the compilation they are again translated into absolute addresses which always give the correct result, no matter how the compilation changed the operation stream. So the new code should have zero performance and stability implications for ordinary executions and is only active when you LRESPR things. And neither Wolfgang nor I managed to crash it anymore, yay!

The end

This, quite frankly, was a lot of work for such an obscure bug. Why did I do it? One reason is that I’ve been suffering under the bug for a decade without even realising it: when I was toying around with ProWesS I always loaded it conditionally using a PROCedure, which sometimes crashed and I could never explain why and just assumed ProWesS itself was the culprit… the other reason is that stress levels at my real job are currently very high and these puzzles are actually somewhat relaxing sometimes.

No matter the reason, SMSQ/E 3.31 is out and you can enjoy your crash free BOOTs and whatnot from now on. Have fun!

Toolkit II – The sequel

How it began

As any QL owner will know, the Toolkit 2 from QJUMP/Tony Tebby was THE toolkit without which a QL was almost unusable, arguably its contents should have been in the ROM from the start. And I always thought it was quite a shame that such an important toolkit hasn’t been updated in two decades. This got me thinking that most TK2 source files at one time were incorporated into SMSQ/E and that with a little bit of work it should be possible to re-create something resembling TK2 from them again. Turns out I was right, but also completely wrong in how “little” that work is. But still, too many hours later, I presented a brand new Toolkit II release I arbitrarily labeled version 2.30.

This is now based on the latest SMSQ/E source code, with new SMSQ/E features left intact if it was feasible (e.g. the “LOAD” command now automatically tries to append a “_bas” extension to the filename) plus many commands that have been added in the last 20 years (EXF, EX_M, FET, FEW, FEX, FEX_M, HGET, HPUT, JOBID, LGET, LPUT, UPUT, WGET and WPUT).

Unfortunately there are also parts of the source code that were completely missing, like the extended MDV driver. In this case I completely reverse engineered it from an existing ROM binary in a way that you can’t tell anymore that this isn’t the original source code.

The old network server code is actually still supplied with the SMSQ/E source code even though it’s not actually used there. The problem with that is that it’s just too big to fit into the ROM anymore and it probably doesn’t make much sense to run it from RAM because of the different timing. Therefore I didn’t include it for the initial release. The ALARM and CLOCK commands had a similar fate.

The ALTKEY code, too, is included in SMSQ/E without seeing any usage. I didn’t want to include it at first, because with the Hotkey System II it’s very much obsolete. But when tinkering with my QL system the HK2 is often not loaded yet and it drives me crazy when ALT+ENTER doesn’t work to recall the last line. So it went back in. Problem then was that the result was about 200 or 300 bytes too big, so I removed the ALTKEY code but left the ALT+ENTER code in.

Let there be networks

The basic problem is that TK2 already didn’t quite fit into 16KB during its last official versions and it became a lot worse once I packed in all the new features available in SMSQ/E. To make it fit again I had to remove something and I chose the network driver code, last having used it myself 20 years ago to prank my dad. Some people weren’t happy with that, which was to be expected, but there is one point which did stick: the network code cannot even be loaded afterwards as the timing needs to be precise and this is (almost) only possible when the code runs from ROM.

So when I found a way to free a few more bytes in the ROM and I thought about what I could do with them and eventually came up with the idea of splitting the network code: I re-added just the timing critical hardware access routines to the ROM so that the network driver can be loaded later and still work. I think this is a fairly good compromise that would make most people happy.

Martyn Hill kindly beta tested the version for me and later inquired if one could have a full network stack in the ROM and remove some other stuff instead. I was hesitant because for me TK2 is foremost a SuperBasic toolkit and most of the remaining stuff is the basic commands. There is only one part that is even a bit bigger than the network code: the “ED” SuperBasic editor. It includes many improvement over the original TK2 code and thus grew considerably in size. Removing it also has the advantage that you wouldn’t put it into actual SuperBasic code, so the toolkit stays 100% compatible. For me personally “ED” is about a million times more useful than the network code but if you absolutely have to bootstrap a QL without any file systems over the network this is the way to go. Also I do provide a standalone binary for loading later.

Choose your poison (downloads)

So here are all the different flavours:

TK2 v2.36 ROM version (with separate NET_bin file for full network support)
TK2 v2.36 network ROM version (with separate ED_bin file for the SuperBasic editor)
TK2 v2.36 RESPR version (this version includes everything though the network code will not work unless burned into a ROM or executed from zero waitstate RAM)
TK2 v2.36 QL-SD version (RESPR version that works together with the low level QLNET routines included in the QL-SD ROMs >=1.09)

Version history

QMake released for free

This has been a long time in the making, actually I wanted to do this many years ago, but the day is finally here: QMake is here for everybody to enjoy. QMake is a “make” tool which basically takes a linker file as input, collects all files that make up the executable and assembles any where the source is newer than the relocatable file. You can also add dependencies like “reassemble win1_fu_asm if win1_keys_bar changes”. I’ve used this tool for well over 20 years to build SMSQ/E and all my other projects and consider it to be pretty much essential.

I’ve created a new page for it, check it out here.