Yearly Archives: 2017

QMenu v8.03

Thanks to Jochen Merz I can now release a very new version of QMenu (menu_rext), along with the original manual: QMenu download page

Earlier releases done by me had a few annoying bugs which I was now able to track down. The problems actually were not located in the QMenu source code itself, but Jochen had to slightly adapt his copy of the Tony Tebby libraries, changes which were sadly missing in my copy of those libraries… a pointer was not saved and thus all sort of havoc ensued. This was for example why the “file extension filter” in the file select box wasn’t working in any binary assembled by me.

Furthermore I tried to address the incompatibilities with QDT. It took me quite a long time to realize that the problem is actually pretty simple: QMenu v7.68 added a “move” button to the LIST dialog and of course this increased the size of the dialog somewhat. No big deal, right?

Well, QDT, not having any main window, employs a special EXE to invoke QMenu and the window outline this EXE provides only has enough space for the smaller pre-v7.68 menu… in v8.03 I have now included the old menu as a fallback option for this special case when the newer menu doesn’t fit. This way QMenu seems to be compatible to QDT again, but if you know of any more problems please contact me or write on the ql-users list.

Updated QPC2 for Mac OS X

Peta Jäger sent me an update for his QPC2 for Mac bundle. It comes in at a hefty 544MB and includes the latest Black Phoenix distribution, QMovie and QBase. Get it from the QPC downloads page.

QPC2 4.0.5 for MAC

QPC2 4.0.5 for MAC

Category: QPC

SBasic Bug Boogie

This will be a highly technical post, but as the last one got some positive feedback, why not 😉 Recently on the QL-users mailing list there was a bug discussed that if you load new Basic extensions using LRESPR within a basic PROCedure or FuNction SBasic would sometimes crash.

Wolfgang did some initial research and found the place where things eventually go wrong but wondered if it’s really worth the effort to try and fix it. And rationally speaking it’s absolutely not: the bug is relatively obscure and the SBasic interpreter code in SMSQ/E is so complex that it takes me huge efforts to barely understand parts of it. But being stubborn I went ahead regardless, starting with the findings Wolfgang already provided.

During the search I found references to bug fixes in the code I made 10 years ago about which I have no recollection whatsoever. The fixes are so deep in the innards of the beast that I apparently once understood what it does, but not anymore. So this time I want to record a few thoughts so maybe I don’t have to relearn everything from scratch next time.

Parser

The SBasic main loop starts at sbm_loop (main_asm). It provides the command line interface and it’s the place where I implemented the command line history. Entering a command using the keyboard or loading a program from disc is basically the same: the strings are read from the channel and fed into the parser stage (parse_asm). It does some basic error checking and translates the commands into numerical “parser tokens”. So 5 spaces for example become $8005 and “IF” becomes $8103. The tokens are defined in the file “parser_keys”.

Compiler

When the program is RUN the action starts at sb_execute (execute_asm). First it calls the compiler at sb_compile (compile_asm) which translates the “parser tokens” into “compiler tokens”. Compiler tokens are similar but not entirely like the parser tokens: “END” and “FOR”, for example, are two tokens in the “parser tokens” space, but when preparing for execution combined into a single “END FOR” token in the compiler token space. These tokens are found in the comp_keys file.

In the next stage the compiler tokens are compiled into operations (compop_asm). This creates the actual stream of operations that is later executed by the interpreter. Pretty much all control structures like GO TO/GO SUB/PROCedure, FuNctions, SELect, FOR/REPeat loops and even IF THEN ELSE are compiled into (conditional) GOTO jumps. At this stage these jumps are still addressed by line- and statement number.

From the operation stream a statement table is built (cmpstt_asm). This table translates line/statement numbers into absolute addresses within the stream.

During the next stage (cmpadd_asm) and using the just built statement table, each line/statement number in the operation stream is replaced by the absolute address within the stream.

Next a table of the DATA statement locations is created, which concludes the compile phase.

Interpreter

After compilation the interpreter (inter_asm) is invoked. It sets up the data structures and uses a jump table to jump to the actual code blocks that executes the operation tokens in a tight loop (sb_iloop). The code snippets that correspond to the operations have the prefix bo_, so the “+” operation for example is executed by bo_add. This is all done by a fairly complex macro, so if you search the source for bo_add you will only find the code itself but no location from where it is called.

Complex structures like PROCedure calls are split into many different operations, like bo_spcall to setup the call, bo_dospr to actually do it and a lot of operations to set up the parameters (bo_formp amongst others). bo_return handles RETurn statements and also things like END DEFines.

The bug

SBasic crashed in the bo_return code because it a) wanted to clean up the PROCedure parameters and b) wanted to return to the location following the PROCedure call in the operation stream. So what happened? The LRESPRed extension calls sb.inipr (inipr_asm) to link in its new commands. This just results in an extension of the name table, which in itself is not a problem and which made me really wonder where things go wrong. It took me a long time to see that these three little lines cause all the ruckus:

        tas     sb_edt(a6)               ; edited! to redo name types
        sf      sb_cont(a6)              ; do not continue
        move.w  #sb.nact,sb_actn(a6)     ; but no action

The LRESPR function that is loading and executing the extension is itself called in sb_icall, which is called from bo_docpr. This checks the sb_cont flag and returns to sb_istop if not set anymore. As the name suggests this stops the interpretation of the program and returns all the way up to the main SBasic loop. The loop checks which action is to be done, which is sb.nact (no action) and then it COMPILES THE PROGRAM AGAIN! This was a bit surprising at first but if you think about it it’s clear that this needs to be done because we’ve just loaded an extension with new SBasic procedures and functions and the programmer might like to start using those now and they weren’t known when we last compiled the program! But now we’re in the middle of a procedure, how can we just continue here? Enter the “return stack”.

Return stack

As the name implies, the stack holds all data necessary to facilitate returns from PROCedures, FuNctions and GO SUBs. Mainly it holds the return addresses in the operation stream and more data pertaining to the parameters. These are absolute addresses… which is usually not a problem… unless we recompile the program during execution and it so happens that the whole instruction stream data block moves! 😮

First fix

What do we always do in SBasic if a block can move? Correct, we just make all pointers relative to the base. Actually understanding the code to a degree to find all those pointers is a wholly different matter, though. And in the end it was all in vain, this did fix most of the crashes but not all, because not only the block as a whole can move but also the operation boundaries within the block, so even relative addresses were not good enough. And as I’m not happy when something only works 50, 80 or even 99% of the time this meant going back to the drawing board.

Final fix

The final code is curiously both more complex and less invasive at the same time: before the program is re-compiled after the LRESPR the return stack entries are now translated from absolute addresses into Line/Statement numbers (using the previously mentioned statement table). The Line numbers are invariant to the compilation process, so after the compilation they are again translated into absolute addresses which always give the correct result, no matter how the compilation changed the operation stream. So the new code should have zero performance and stability implications for ordinary executions and is only active when you LRESPR things. And neither Wolfgang nor I managed to crash it anymore, yay!

The end

This, quite frankly, was a lot of work for such an obscure bug. Why did I do it? One reason is that I’ve been suffering under the bug for a decade without even realising it: when I was toying around with ProWesS I always loaded it conditionally using a PROCedure, which sometimes crashed and I could never explain why and just assumed ProWesS itself was the culprit… the other reason is that stress levels at my real job are currently very high and these puzzles are actually somewhat relaxing sometimes.

No matter the reason, SMSQ/E 3.31 is out and you can enjoy your crash free BOOTs and whatnot from now on. Have fun!

Toolkit II – The sequel

How it began

As any QL owner will know, the Toolkit 2 from QJUMP/Tony Tebby was THE toolkit without which a QL was almost unusable, arguably its contents should have been in the ROM from the start. And I always thought it was quite a shame that such an important toolkit hasn’t been updated in two decades. This got me thinking that most TK2 source files at one time were incorporated into SMSQ/E and that with a little bit of work it should be possible to re-create something resembling TK2 from them again. Turns out I was right, but also completely wrong in how “little” that work is. But still, too many hours later, I presented a brand new Toolkit II release I arbitrarily labeled version 2.30.

This is now based on the latest SMSQ/E source code, with new SMSQ/E features left intact if it was feasible (e.g. the “LOAD” command now automatically tries to append a “_bas” extension to the filename) plus many commands that have been added in the last 20 years (EXF, EX_M, FET, FEW, FEX, FEX_M, HGET, HPUT, JOBID, LGET, LPUT, UPUT, WGET and WPUT).

Unfortunately there are also parts of the source code that were completely missing, like the extended MDV driver. In this case I completely reverse engineered it from an existing ROM binary in a way that you can’t tell anymore that this isn’t the original source code.

The old network server code is actually still supplied with the SMSQ/E source code even though it’s not actually used there. The problem with that is that it’s just too big to fit into the ROM anymore and it probably doesn’t make much sense to run it from RAM because of the different timing. Therefore I didn’t include it for the initial release. The ALARM and CLOCK commands had a similar fate.

The ALTKEY code, too, is included in SMSQ/E without seeing any usage. I didn’t want to include it at first, because with the Hotkey System II it’s very much obsolete. But when tinkering with my QL system the HK2 is often not loaded yet and it drives me crazy when ALT+ENTER doesn’t work to recall the last line. So it went back in. Problem then was that the result was about 200 or 300 bytes too big, so I removed the ALTKEY code but left the ALT+ENTER code in.

Let there be networks

The basic problem is that TK2 already didn’t quite fit into 16KB during its last official versions and it became a lot worse once I packed in all the new features available in SMSQ/E. To make it fit again I had to remove something and I chose the network driver code, last having used it myself 20 years ago to prank my dad. Some people weren’t happy with that, which was to be expected, but there is one point which did stick: the network code cannot even be loaded afterwards as the timing needs to be precise and this is (almost) only possible when the code runs from ROM.

So when I found a way to free a few more bytes in the ROM and I thought about what I could do with them and eventually came up with the idea of splitting the network code: I re-added just the timing critical hardware access routines to the ROM so that the network driver can be loaded later and still work. I think this is a fairly good compromise that would make most people happy.

Martyn Hill kindly beta tested the version for me and later inquired if one could have a full network stack in the ROM and remove some other stuff instead. I was hesitant because for me TK2 is foremost a SuperBasic toolkit and most of the remaining stuff is the basic commands. There is only one part that is even a bit bigger than the network code: the “ED” SuperBasic editor. It includes many improvement over the original TK2 code and thus grew considerably in size. Removing it also has the advantage that you wouldn’t put it into actual SuperBasic code, so the toolkit stays 100% compatible. For me personally “ED” is about a million times more useful than the network code but if you absolutely have to bootstrap a QL without any file systems over the network this is the way to go. Also I do provide a standalone binary for loading later.

Choose your poison (downloads)

So here are all the different flavours:

TK2 v2.36 ROM version (with separate NET_bin file for full network support)
TK2 v2.36 network ROM version (with separate ED_bin file for the SuperBasic editor)
TK2 v2.36 RESPR version (this version includes everything though the network code will not work unless burned into a ROM or executed from zero waitstate RAM)
TK2 v2.36 QL-SD version (RESPR version that works together with the low level QLNET routines included in the QL-SD ROMs >=1.09)

Version history

V2.36
- Fixed ED #0 editing (don't write errors then). Also fixes an address error

V2.35
- Don't add default directory to names starting with a drive

v2.34
- Fixed a crash on LOAD/LRUN/MERGE

v2.33 (Per Witte)
- Added SUSJB procedure to suspend a job for a time or indefinitely
- Added FDEL as a function version of DELETE

v2.32
- Added network hardware access code to ROM so LRESPRed server can work
  Also an alternative ROM with full network stack and missing ED command

v2.31
- Fixed a bug in CDEC$

v2.30
- First reborn version

QMake released for free

This has been a long time in the making, actually I wanted to do this many years ago, but the day is finally here: QMake is here for everybody to enjoy. QMake is a “make” tool which basically takes a linker file as input, collects all files that make up the executable and assembles any where the source is newer than the relocatable file. You can also add dependencies like “reassemble win1_fu_asm if win1_keys_bar changes”. I’ve used this tool for well over 20 years to build SMSQ/E and all my other projects and consider it to be pretty much essential.

I’ve created a new page for it, check it out here.

German Minerva-ROM

Since changing from MGG to the Minerva ROM a few decades ago I’ve never used an unmodified Minerva ROM, I always used one where the German keyboard tables were patched into the binary somehow (by Jochen Hassler I think). This changed when I installed the QL-SD interface in my QL as it comes with a vanilla ROM where the German keyboard support had to be loaded every time, which I found fairly annoying. But now that the Minerva sources are publicly available changing this is fairly easy: take the Minerva source code plus the source of the German language pack, stir a bit and voila, a new ROM. The only thing missing is the printer translation table as that pushes the ROM beyond the 48KB limit.
As probably not everybody can do such a thing I provide the resulting binaries here, in case somebody finds it useful:

Minerva

As for an EEPROM programmer I ordered one at my currently favourite Chinese store Ali-Express (being so cheap that I spent a truck-load of money there this year…). In this case I got myself the somewhat more expensive TL866A for just 60€, which also includes a lot of adapters:

TL866A universal chip programmer

But the smaller brother TL866 can be had for less than 35€ including shipping. This is an amazing piece of hardware for the price (usually it’s also available on eBay for slightly more) that finally lets me retire Jochen Hassler’s excellent but nowadays slightly outdated QL Eprommer II board:

Jochen Hassler’s Eprommer II board

Of mice and men (but mostly mice)

Mice for the QL always have been kind of a problem. In the age that predated USB by two decades there was simply not the one mouse standard to rule them all. Every system did pretty much its own thing and getting a mouse that actually worked wasn’t that easy to begin with. In fact I have never even owned a mouse for the QL that worked right out of the box, every mouse had its electronics ripped out and replaced by something else to work with my trusty SuperQBoard clone. This posed a problem for me as I have recently resurrected my trusty QL but apparently the mice were thrown away at some point or other. Jochen Hassler kindly gifted me one of his old mice, but it was brown with age, the ergonomics is questionable and I really didn’t remember what a pain the ball-based mice were to use compared to their optical brethren.

This lead to the idea of implementing a converter, like taking a cheap Arduino Pro Mini to interface a modern USB mouse to the old SuperQBoard port. Fortunately before starting such a project I had a look at what’s already there and found the “New MKIV Amiga/Atari USB mouse adapter” for about 16€ plus postage. Atari mice are generally the type used for the QL, too, so I just bought it. The first tests a few days later weren’t that encouraging, nothing worked at all. So I had a look at the schematics of the SuperQBoard and found out that although electronically it’s Atari/QIMI compatible, it used a completely different pin layout!

Long story short, this is the pin mapping I came up with

SuperQBoard QIMI/Atari Signal
1 9 Right button
2 6 Left button
3 1 XB
4 3 YA
5 7 +5V
6 4 YB
7 2 XA
8 5 n/c or middle button
9 8 GND

After building a suitable adapter cable the optical mouse began to light up, indicating the power is there, and the buttons worked, too, indicating a working mouse protocol on the USB side (or rather PS/2 side really, the adapter only works with mice that can speak the PS/2 protocol over the USB connector) but the pointer only moved very erratically. So after some hair pulling (as if I still had any) I went back to the schematics and saw that apparently unlike QIMI my SuperQBoard clone only had pull-up resistors on the mouse button lines and not on the lines responsible for X/Y movement! So I decided to integrate them quick&dirty style into the adapter cable:

Quick&dirty integration of pull-up resistors…

… but in the end, who can tell

And what can I say, it worked so smoothly it was a pure joy to use.

So, lessons learned:

  • The adapter might be used together with a QIMI interface right out of the box
  • SuperQBoards used a pinout different from QIMI
  • SuperQBoards might be lacking the necessary pull-up resistors
  • Hardware tinkering is still fun

New QPC2, DISA and more

Every year I take the birthday of my daughter as an incentive to get stuff out of the door.

So this year I present to you:

QPC2 v4.05

This is mostly a bugfix release. It includes the latest SMSQ/E 3.28. One option was added that instead of a fixed resolution it can use the maximum resolution of the current monitor. Select “Max” as a resolution to do this.

Get it from the Downloads page.

… but there is more

The disclaimer for the following software: you may not sell the software and they come with no support whatsoever.

DISA

My old friend Jochen Hassler was over for dinner a few days ago and I asked him if I may release his software. He said yes, so here I present to you DISA, a pretty revolutionary disassembler at its time. V3.04 was the last official release done by Jochen in 1999. I adapted the source code for high colour in 2003, but this release 3.05 has never left my hard drive until today. So here they are:

DISA 3.04
DISA 3.05
DISA examples

As I got most of my software directly from the authors I usually lack the original manuals, as it is in this case. If somebody could provide that to me I can add it to the distribution.

Update: Thanks to Albin Hessler (who provided the Word document for v2) and David Westbury (who provided a scan of v3) I can now provide the latest manual here: DISA3E.pdf

ATR device

I didn’t have the time to sort through the source and binaries I have of the ATR device, but I know for example that Dilwyn has a few ROMs or binaries and those can now be used freely, too.

QMenu

And finally I had a chat with another old friend of mine, Jochen Merz, and he agreed to officially release QMenu for everybody to use. So here is my latest menu_rext file:

Menu_rext 7.66

Update: apparently I’m not up to date in regards to QMenu, but 7.66 is the latest version I had the source code to. I will do a follow up when I know more about the later versions.

Was this a great birthday or what? 🙂

Have fun, Marcel

HomeMatic: Gehäuse/Verstärker für HM-OU-CM-PCB

English: HomeMatic home automation articles will be in German as this system is mainly used there.

Den Anfang meiner (zumindest geplanten) HomeMatic Serie macht ein bereits älterer Aufschrieb über den Bau eines Gehäuses für den HM-OU-CM-PCB. Als Ausgangs-Basis dient ein bei Pollin erhältliche “Tragbarer Aktiv-Lautsprecher PULSE”. Anders als bei Pollin üblich ist das Ding auch ein Jahr nach meinem Bastel-Projekt noch erhältlich, mittlerweile sogar schon für wahnsinnige 3,95€!

Der eingebaute Verstärker liefert mit 3 Watt viel Bumms, läuft allerdings mit 5V während der HM-OU-CM-PCB mindestens 8V fordert. Der “richtige” Weg wäre es wohl 8V außen anzuschließen und dann intern auf 5V zu senken, wie es der HM ohnehin macht. Bedeutet aber größere Arbeiten an der Verstärker-Platine wenn man die vorhanden Buchsen des Verstärkers weiter benutzen will und Wegfall der USB Versorgung. Ich hab mich dafür entschieden den Step-Down Wandler auf dem HM zu umgehen und die Platine direkt mit 5V an Pin MP11 zu speisen. Auf eigene Gefahr, man verliert damit den Verpolschutz! Dies stellt dann auch erhöhte Anforderungen an das verwendete Netzteil, die 5V müssen sehr sauber reinkommen sonst gibt’s jede Menge Stör-Geräusche! Ein altes iPhone Netzteil bzw mittlerweile sogar ein billiger 5€ ALDI Zwischenstecker mit USB-Ladeausgängen verrichten hier bei mir aber sehr gute Dienste.

Den internen Verstärker des HM sollte man wie in der Anleitung beschrieben abklemmen und den Ausgang dann mit der Klinken-Buchse am Verstärker verlöten. Dann noch Masse und die 5V rüber und fertig ist die Kiste. Für den physischen Halt habe ich den Batterie-Halter etwas aufgedremelt so dass man die Platine hineinschieben kann. Hält ganz gut und man kommt nach dem Öffnen des Deckels bequem an die SD-Karte.

Den Rest erklären hoffentlich die Bilder.