Author Archives: Marcel

Wrapping stdcall functions in Node FFI

A colleague of mine needed help wrapping some C DLLs for NodeJS. He’s using the Node FFI API for the wrapping, which unfortunately is a little light on documentation. The problem was that by default Node FFI wraps cdecl functions and there is no mention of stdcall anywhere. In fact the consensus on the net seems to be that it’s not possible at all. Reading the sources proved more fruitful and so I document the way here for future generations to find:

The standard way of initializing a FFI wrapper using the array in the constructor

actually takes an additional parameter with the calling convention, but in stdcall the function names are often decorated with an additional “@x” which denotes the bytes the function parameters need on the stack. Problem is, blah@x is not a valid identifier and in the array the Javascript function pointer always has the same name as the function in the DLL. The solution is to manually wrap the functions, which I actually prefer anyway as the function pointer doesn’t end in a struct but can be used directly afterwards:

The resulting wrapper is called using “AdsPortOpen()” and it accesses the function with the decorated name “_AdsPortOpen@0”. The easiest way to get the decorated name is using the dumpbin utility from Visual Studio with the “/exports” parameter:

C:\Windows\System32>dumpbin /exports gdi32.dll
Microsoft (R) COFF/PE Dumper Version 10.00.40219.01
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file gdi32.dll

File Type: DLL

  Section contains the following exports for GDI32.dll

    00000000 characteristics
    573C8294 time date stamp Wed May 18 16:56:20 2016
        0.00 version
        1000 ordinal base
         727 number of functions
         715 number of names

    ordinal hint RVA      name

       1012    0 00043F40 AbortDoc = _AbortDoc@4
       1013    1 00045806 AbortPath = _AbortPath@4
       1014    2 0003BA5C AddFontMemResourceEx = _AddFontMemResourceEx@16
[...]

HTTPS everywhere

Out of the blue my hosting provider finally offers one SSL certificate for free to use with my site. I’ve been using StartSSL certificates for my home servers for a long time now, but on the shared hosting servers I couldn’t just install my certificates and I felt the prices for regular ones are a pure rip-off.

In any case, kilgus.net finally got HTTPS support and I have adapted the site accordingly. I had to change all download links for this to work, so if I have missed a broken link or in case of any other problems, give me a shout.

QPC2 for Mac OS X

Sometimes things happen out of the blue. I haven’t heard from my fellow QL friend Peta Jaeger for years and yesterday he mailed me a complete ready-to-run distribution of QPC2 for MacOS X. Under the hood it employs the WINE Windows emulation layer, so QPC2 is of course still not a native Mac application, but it almost feels like one. I only have a headless Mac at home (meaning without a monitor or keyboard) and I’m currently 5km away from that, but even through the remote VNC connection it felt absolutely usable, which I find pretty amazing. Peta also included the Black Phoenix QL software distribution so you can start clicking through various applications right away. It is installed like pretty much any other Mac application, but if you have questions feel free to contact Peta about it.

Interested? Check it out on the Downloads page.

QPC2 running on Mac OS X

QPC2 running on Mac OS X

Analyzing the HomeMatic CBC authentication algorithm

This is my first post about my new hobby, smart homes in general and specifically HomeMatic, and it’s already quite a technical  one: solving the AES authentication problem with the new HM-SEC-SD-2 smoke detectors. The detectors use cryptographic authentication between them so that not every grumpy neighbour can activate the alarm. This is good, but also a problem when you want to activate the alarm yourself, as part of a burglary system for example.

The main tip came from Michael Gernoth, who mentioned that the CCU2 software used to contain a function called performCBCAuthentification, which sounded like it could fit the given scenario. Fortunately reverse engineering is something I’m usually pretty good at, so I set out to have a look. Reverse engineering binary code is somewhat akin to assembling ground coffee back into beans. Challenging but also satisfying when it works. The tool of choice is a somewhat expensive tool called IDA which my employer fortunately bought for me to do my job. Unfortunately not the very expensive version that can actually output C-like code, just the disassembler one, but we can work with that 😉

From ground coffee…

On the high level overview the function in question looks like this:

High level IDA function view

Which is actually not too bad. Somewhat regular and not too much code overall. Zooming into one part looks like this:

Inside the function

This may look wild to the untrained eye but is actually great news: one one hand, it’s ARM assembler, which I have never read before and also, it’s C++, which is a pain to read in assembler, but on the other hand we see that IDA found many symbols and that most function calls actually have names. This is like winning the lottery! One function call in particular shows what it means to have C++ code at hand:

BL _ZNSt8_Rb_treeIiSt4pairIKiSsESt10_Select1stIS2_ESt4lessIiESaIS2_EE17_M_insert_unique_ESt23_Rb_tree_const_iteratorIS2_ERKS2_ ; std::_Rb_tree<int,std::pair<int const,std::string>,std::_Select1st<std::pair<int const,std::string>>,std::less<int>,std::allocator<std::pair<int const,std::string>>>::_M_insert_unique_(std::_Rb_tree_const_iterator<std::pair<int const,std::string>>,std::pair<int const,std::string> const&)

Everything clear now? Thought so. Unfortunately our newly found fortune doesn’t cover variables and object properties, so parts like the following look a lot easier visually but without any clue what the data is, is a lot harder to analyze:

More ground coffee

It looks like it’s traversing a doubly linked list in search of the encryption key, but I haven’t analyzed this part of the code 100% and probably don’t need to.

… to the beans…

In the first step I tediously transcribed the code I’ve seen back into something somewhat resembling source code. By hand…

This is not supposed to compile and there might be transcription errors or bugs in it. It’s just to get your notes in order and is certainly a lot more readable than the code we’ve started with, so we can finally deduct the algorithm from it!

… to the algorithm …

Given the example frame of:

00 01 02 030405 060708 09 0A 0B 0C 0D 0E 0F101112
01 14 41 44E347 44E347 01 01 96 00 00 03 0A802A78
 | Flg | Sender Recvr   |  |  |    IV1 |  
Cnt  Type             Chn No State    IV2
  1. Build an initialization vector containing:
    1. One byte with the value 49
    2. The sender address (3 bytes)
    3. The receiver address (3 bytes)
    4. The byte marked “IV1”
    5. The byte marked “IV2”
    6. The Cnt byte
    7. 5 bytes of zero
    8. One byte with the value 5

    In our example frame this adds up to “4944e34744e347000301000000000005”.

  2. Encrypt this using the AES key, which results in “d064bd7e0161592f1175f1c9e822f87b”.
  3. Build a second frame using:
    1. The Cnt byte
    2. The Flg byte
    3. The Chn byte
    4. All bytes from field “No” up to FrameSize – 6 (just leaving out the signature and the IV bytes)
    5. Fill the rest up with zero so that we have 16 bytes again

    This gives us “01140101960000000000000000000000”.

  4. XOR the result of the AES encryption with our second frame. This gives us “d170bc7f9761592f1175f1c9e822f87b”
  5. AES encode the result again. This gives us “d051f2c911e8cb716057a89d0a802a78
  6. Lo and behold, the last 4 bytes match the signature in our example frame! Perfect

… to the end

The CCU2 code only contains the routine to check a signature, not to generate one. So at this stage it’s unknown to me why the two IV bytes have the value they have, as I don’t own the hardware yet nor was I given any more example frames to work with. Maybe they can be chosen freely, maybe they are counted up. In any case, the remaining work should be fairly easy to do in comparison. Hope I could help!

New QPC v4.04 plus SMSQ/E v3.26! With stipples!

A little over two years ago QPC2 v4 was made available for free to celebrate the birth of my little girl Marla. I wanted to release 4.03 for her first birthday but missed the date and then forgot about it (the changes weren’t that important anyway). So I planned to release what I have for her second birthday and missed that, too… then I noticed the 18th birthday of my web presence came up last week, so I was going to release it then for sure! Spoiler-alert: I didn’t make it. But this time for a good reason at least: Wolfgang wanted to include a few last-minute changes to SMSQ/E that would allow it to draw alpha blended blocks. I liked the idea in general, but just supporting blocks seemed a bit halve-baked to me, so I foolishly offered to extend the alpha blending support to the whole graphics sub-system!

Okay, this is a job that is, in principle, actually not that difficult, but in practice and when you want to achieve at least some level of performance it can be a bit of a pain to develop. Especially if you’re crazy enough to also aim for supporting the old-timey stipples, and I’m no man for halve-finished jobs, so stipples there will be! So, after the girls go to bed I usually have one or two hours to myself and I made some good use of them. Today I can finally show you the result:

Alpha blending (for all plattforms!)

The alpha blending support is, like INK, PAPER or OVER, a part of the settings of a window. The setting can be changed using the aptly named ALPHA_BLEND SBASIC command. It takes two parameters, a channel and an alpha weight from 0 to 255 with 0 being transparent and 255 being opaque. So, after executing for example ALPHA_BLEND #1,128 all future graphics commands on channel 1 including BLOCK, CIRCLE, LINE and PRINT will draw their contents halve-transparent over the existing background until alpha blending is disabled again (by setting the weight to the default of 255: ALPHA_BLEND #1,255). Here’s a little example:

alpha-blending

The trap to control the mode from other languages is defined as thus:

|                                                                             |
|  Trap #3    D0=$62                                               IOW.SALP   |
|                                                                             |
|       Set the alpha blending weight for a window                            |
|                                                                             |
|  Call parameters                      Return parameters                     |
|                                                                             |
|  D1.B alpha weight (0..255)           D1   preserved                        |
|  D2                                   D2   preserved                        |
|  D3.W timeout                         D3   preserved                        |
|                                       D4+  all preserved                    |
|                                                                             |
|  A0   channel ID                      A0   preserved                        |
|  A1                                   A1   preserved                        |
|  A2                                   A2   preserved                        |
|  A3                                   A3   preserved                        |
|                                       A4+  all preserved                    |
|                                                                             |
|  Error returns:                                                             |
|        ICHN  channel not open                                               |

I don’t actually have any use for this feature myself, I just wrote it because, well, I could. I certainly hope somebody else finds it useful, though 🙂 And after all these years of developing SMSQ/E this is actually the first trap that I have defined myself (all other functions I introduced were vectored routines)!

DOS device rewrite (QPC only, obviously)

I’ve written many hundred thousand lines of assembler in my life and 20 years ago when hacking on an 8Mhz 68008 or even a 66 Mhz 486 this still made a lot of sense. But the times have changed a lot, not only are CPUs so fast that speed doesn’t really matter that much anymore, compiler became so good that you will have some real trouble beating them with your average assembler code! Also, CPUs have been optimized to execute compiled code and if you do some clever trick in assembler that a compiler wouldn’t do you could actually end up with much slower code on a new CPU generation (this has happened with QPC before).

To cut a long story short, every time I had to touch a sub-system of QPC in the last few years I rewrote it in C first in order to not go insane for writing any more lines of x86 assembler. This time it was the DOS device that got the C treatment. Functionality wise not much has changed, except one thing:

RENAME SUPPORT!

It always bothered me that the DOS device could not rename files and if I had known 15 years ago what I know today I could have implemented it back then, too, but alas I didn’t. The feature as it is now implemented needs Windows Vista+ to work, so XP and lower still cannot rename files (boo hooo!). But if you’re still using XP I strongly urge you to update anyway.

And the rest

The rest is mostly minor bug-fixes, you can check out the details in the version history. Perhaps one interesting change is Respect “Keep aspect ratio” when going into fullscreen mode. The story behind this is that QPC2 v4 does not alter the physical screen resolution to go full screen anymore, it just stretches the content of the QL screen to the extent of the monitor. This is a bit of a problem when the screen resolution is for example 1366×768, because in order to stay compatible to the QL colour mode QPC enforces the screen x-resolution to be divisible by 8, and 1366 is not. So QPC will round down and try to stretch a 1362×768 QL screen to a 1366×768 monitor, which can only be done by doubling a few pixel columns. If you are in this situation then enabling the “Keep aspect ratio” option will now make sure that the screen will not be stretched, instead there will be four black columns on the right hand side of the screen instead, as it was with QPC2 v3.

So this is it, the first new release in two years. I hope you’ll like it, enjoy!

Happy 18th birthday…

… to this very website. According to a copy I obtained from Archive.org my website had its 18th birthday yesterday (back then it was under a different URL which I won’t disclose here because of embarrassment and stuff). I wanted to celebrate this by releasing a new version of QPC, but unfortunately one last minute SMSQ/E feature wasn’t finished in time.

Still, 18 years is a frighteningly long time and reaches back into some pretty early stages of the web. It was quite common to have a “visitor counter” and I actually had one until the last redesign. The last value it displayed was 153200, but I’m not even sure it worked properly the last few years. In the heyday of QPC there was quite a lot of traffic because of it. Later most people found the site because they were searching for some Wifi hardware that I analyzed in the “hacking” section I used to have. Nowadays it’s mostly QPCPrint that is still interesting. And occasionally people apparently search for my name, for whatever reason. My name used to be globally unique for a long time but thanks to Facebook I’m now aware of at least one more person sharing it, so perhaps they want to find him instead 😉

Anyway, that’s it for now. Next time, new QPC. Probably.

EasyPtr and me

In 1991 Albin Hessler released the EasyPtr package to the world. It was quite revolutionary for its time, combining a powerful SuperBasic extension with a true WYSIWYG dialog editor. I was a young lad back then, about 12 years of age, and luck would have it that Albin was friends with Jochen Hassler, who happened to be working with my dad. If it wasn’t for Jochen I probably would never have owned or used a QL, by the way, but I digress. Due to my connection I was one of the first people getting their hands on EasyPtr and I actually still remember the excitement I felt when reading through the manual for the first time. Yeah, I was probably never what you would consider a “normal” child.

I reported back a few bugs I found through Jochen and after a few days I got a package in the mail with a copy of a QL game called “Brain Smashers” as a thank you. I was overfilled with joy and it actually still pains me a bit that I have never really thanked him for it because, believe it or not (and most people knowing me today will probably opt for “not”) I was just too shy back then! But I actually still have it after all these years, including the ring binder manual.

Albin was also head of the local QL chapter, often the meetings were held in his home, and in time I considered him a friend. Along with Jochen we traveled to a lot of QL shows in a road-trip kind of way all over Europe for years to come, which was really a lot of fun. And when Albin and Jochen left the QL scene I continued this tradition with Jochen Merz for even more years, including many trips to the USA that I remember quite fondly.

After I had written what is now called WMAN2, the window manager that could handle the new high colour drivers, voices grew louder for an update to EasyPtr to support the colours, too. So, after much prodding by some very persuasive people, mainly Per Witte I guess, I finally gave in and updated EasyPtr around 2004. This was a huge task because while later products from Albin Hessler like CueShell were very well written, EasyMenu’s code was, let’s say, a bit harder to read and maintain. Due to the amount of work this update was released commercially. It sold very well at the official release meeting in England, but after that I never saw another cent coming from the island… I can only speculate if no more copies were sold or if the license fees just never reached me. So apart from the handful copies sold through Jochen Merz this was not a successful venture, but as the package was also such a huge part of my childhood I still don’t regret doing it.

Finally last month I got an inquiry about how to obtain a copy which triggered me to do the next logical step in EasyPtr’s live: releasing it for free. So here you have it, in all its glory:

Files: EasyPtr4.zip
Manual: EasyPtr4.pdf

A bit late perhaps, but better late than never, right? So enjoy, tell me if you like it or hit the tip jar if you want, it’s completely up to you. Have fun with it!

SMSQ/E boot sequence

I promised last time that I will only write a new post if I have something interesting to say. Today’s topic is interesting to maybe three people in the world, but I’m sure they will be thrilled. Today I was asked a question about the SMSQ/E boot sequence and it’s been so long that even I had to research again how it works, so I might as well write it up now. In the following text I’ll try to explain the early boot sequence of both QPC and Q40.

Attention: The whole text is based on me looking at the sources for roughly two hours. I took care to be accurate, but I’ve been wrong before in my life. Twice or so.

SMSQ/E is a bunch of modules

An SMSQ/E file or ROM consists of separate modules, one concatenated after the other (hardware initialisation, screen driver, disc driver, etc.). Every module has a header except the first one, the host module. As most SMSQ/E files can be executed directly from another host OS, the host module must start with code. It has a trailer instead, which is at the end of the whole SMSQ/E file (so one could say the host module includes all other modules).

ROM locations

On the Q40, the ROM is seen in two places: $00000000-$00018000 (only first 96kb) and $FE000000. The lower region has one special feature: when the ROM is visible, all write accesses go to a shadow RAM area (default mode). The hardware can switch the mode (by writing to address $FF018000) so that a read will be from the shadow RAM and any write will be ignored. The ROM is hidden in this case. The mode can be switched back by writing to address $FF010000.

The CPU will get its initial instruction pointer from address $00000004, i.e. in the lower ROM area. In case of SMSQ/E this will point to $FE000062, so execution continues in the upper ROM copy. The lower copy of the ROM is not used again after that.

On QPC the SMSQE.BIN file is simply loaded to $00030000 and the virtual CPU starts executing instructions from there.

1st module: Host module

Q40: sys_boot_q40_rom_asm

  1. Offset 0: File was loaded through LRESPR from another OS
    1. Disable interrupts etc.
    2. Copy the whole SMSQ/E binary to $28480 so that it is at a known and safe place
    3. Disable caches
    4. Jump to 2nd module
  2. On ROM entry point:
    1. Disable caches
    2. Setup dummy stack
    3. Clear first 256 bytes of legacy screen RAM at $00020000 for whatever reason. Probably for debugging purposes.
    4. Jump to 2nd module

QPC: smsq_qpc_host_asm

  1. Setup dummy stack
  2. Jump to 2nd module

2nd module: SMSQ loader

File: smsq_smsq_loader_asm

  1. Call hardware initialisation in 3rd module. This will return hardware dependent data that will be used in the rest of the code.
  2. Copy the code of sms_wbase to an area below RAMTOP. sms_wbase is a simple function that writes a word located in D0 to the address in A5. It is used to write to the low RAM area, which for Q40 needs the special bank switching mentioned above. In QPC the RAM area is nothing special and the whole routine consists of one instruction: move.w d0,(a5)+
  3. Update the vector sms.wbase with address of the newly copied sms_wbase
  4. Copy code area from label ldm_reset to lpm_resetend (the reset routines) to first slice in the module table (see below). The first slice is always at sms.base = $900.
  5. Copy remaining modules to available slices. If no fitting slice could be found, copy to just below RAMTOP. Copying is done by the routine returned by the hardware initialisation (q40_mdinst/qpc_mdinst).
    This means that after this step the original ROM or RAM copy of the SMSQ/E file is not used anymore!
  6. Jump to 4th module, which is the OS initialisation, but this time to the new RAM copy!

3rd module: Hardware initialisation

Q40: smsq_q40_hwinit_asm
QPC: smsq_qpc_hwinit_asm

  1. Execution starts at label hwinit
  2. Do the actual initialisation (scan RAM size, setup MMU, etc.)
  3. Setup data and pointers for SMSQ loader:
    1. d7 = RAMTOP
    2. a2 = config block
    3. a3 = sms_wbase (the routine mentioned above that copies d0 to (a5))
    4. a4 = module table. This is a table that denotes special regions (“slices”) of memory where modules can be copied to and that would otherwise be unused.
      On Q40 this is only one slice:
      “sms.base-4 to sms.base”. The first slice is special, it will always be used to host the reset routine, no matter the size. This is why it’s only 4 bytes
      On QPC there are 3 usable slices:
      “sms.base-4 to $4000”
      “$4200-4 to $c000”
      “$10000-4 to $17d00”
    5. a5 = loader communication block
    6. a6 = module initialisation function
      On Q40 this is q40_mdinst, on QPC this is qpc_mdinst. Both copy the memory pointed to by (a0) to 4(a1) up to (a2). The Q40 just adds the ROM bank switching to the mix.
  4. Return to SMSQ loader

4th module: OS initialisation

File: smsq_smsq_base_asm

  1. This starts at label smsq_base
  2. Initialise real stack now
  3. Jump to sms_reset
  4. Clear memory
  5. Initialise vector area ($0008-$0400)
  6. Initialise trap vectors
  7. Initialise system variables
  8. Initialise SBasic stuff
  9. Call init_ext, which will call the initialisation code of all remaining SMSQ/E modules
  10. … from here on Job 0 (SBasic) basically starts running and the OS initialisation is finished

The end

The whole boot scheme is pretty clever but also hard to understand at first glance. But it does the job of catering for many different platforms and usage scenarios (boot from ROM, loaded from other host OS etc.) very well. For me it was enjoyable to once again dig a bit deeper into the OS I once have loved and known so well and I hope at least 3 other will join me in my delight 😉

Updated with input from Peter Graf.

A new ho(p|m)e

When I built my last site Netscape Navigator was still the rage and CSS very much in its infancy, so I thought it’s time for a slight update…

To this end I had a look at some content management systems and what people thought about them. WordPress doesn’t have a good reputation in my circles, Joomla seemed to be even worse and Drupal gave some vibes of being very difficult to use but otherwise being a professional alternative. So I went with Drupal, mainly because I was expecting a higher security level from it. This was a mistake. And I went with Drupal 8, because when you start something new, you want to use the latest technology, right? This was an even bigger mistake.

Drupal was a pain, and not because I didn’t understand it or anything but because nothing works out of the box. Nothing. Everything is almost comically difficult to do (adding text to a contact form? Who would ever do something like that?). It sees itself as a toolbox that can only be made into a working site using a plethora of plug-ins and configuration. Drupal 8 was especially difficult because it’s so young that almost none of the documentation on the web has been updated for it and many plug-ins were not available yet or at best still beta.

Anyway, I got everything to work and look the way I wanted… then the first critical security update arrived, only days after I have started using Drupal. So much to “more secure” 🙁 OK, so where is the update button? Spoiler alert: there is none. The official update path is basically “backup and then delete everything, unpack the new version and copy back any files you had changed previously”. I did this but forgot to copy back the plug-in files, leaving me with a broken site that didn’t have the plug-ins while at the same time also refusing to install them again because they noticed that there were already settings for them in the database!? Of course, copying them over at this point didn’t help either because apparently Drupal already remembered that they were gone or something like that… and I thought OwnCloud was bad when it comes to updates! And this was only a minor 0.0.0.x update!

The breaking point was when I tried to have a simple image gallery and noticed that Drupal core offered nothing of the sorts and the plug-in situation for Drupal 8 wasn’t much better. That was the point I noticed that this content management system mainly managed itself and not the content. So I gave up and installed WordPress. It also took a while to get it to look and work exactly as I wanted it to be, but at least the basics are all there or easily available and the one update I have experienced was decidedly painless.

The resulting site is probably not the most beautiful site imaginable, but Dammit Jim, I’m a doctor^Wdeveloper, not a designer. See how I got a Star Trek and a Star Wars joke, combined with knowledge of regular expressions and arcane terminal control codes, into the same post? That’s how I roll, baby. Anyway, I hope there will be the occasional post here with interesting stuff and be assured if I have nothing interesting to say this space will remain empty. So stay tuned. Or don’t, it’s your choice.