Reverse Engineering the HRD Remote Serial Protocol
March 3, 2011 1 Comment
Ham Radio Deluxe (HRD) is a free program that servers as a user interface to radio transceivers. Modern transceivers can be controlled either from their front panel controls (if they have a front panel), or from a computer through a serial or USB connection. HRD allows you to connect to a radio, or even to a few simultaneously, and to control it through on-screen controls. Its user interface is absolutely fantastic. It comes bundled with a bunch of other radio-related programs, but in this post I am only interested in HRD itself.
One cool and useful feature of HRD is that it can be used to control a remote radio. You run an HRD server program on the computer to which the radio is connected. You run HRD itself (the user-interface program) on another computer, and tell it that the radio is connected to the first computer. HRD establishes a network connection with the remote computer and controls it.
I want to use this feature, but I want to connect the radio to a computer running Linux, not Windows (I have good experience with remote radios connected to a Linux box, whereas I don’t like to run Windows remotely). Alas, HRD and it’s remote server only work on Windows. Searching for Linux server programs that would work with HRD revealed that none exist, and that Simon Brown, the developer of HRD, does not want to release a specification of the remote protocol or code that would explain the protocol. (I didn’t try to figure out the reason for this, but I assume he has a good reason not to release the protocol/code, and I respect that; I am grateful for the program itself and for the public specification of another important protocol related to HRD, the one that allows other programs to talk to HRD.)
Serial communication is not terribly complicated, so I thought that I might be able to reverse engineer the protocol and to write the Linux server code myself. I was right. I now have Linux code that HRD thinks is its remote server. It’s not finished yet, and I didn’t connect the radio yet to the Linux box, but this should be just a matter of time before it all works together. The rest of the post explains how I figured out the protocol. I’ll share the code itself once it’s actually working with a radio (if you’d like to help develop and test the code, please drop me a line).
I initially assumed that the protocol would be pretty simple, and decided to figure it out using a Java program that I could configure either as a server, playing the role of a remote computer connected to a radio, or as a client, playing the role of the HRD GUI program. I would run the program as a server and try to connect to it from HRD. My code would dump the data it received, and this would tell me how HRD starts the exchange. I would then run it as a client, connecting to HRD’s own server, and send the data I received previously. The real server would respond, and now I would know what the first response looks like. I did this, repeating the process for 3 different request-reply exchanges. The messages were pretty simple, and I was able to determine where the fields were within the message and what they meant. For example, I discovered that messages start with a 4-byte length field, followed by a 4-byte fixed signature (HRD*), and so on. I did make a few mistakes (initially I thought that the length field was fixed, for example), and there might be fields that were always zero in my messages so I could not detect or understand them, but I discovered enough to make HRD happy with the connection process. The 4th message was much more complicated, and I realized that I would not be able to figure out how to parse it using this technique. I needed a more global view.
I obtained the global view using two free utilities, wireshark and portmon. Wireshark allows you to capture network packets. I used it to trace an entire connection between HRD running on one computer and its serial server running on another (Windows) computer connected to a radio. This made me realize that following the first 3 messages that establish the connection, there are a number of messages, all 520-bytes long, that probably configure the serial port. They were followed by serial read-write messages, and by a disconnection sequence. The 520-byte messages were all complicated, like the one I could not understand. In the picture on the right you can see a screen shot of wireshark displaying this trace.
I then switched to portmon, a utility that allows you to capture serial-port activity. In the screen shot on the right, you can see a trace of HRD connecting to a radio over a serial port. The important steps are at the top, where HRD configures the port; these steps correspond to the network messages I could not understand.
I was still stumped for a while. The messages just looked too complicated, with random looking binary numbers all over.
At some point, I realized that the fact that HRD configured the remote serial port using so many messages probably meant that it was basically just sending to the server exactly the same operating-system calls that it was making to configure a local port. In principle, HRD could have sent all the information required to configure the port in one message. The fact that it did not suggested that the protocol was designed to keep the server simple and to avoid duplication of code. I then studied the different system calls that Windows programs use to configure serial ports. This allowed me to reverse engineer enough of the protocol to make my server look like the original server to HRD. There are certainly details that I have not figured out yet, but I’m pretty sure I’ll be able to get my radio to run remotely when connected to a Linux box running my server.
The final screen shot is of HRD displaying the welcome message from my server. The window at the back shows diagnostic messages that my server prints. That’s it for now. The current state of things are that the server code has been ported from Java to C (Java is not good for controlling serial ports), and that I still need to test it with the radio. Hopefully I’ll do that over the weekend. Stay tuned.