WSPR is Hopping

I managed to get WSPR to frequency hop!

It took me a while to understand the structure of the program, which is written in Fortran, Python, and a bit of C. All the signal processing is done in Fortran, and so is the scheduling of the transmit/receive cycles. The GUI, including the manual frequency changes, is in Python. The C code implements some external interfaces, to the audio subsystem, to start threads, and so on.

The changes I made to the Fortran code were minimal. I added two global variables that are accessible from both Fortran and Python. One is a binary flag that is used to tell the Python code when frequency hopping is allowed. It is set by the Fortran code when it finishes transmitting of receiving. It is cleared by the Fortran when it starts transmitting or receiving, and it is also cleared by the Python immediately after hopping, to avoid double hop. The other variable is set and cleared by the Python, and it tells the Fortran whether automatic hopping is active or not. This variable is used by the Fortran code when a new transmit/receive cycle begins, at the beginning of an even whole minute. If the variable is set, the Fortran code choses between receiving and transmitting completely randomly, with probabilities that reflect the user’s requested transmit fraction. If the variable is not set, the decision rule is the one that was in use before hopping (it’s a bit more complicated, to disallow consecutive transmit cycles in most cases).

I made two changes to the Python code. One is a new set up panel for frequency hopping. You can see it in the screen shot above. It has a check box for every band and a slider to set the transmit fraction. In the screen shot, I hop on three bands but transmit on none of them. If one of the check boxes is checked, we are in hopping mode (this affects T/R scheduling, as I explained above). The second change was to inspect periodically the variable that tells us whether it is a good time to hop. If so, we clear, search a band to hop to, and hop. This is basically it.

Joe Taylor, WSPR’s author, added me to the list of developers, which allowed me to upload the new code to the subversion repository. Joe tested it and it seems to work for him too. Hopefully this feature will get released soon. Currently, the hopping feature will show up in the menu only if the user adds a file called hopping.txt to the WSPR’s program directory. Without the file, the program won’t hop and its behavior is essentially unchanged. This should allow people to test it without everybody suffering from bugs if there are any

Martin Ehrenfried pointed out on the WSPR forum that when switching bands, it is worth transmitting a bit, even for a short while, to allow automatic antenna tuners to tune to the new frequency. In fact, this is one of the features of the existing T/R scheduling in WSPR: it transmit in the second cycle after startup, to allow you to tune (I didn’t figure this by myself; Joe explained this to me). The current idea is to add an option to transmit for a few seconds in the period between the end reception or transmission in one cycle and the beginning of the next, if we hopped. Each transmission or reception is 110 seconds long, so we have 10 seconds available until the next cycle begins. I have not started working on this yet, but I plan to. But Joe already added a feature to invoke a user script of program after hopping, which he uses to switch antennas.

Here is the log of my first hopping operation; it shows spots on 7, 10, and 14MHz.

Timestamp	Call	MHz	SNR	Drift	Grid	Pwr	Reporter	RGrid	km	az
2010-11-23 06:26	OK1ITK	7.040077	-21	0	JN79is	5	4X6IZ	KM72jc	2573	132
2010-11-23 06:24	DK8FT	7.040027	-21	-1	JN58pe	5	4X6IZ	KM72jc	2661	124
2010-11-23 06:22	IZ3ATV	7.040140	-27	0	JN55vk	5	4X6IZ	KM72jc	2470	119
2010-11-23 06:22	DF6MK	7.040009	-16	0	JN68ik	5	4X6IZ	KM72jc	2591	126
2010-11-23 06:22	EA1URO	7.040092	-13	0	IN62bh	5	4X6IZ	KM72jc	3901	93
2010-11-23 06:20	G6HUI	14.097090	-26	1	IO81wl	5	4X6IZ	KM72jc	3682	111
2010-11-23 06:20	DL2RMC	14.097042	-21	1	JN68ig	5	4X6IZ	KM72jc	2580	126
2010-11-23 06:18	PA2PF	10.140189	-28	0	JO22jq	5	4X6IZ	KM72jc	3319	122
2010-11-23 06:14	IQ4AX	10.140202	-24	0	JN54ks	1	4X6IZ	KM72jc	2499	116
2010-11-23 06:14	ES1HJ	10.140141	-18	-1	KO29hj	5	4X6IZ	KM72jc	3127	162
2010-11-23 06:10	F6BIA	10.140231	-27	-1	JN18dq	2	4X6IZ	KM72jc	3274	112
2010-11-23 06:00	KC1PO	7.040148	-26	0	FN42ih	5	4X6IZ	KM72jc	8827	56
2010-11-23 06:00	K4MOG	7.040098	-14	0	EM73qw	5	4X6IZ	KM72jc	10316	48
2010-11-23 05:58	F6BIA	10.140231	-25	0	JN18dq	2	4X6IZ	KM72jc	3274	112
2010-11-23 05:56	LA3RK	7.040120	-26	0	JO59hw	5	4X6IZ	KM72jc	3569	139
2010-11-23 05:56	LY2BOS	7.040039	-13	0	KO24or	1	4X6IZ	KM72jc	2627	159
2010-11-23 05:56	IZ3ATV	7.040139	-25	1	JN55vk	5	4X6IZ	KM72jc	2470	119
2010-11-23 05:56	G7BZD	7.040095	-24	0	IO90fq	5	4X6IZ	KM72jc	3612	111

Compiling WSPR on Windows and Linux

The first step in trying to add frequency hopping to WSPR is to be able to compile the program myself. Joe Taylor, the program’s author, wrote a web page on how to build WSPR, but it is not very detailed; the web page helped, but I still struggled quite a bit. WSPR is written in a combination of Fortran (most of it), C, and Python, and it uses several libraries, so it is not a trivial program to build.

The code is stored on a public SVN server, so downloading it was not a problem; I used Eclipse do do that (and to edit the sources later).

I started on Windows. I already had Python and MinGW (the C compiler) installed. I installed G95 (the Fortran compiler) and tried to build the program, using the included makefile. It did not work. There were many messages, and I could not tell which ones were errors and which ones were warnings. I quickly gave up and decided to try my luck on Linux, where it is often easier to build open-source codes.

I started by installing on Ubuntu all the software pieces that were potentially missing, as listed on Goerge Smart’s wiki: sudo apt-get install python2.6-dev python-numpy python-imaging-tk python-pmw libportaudio2 portaudio19-dev libsamplerate0-dev gfortran cl-fftw3. George’s command line also included subversion, but I already had a working SVN client in Eclipse. (Update from Jan 21st, 2011: on another machine I also had to install g++; if you don’t have g++, the configure script works, but the build fails with a strange message from libtool.)

At this point, I typed ./configure, then make, then make install. No errors, so the build was complete. Very easy.

I tried to run the program. WSPR changes the radio’s frequency using a command-line utility called rigctl, which was missing. It is part of a library called hamlib. I tried to install hamlib using apt-get, but there is no ubuntu package for it. I downloaded the sources, built and installed the library. Now there was rigctl, but it was not working; it didn’t find it’s dynamically linked library, libhamlib.so.2. It turns out that you need to a file local.conf containing the line /usr/local/lib to the directory /etc/ld.so.conf.d, and then run ldconfig. This tells Linux where to find hamlib’s dynamically-linked library. Now rigctl works.

It works, but it does not support the Softrock. A little digging into the sources of hamlib revealed that it only compiles the Softrock support if a library called libusb is installed. Istalling libusb-dev with apt-get and then building and installing hamlib again fixed the problem. Now rigctl supports the Softrock.

But it still did not manage to actually change frequencies. It turned out that the Linux kernel did not manage to recognize the Softrock’s USB interface. I was running Linux under VirtualBox, and for some reason Linux was able to see the USB device, but not to communicate with it.

This is when I gave up and decided to try Windows again.

On Windows, the build crashed with a message f2py crashes with a message “ValueError: invalid version number ’2.18.50.20080109′”. Digging into the python source that generated this message revealed that it does not like the final .20080109 in the version of binutils (e.g., that’s what “ld –version” returns). It’s not that it thinks that this version is broken, it just does not like the 4-part version number. At this point I realized that the Python code that builds Fortran and C libraries for use with Python is quite sensitive to the version of MinGW you use. I tried to find an older version of MinGW to try but I didn’t find any.

I then discovered that somebody else who faced the same problem, Giovanni Bajo, created a distribution of MinGW that is known to work with Python. I installed it, and together with version 4.1.2 of G95 and Python25, it almost worked.

Almost, because the linker could not find the library file for the argument -lwinmm. I found out that there is a libwinmm.a in the MinGW lib directory, so I replaced -llibmm with the path to this library file. Now the build finished with a binary.

When I tried to run WSPR, it crashed immediately. I have Visual Studio installed, so the crash caused Windows to offer me to debug the code. The debugger showed that the crash was inside the C run time library, msvcr71.dll. I remembered that this MinGW distribution contains a utility to tell the compiler which run time library to use; I ran it with an argument that tells it to use msvcr71.dll (gccmrt 71), and built WSPR again. This did the trick. Now my build of WSPR works.

A Wideband Receiving Loop

The antenna I’ve been using the most recently is a wideband receiving loop. The wideband nature of the antenna allows me to quickly find out what’s going on on different bands without having to re-tune the antenna.

The loop element is the one I built for the active tuned loop. The amplifier was designed by John Hawes; I built about two years ago in an enclosure designed for wire loops. The wire loops did not work well. But I found that the amplifier worked well with the aluminum tubing loop element, so I built a new enclosure that can be weather-proofed. This setup works very well.

The original enclosure, shown below on the left, was great for experimenting with different loop elements, but the connections where exposed to the elements when the amplifier was left outdoors. I used it not only with loops, but also with a short dipole (old telescopic TV rabbit ears); it worked, but not terribly well. The loop element works much better.

The antenna works well from at least 7MHz to 14Mhz, which is the range of my Softrock radio. In the map below you can see WSPR reception reports from one night, mostly at 7MHz. It also works at 18MHz (the Softrock received some WSPR signals there even though it’s input filter attenuates them significantly). I suppose that the antenna works well below 7MHz, but I can’t test this right now. An article by Chris Trask shows that the resistance of a 1 meter loop element is low up to about 15 MHz, where it starts raising very quickly (more on his article later). So I suspect that the amplifier, which has a balanced 50Ω input, is reasonably well matched to the loop only up to about 15MHz. I’ll test the loop at higher frequencies at some point.

The main problem with wideband preamplifiers, like the one I used, is that they are prone to overloading by large signals at frequencies far from the one you are interested in. I did not experience any obvious overloading problems. But it’s of course possible that if somebody starts transmitting close to me, the amplifier will overload. Chris Trask’s article, which I mentioned above, describes a wideband loop preamplifier that’s a lot more sophisticated than the one I built. It’s dynamic range is probably much higher. But since I did not experience any problems yet, I was not eager to build the more complex circuit. (His article, wideband loop antenna amplifier, is available on a Yahoo group; you need to register to access it.)

My build of John Hawes’ amplifier is almost an exact reproduction, except that I used a ready-made 1:1 output transformer from Coilcraft, a WB1010-PCL with a 780μH inductance, rather than a ferrite torodial transformer. It works well.

WSPR Should Frequency Hop

One feature I would love to see in WSPR is frequency hopping. Like most WSPR users, I connect the radio to the laptop, start the program, and let it run. I check once in a while what it has received (and if I’m transmitting, who has received me), but for the most part is a passive activity. The computer and radio are doing their thing, and I’m doing something else.

One active aspect of WSPRing is deciding which on which band to run it. Running it on a band with good propagation and lots of stations is exciting, because I receive a lot of stations and others receive me. But running it on a band with poor propagation is more useful, because the spots tell us when the band opens. If there is no activity on bands that are closed, we don’t know when they open. Similarly for bands with few stations: less exciting, but the spots are a lot more useful than on a band with lots of activity.

A good way to combine both the excitement of operating WSPR on active bands and the utility of operating on dead bands, as well as to even out the activity across bands, is to add frequency hopping. The computer will automatically tell the radio to switch to a new band at the end of a 2 minute cycle, moving between several bands on which the radio and antenna both work.

Doing this in receive-only mode should be fairly straightforward. I would mark in the user interface not one band but several, and the program would hop between them in a random order (to ensure that every transmitting and receiving station are tuned to the same band every once in a while).

Hopping in transmit mode is more challenging, because the assumption in WSPR is that you monitor a band in receive mode for a while, find out on which frequencies stations are transmitting, and choose a frequency that seems free. The best solution is probably to let the program choose a free frequency after it has been monitoring a band for a while. A probably less controversial solution is for the program to start in “don’t transmit” mode on all band. When it is receiving on a band, the user interface would show the reception reports from that band only (both graphically and in the band map). To start transmitting on a band, you will need to click on the band display to tell the program where to transmit.

Another issue with transmitting is the fact that a station may be able to receive on a band but not transmit (e.g., because the antenna needs manual tuning). So the user interface should allow users to mark separately bands where they want to only receive and bands where they want to enable transmission. If the click to choose a transmit frequency idea is adopted, maybe we don’t need to separate transmit and receive in the user interface, but it would still be useful to have an indication of where the program is only receiving and where it is also transmitting.

I looked at the sources of the program and discovered that WSPR is a pretty large program, so I’m hoping that somebody else with experience in WSPR development will implement this. But if nobody does, I’ll probably try to do it some time in the future.

A Microcontroller AM Transmitter (and a Lesson in EMI)

It turns out that my post on the PLL of the LPC1768 was a bit too harsh. It is true that it can’t generate too many high frequencies when the reference frequency is in the kHz range. But it can generate quite a bit of frequencies with a 1MHz reference. Around 1MHz, it can synthesize frequencies spaces 2kHz apart. Around 10MHz, the frequencies are 20kHz apart, and so on. You get the idea.

When I realized that I can do this, I decided to test this by generating a signal that an AM radio can pick up. I succeeded. But let’s talk about the PLL first.

The idea is simple. We will divide the 12MHz clock signal of the mbed board by 12, to generate a 1MHz reference. Now we multiply it by an integer between 275 and 550 to generate the PLL output frequency. To generate a clock signal that is slow enough to drive the microcontroller (100MHz or less), we need to divide by at least 6. I will assume that we divide by 5 and that we don’t multiply by 500, to keep the math simple. We have a CPU Clock frequency between 55MHz and 100MHz; the possible frequencies are 200kHz apart (the 1MHz spacing of the output of the PLL divided by 5). This clock signal will drive a timer or PWM module (I used PWM1), which will divide it again to obtain the output frequency and drive one of the pins.

I tried this first with a very slow output frequency of 1Hz to make sure the PLL works as I think it should and to test everything. The PWM module drove pin PWM1.2, which is connected to one of the mbed on-board LEDs.

Next, I upped the frequency to 1MHz and tested it on a scope; the signal was a nice 1MHz square wave. By connecting and disconnecting this signal from a pin using an interrupt-service routine driven by a another timer (I used the systick timer), I modulated the 1MHz signal. I used a modulation frequency of 1kHz. Connecting the pin of the mbed to a 30cm piece of wire generated a signal strong enough to be easily picked up by an AM radio at a distance of up to a meter or so. Turn on the mbed and you hear the 1kHz signal on the AM radio, turn it off and you don’t.

There are obviously better ways to modulate the RF signal. One is to vary the width of the pulse, like Michael Kleinigger did with an AVR microcontroller. This works reasonably at low RF frequencies, when the divisor you use to generate the RF signal from the CPU frequency is high (a large divisor gives you many pulse widths). It does not work well at higher frequencies. On the LPC1768 another option is to use the digital-to-analog converter to drive an analog modulator.

Also, the signal emitted by my 30cm wire is not a 1MHz modulated signal; it is a modulated square wave, which also contain large amount of energy in harmonics of 1MHz. To eliminate them, you need a filter; Michael’s article shows a very simple one. For regular use (where legal), a more sophisticated filter is necessary, and a small power amplifier also won’t hurt, at least to isolate the microcontroller from the antenna.

This little experiment is also an interesting lesson in how easy it is to generate electromagnetic interference (EMI). All I did was to connect a 1MHz digital signal to a 30cm piece of wire. This generated enough electromagnetic energy to be easily received by a radio. I did this intentionally, but it’s very easy to do that unintentionally, with long wires on a PCB or in a box. If it’s not shielded (or driven differentially to cancel out the field), it will emit.

A PLL Mystery

In a previous post I built a phase-locked loop (PLL) in which parts of the loop was implemented by a microcontroller. Some microcontrollers (MCUs) have an on-board PLL for generating their CPU clock. In particular, can we generate frequencies that are spaced fairly closely, say 8kHz apart? Many MCUs have PLLs that can generate frequencies that are about 1MHz apart. But how about closer spacing?

When I received an mbed kit with an LPC1768 MCU, I discovered something interesting in the user manual of the ’1768. It has a PLL that generates the CPU clock. The reference clock for the PLL can be either a high-frequency crystal oscillator (12MHz on the mbed), or a 32kHz crystal oscillator. In principle, this might allow the generation of closely-spaced frequencies. You can even divide the 32kHz clock down first, to generate frequencies that are, say, 8192Hz apart.

Reading the manual more carefully revealed that the PLL effectively has two modes. In both modes, the PLL output frequency is

Fout = (2 × M × Fin) / N

The output must be between 275 and 550MHz. It must be then divided to produce a CPU frequency of 100MHz or less. The value N is an integer between 1 and 32. The value M is also an integer. It can assume any value between 6 and 512, plus 65 specific values between 4272 and 24170. When Fin=32kHz, only the 65 large M’s are relevant; smaller M do not allow Fout to reach the minimum value of 275MHz.

Why 65 specific values rather than any 15-bit value? I could not think of a reason. If arbitrary values work, then I would be able to generate closely-spaced CPU frequencies. For example, by using Fin=32kHz and N=1, values of M between 4197 and 8390 generate valid output frequencies, spaced 64kHz apart. Division by 6 to bring the PLL’s output frequencies below 100Mhz leads to CPU frequencies of 46MHz and 92MHz, spaced 11kHz apart. To obtain lower CPU frequencies we can divide by more than 6, say by 12, which will lead to even finer spacing.

Does it work? No. I wrote a program to test this. What I discovered is that the PLL works fine with the 65 values of M that are specified in the user manual, but not with other values. I am really stumped by this. The user manual indeed says that other values won’t work, but I can’t figure out why. The 65 valid values don’t have a common divisor; they seem to be arbitrary. Too bad. If somebody knows what’s the PLL structure that allows only these 65 values to work, let me know; I’m curious.

A few words about the mbed kit. It is an interesting concept, designed to make it as easy as possible to get started with embedded programming. The kit is a printed-circuit board that is designed to plug into a breadboard. The board contains a microcontroller, some associated components (a 32kHz crystal and a 12MHz oscillator, for example), and a custom interface chip. The interface chip has a USB interface and is programmed to appear to the USB host as a storage device. You copy a binary file containing an LPC1768 program to the storage device. The next time you press the reset button, the interface chip downloads the program to the ’1768 though a JTAG interface. This setup allows this to download programs to the microcontroller without installing any driver on your PC; this is pretty cool. The mbed website contains an on-line compiler. You author your program in a web browser. When you are done, you click on a button and the web site compiles the program and the browser downloads the binary program. So you don’t even need to install a compiler on your PC. Again, pretty cool.

Thanks to Pete Cooper who ported some of the mbed support code to gcc, I was also able to compile programs for the mbed on my laptop (using Code Sourcery’s free G++ Lite Edition build of gcc and it’s tools).