Akademisk Radioklubb


Quick analysis of the LM100UKA logs using Python and Pandas

UKA-17 is over, and LM100UKA is no more. As a part of the series, “LA3WUA and LA9SSA discover that Python is actually quite nice” ([1], [2]),  we used this opportunity to experiment with some plotting of the contacts from the logs generated by our logging program, N1MM, using Python, Pandas and pyplot.

Pandas is a data analysis library which can make it more convenient to do data wrangling in Python. It is built on top of numpy, with all the efficiency and ease of manipulation which that entails. In addition, it has convenient data structures, capabilities for metadata, easy slicing, merging and grouping, and a whole clusterfuck of various operations that can be applied efficiently to rows or columns.

N1MM saves its logs as sqlite files by default, which constitute complete, file-based SQL databases. Pandas has convenient facilities for reading various input data, including SQL queries, and we’ll start from there.

Given an N1MM log file, e.g. qsos.s3db, an SQL connection can be established to this file using

import sqlite3

sql_connection = sqlite3.connect('qsos.s3db')

If this SQLite file contains ham radio logs generated by N1MM, it will contain a table called DXLOG. All of its information can then be read into a Pandas dataframe using

import pandas as pd

qso_data = pd.read_sql_query("SELECT * FROM DXLOG", sql_connection)

qso_data will now contain all the QSO information in a large data matrix, with each row containing a contact, and each column corresponding to a field in the SQL database table. We can, for example, extract all unique operators in the log using qso_data["Operator"].unique(), or qso_data.operator.unique(), or even numpy.unique(qso_data["Operator"]). We could extract all calls and timestamps run by LA1BFA using qso_data[qso_data.Operator == "LA1BFA"].[['Call',
, or all contacts run against LA-callsigns using qso_data[qso_data.CountryPrefix == 'LA'].

Using basic functionality, we were able to generate a few plots from this data.

The number of QSOs distributed across the days since we started using the callsign. The general level of activity is more or less equal before and after the start of UKA, with the most activity occurring during CQWWRTTY and a burst of activity at the very beginning.

The number of QSOs as a function of hour during the day (UTC). This distribution follows more or less what is expected from the distribution of people at ARK, with a post-work/post-study day frenzy at around 16.00 NT and after 18.00 NT (UTC time had an offset of -2 hours from Norwegian time during UKA).

The same plot as above, but split on the used band. “Day” and “night” bands (14.0 MHz and 7.0 MHz, respectively) can clearly be seen, along with spurious use of e.g. 10.0 MHz during the afternoon.

The distribution of QSOs among the various operators.

The distribution of operators during the building period and UKA itself. A variety of operators participated during the building period, while the start of UKA itself posed some difficulties in both avoiding  interference and being able to get into Samfundet due to stricter access regimes. Mainly LB5QG and  LB1HH kept going during this last phase.

The distribution of operators during the hours of the day. LB5QG has done a considerable effort in producing QSOs at all hours of the day.

World map with filled countries corresponding to the countries we have run. This also involved reading of a HTML table from the ITU website into a Pandas dataframe in order to convert from callsign prefix to country, made easy by pandas.read_html(..) :-). Here, we could also technically have used the ClubLog API to look up country information from the callsigns, but would probably put a bit of a strain on the servers if we were to do that for all calls in the log. Instead, we hacked together a solution using the official ITU allocation tables.

The script used in generating the above plots can be found at github. This is made to analyze all .s3db-files in a folder, and differs a bit from the code snipped shown in the beginning of this post.

Only our most recent, still-in-use logs are in .s3db-format, however, as we strive to convert to ADIF format or similar during the archival process. We’ll also look into what we can extract from the larger bulk of older logs.

LM100UKA is over

The UKA festival is finally over, and we are back to normal here at the club. It has been a great time running LM100UKA for the 100 year anniversary, and in the two last months we have run a total of 1602 contacts! These are mostly on 20M and 40M. This includes the CQWW RTTY competition, but most contacts are otherwise regular QSOs.

This has been some wild weeks! The festival happens every two years, and it first involves the restoration and refurbishment of one of the locales at the Student Society, as well as building of all the necessary props. The theater group prepares the revue and some small plays, while other groups plan out big events and concerts. The chefs at the restaurant in the Student Society even prepare a specialized menu. We have mostly been able to run radio during local daytime since the Student Society usually has been closed off for us in the evening.

Once the festival starts, everything happens. There are shows, concerts, events, and parties running for 3 weeks straight, not to forget one of the most important parts of the festival: The student revue! Of course, this did not stop us from using the radio every now and then. We did however need to limit our output power when things were going on in other parts of the house.

We will be sending out QSL cards to everyone in about a month’s time. Logbook of the World upload is pending activation of the callsign, we will update this blogpost once the logs have been uploaded.

Parsing a DX-Cluster using python and Club Log

LA1K is always interested in working new DXCCs. To assist with this we wanted to see if we could make a piece of software that could alert us when these opportunities occur.

A DX-cluster is a telnet server where clients can report different callsigns that they have heard or worked. To listen to the DX-cluster we will use Python’s telnetlib library. We parse the cluster output with regular expressions using Python’s re library.

Sample output from DX-cluster at LA3WAA.DDNS.NET:8000

The code snippet below opens a connection to the DX-cluster running on LA3WAA.ddns.net, using telnetlib in Python.

import telnetlib
import re

# Open connection to telnet
tn = telnetlib.Telnet("la3waa.ddns.net",8000)
tn.read_until("login: ")

Once the telnet server replies with login:, a callsign can be sent as a reply:

tn.write("LA1K \n")

The output from the telnet server follows a regular structure, as seen in the image at the start of the post. For our DX-cluster, the structure is “DX de [CALLSIGN]:   [Frequency]  [SPOTTING_CALLSIGN]  [Comment] [Time]”. To parse this, we use some straightforward regular expressions. First is the callsign, which is simplified to any combination of the letters A-Z, numbers 0-9 and the “/ ” operator.  Similarly the frequency contains the numbers 0-9 and the “.” operator:

# Define regular expressions
callsign_pattern = "([a-z|0-9|/]+)"
frequency_pattern = "([0-9|.]+)"
pattern = re.compile("^DX de "+callsign_pattern+":\s+"+frequency_pattern+"\s+"+callsign_pattern+"\s+(.*)\s+(\d{4}Z)", re.IGNORECASE)

Once a new line is found in the DX cluster output, it is checked against the previously compiled pattern. Matches are grouped so that the 0th match contains the whole string, the 1st match contains the first regular expression match (if it exists), 2nd match contains the second regular expression match and so on. We sort these into named variables.

# Parse telnet
while (1):
    # Check new telnet info against regular expression
    telnet_output = tn.read_until("\n")
    match = pattern.match(telnet_output)

    # If there is a match, sort matches into variables
    if match:
        spotter = match.group(1)
        frequency = float(match.group(2))
        spotted = match.group(3)
        comment = match.group(4).strip()
        spot_time = match.group(5)
        band = frequency_to_band(frequency)

Processing DX-cluster information

We are going to further process this information using some features from ClubLog’s Application Programming Interface (API). ClubLog is an online tool to manage your amateur radio logs. To access this API, you need a couple of things:

After you have uploaded your logs, ClubLog provides a very neat DXCC-matrix that shows which entities you have worked across the various bands. By comparing the infomation from a DX-cluster with this matrix, we are able to alert the user of potential DXCC opportunities.

We start by retrieving the DXCC matrix from Clublog’s API. The DXCC matrix is fetched from a URL as a JSON structure. Our solution is to use a cron-job that fetches the JSON file at regular intervals, as the matrix might be updated once we upload more logs.


curl -s "https://secure.clublog.org/json_dxccchart.php?call=$callsign&api=$API_key&email=$email&password=$password&mode=0" > ./dxcc_matrix.json

Save the file and make it executable using

chmod +x

Finally register the file with cron by typing

crontab -e

This will open the crontab file in your favorite text editor. Next we will add a line that runs the script once every hour:

0 * * * * [YOUR_PATH]/get_clublog_dxcc_matrix.sh

Save the file and run the script once manually so that you have some data to work with for the next step.

The DXCC matrix sorts countries by DXCC number.   As an example, the entry for DXCC number 1 (Canada) may contain


This means that you have worked Canada on 10 m, 12 m, 15 m, 20 m, 40 m, 60 m and 80 m. The missing bands are 6 m, 17 m, 30 m and 160 m.  Conveniently, ClubLog also provides an API to get DXCC info for a given call. We retrieve this data for spotted callsigns and check it against the matrix. First we define functions to query a callsign for DXCC info, and to check whether the DXCC is contained in the logs for the specific frequency band.

def query_dxcc_info(callsign, api_key):
    return json.load(urllib2.urlopen("https://secure.clublog.org/dxcc?call=%s&api=%s&full=1" % (callsign,api_key)))

def dxcc_in_matrix(dxcc, band, matrix_filename):
        with open(matrix_filename) as dxcc_json_data:
            dxcc_data = json.load(dxcc_json_data)
            return True
    except KeyError:
        return False

We can check whether the spot is a new DXCC and print relevant information to the terminal using the following snippet:

# Get DXCC information from clublog API
spotter_data = query_dxcc_info(spotter, api_key)
spotted_data = query_dxcc_info(spotted, api_key)
spotted_dxcc_route = str(spotted_data["DXCC"])

# Compare DXCC number to DXCC matrix, if there is an error the band has not been worked before
if band and spotted_dxcc_route and not dxcc_in_matrix(spotted_dxcc_route, band, dxcc_matrix_filename):
        print "New DXCC! %s (%s, %s) at %s by %s (%s, %s) %s" % (spotted,spotted_dxcc_route,spotted_data["Name"],frequency,spotter,spotter_data["Name"], comment, spot_time)

The DXCC is new if there’s no data for the given band. Maybe you can work it!

After running the script for some time, we noticed that it would give a lot of unnecessary data. An example is Asian calls spotting other Asian calls, which is useless in Norway, as it is unlikely that we would be able to hear them. To make the software more functional, some filtering with respect to distances and locations should be employed. This is possible using information obtainable from the ClubLog API. We will come back to this in a later post.

The script currently prints all information to the command line. The information can be pushed further to notify remote users in real-time through e.g. IRC or email, and can serve as a building block for several nice applications.

For now you may find the code and usage instructions available on github.

A quick sleeve metal dipole for 23 cm

We have been looking for a horizontally polarized omnidirectional antenna that could be used on a future 23 cm beacon. While a perfectly omnidirectional antenna would be best we are able to cope with some nulls in the antenna-pattern.  A quick and easy antenna that does this is a half-wave dipole, which is largely omnidirectional, but may have some nulls at certain elevations.

The half-wave dipole has a feed impedance of around 72 Ohms. To get even better performance the dipole can be matched to 50 Ohms with a balun. We chose to use a quarter wave impedance transformer, which is often called a sleeve or bazooka match. The sleeve also prevents unintentional radiation from the coaxial  feed cable.

The rest of this post is a tutorial on how to build the 23 cm sleeve dipole. For outdoors mounting it is made extra waterproof, this adds some additional design considerations.

Required materials:

  • 10 cm or longer coaxial cable (we use RG-223 for this guide)
  • 20 cm of 6 mm inner diameter 7 mm outer diameter copper tubing. If you use a thicker cable than RG-223 you can use a wider copper tube. This will increase the bandwidth of the antenna but lower the efficiency somewhat.
  • 20 cm adhesive lined shrink tubing 5-6 mm shrunk diameter
  • 5cm self-amalgamating tape (heat activated)

Required tools:

  • Hacksaw
  • Cable cutter
  • Knife
  • Soldering iron
  • Heat gun
  • Measuring tape or caliper
  • SWR-meter or network analyzer covering 23 cm (up to 1300 MHz, preferably higher)

Start by cutting three 55 mm long sections of copper tubing. I used 6 mm inner diameter and 7 mm outer diameter for my copper tubing in order to fit around the cable I decided to use. If you decide to make the antenna for a different frequency the cut length can be calculated roughly as (where f is the frequency in Hz):

The copper tubing is used for each leg of the dipole, as well as the quarter wavelength sleeve.

The next step is to prepare the coaxial cable. Three cuts should be made, one to remove 5 mm from the end of the jacket, where the dipole legs should be soldered. The two remaining cuts should be made at 55 mm and 60 mm, where the sleeve will be soldered to the coaxial shield.

The next step is to prepare the end of the coaxial cable for soldering to the copper tubing. If you are using a coaxial cable that is already terminated you must make sure to thread the sleeve tube and a piece of heat shrink that is 7-10 mm long over the coax before soldering the dipole legs.

Take extra care to avoid solder bridges between the left and right leg of the dipole, they should be electrically separate. When soldering the sleeve in the next step, make sure to avoid contact between either leg and the open end of the sleeve.

Next up is tuning the resonance. This is done by cutting the dipole legs until the antenna resonates at the desired frequency, for me this is 1296 MHz. One very interesting thing to note when adding heat shrink tubing on the outside of a radiator is that the resonant frequency will shift down. This is due to the increased dielectric constant of the heat shrink plastic compared to air. To mimic this behavior during measurements I place the heat shrink tubing over the copper tubing, but I don’t shrink it until I am satisfied with the resonance. Keeping in mind the dielectric effect, I tune the antenna a little bit too high to compensate for the dielectric constant, which gets a little larger when the plastic is fully shrunken.

Now all that remains is to add heat shrink to the dipole legs. In order to make it waterproof at the middle and at the ends I use a type of self-amalgamating rubber that melts into a solid mass when heat is applied.

The end result is a cheap and simple antenna that performs well. In total the antenna took about one hour to make.

Kuhne amplifiers and 1 to 10 GHz stage three

ARK is currently working on a project that will allow us to work Earth-Moon-Earth, satellites and various scattering modes on the amateur bands between 1 and 10 GHz. Our solution uses a 3 m parabolic dish together with a set of discrete amplifiers, the entire system is excited by a USRP SDR.

We have split the 1 to 10 GHz project into four sections. Up until now we have completed stage one and two, while stage three and four are still remaining. The stages are roughly:

Stage 1: Literature study, ordering of components – Further details in “An update on the 1 to 10 GHz project”

Stage 2: Construction of the parabolic dish and mast – Further details in “3m parabole dish ready”

Stage 3: PCB development and integration onto parabolic dish – Further details in this blogpost 😀

Stage 4: Long term projects with the dish, software, amplifiers, new antenna feed – Further details in the future.

This is a good time to elaborate more on our plans for stage 3 as the amplifiers that will be used for the project just arrived! We have purchased:

144/432 MHz IF to 10.5 GHz mixer – MKU 10 G4, 3 cm Transverter

To keep the work more organised we have split stage three into four sections. The sections are three PCB-design projects and one final assembly of all the components at the back of the parabolic dish.

1: Wideband driver amplifier
The Kuhne amplifiers and transverter seen in the previous section, will bring the output power to the level that is required to achieve Earth-Moon-Earth communications in the amateur radio bands. In order to be able to excite the Kuhne amplifiers and transverter from a USRP SDR with 10 mW max output power, an intermediate stage is required. The next step is to design and construct this amplified.

Very simplified schematic of 1 W driver.

We have conducted a study of available parts, and concluded that it is indeed possible to create an amplifier that will deliver 1 W across the frequency band from 0.1 GHz to 6 GHz.

In the figure above an example schematic using Guerilla RFs GRF4001 together with Analog Devices HMC637LP5 is shown. The device will deliver 1 W across 0.1 GHz to 6 GHz. This will allow exitation of all amplifiers, as well as the transverter that enables 10 GHz coverage. The gain should be on the order of 30 dB in order to avoid operating the USRP at its saturation power, where it is known to be quite noisy.

2: Wideband low noise amplifier
Another component that we want to develop ourselves is the low noise amplifier (LNA). There are not many good and cheap LNAs available for the amateur radio market, despite there being integrated circuits that boast very good performance for this application. If we are able to make an LNA and provide the design notes as open source, it will likely be beneficial for many people.

The LNA is one of the more challenging circuits. It needs to work using relatively cheap equipment while being largely immune to electromagnetic noise. A lot of work will likely be spent on making the supply-lines that power the HMC753 LNA circuit noise free, as well as ensuring that the metallic shielding is sufficiently tight. Another consideration is that the amplifier must be able to sustain relatively high input powers that will leak through the coaxial relays during transmit.

Outline of LNA.

The figure above shows a draft for a test assembly for performance testing of HMC753, which is a device that could be used in our LNA.

3: Controller board
Interfacing with the amplifiers will be handled through a controller board that communicates with either a computer or the USRP directly over the serial protocol RS232. The interface board is responsible for managing power supply states for all amplifiers as well as startup sequencing.

A set of RF relays are used to select which of the discrete amplifiers should be connected to the different points along the circuit. These are available as surplus devices on auction sites such as eBay.

Essentially, the interface board is responsible for ensuring that all connections and devices in the figure below are connected and powered correctly for a given configuration. It should also be able to alter the configuration in a rapid way.

Relays, amplifiers, transverters and SDR connection diagram.


4: Mechanical integration

After the three sections above are complete, the mechanical integration of the RF system onto the dish can start. This is an extensive effort as there are many concerns to deal with. Thermal management and waterproofing are two likely issues. So far we have an idea revolving around a gutter heater solution to keep the system from freezing during the winter. To keep the system cool enough we are experimenting with different heatsinks and weatherproof fans (IP68).

We hope to have the first three stages finished by the end of november, and the mechanical work started and delivered some time early next year. The time it takes to develop the PCBs gives us a good chance to secure the final funds that are required for the mechanical work (cabinets, fans, heatsinks). Overall we are really excited to see the project taking shape.

Before we finish all the sub-projects in stage three we might try to work some contacts using the 23cm module on our IC-9100, a MKU 131 AH 23 cm LNA, coaxial relays and the 200 W 23 cm PA we just bought. More on that in a later post.

« Older posts