Sunday, September 13, 2020

LoRa Radio for the Amateur Radio Operator

 LoRa is a long range spread spectrum modulation technique derived from chirp spread spectrum (CSS) technology. Generally these devices are long range, low power, devices that are used for the Internet of Things (IoT) networks. 

Generally these devices are low in cost, and programmable using the Arduino IDE. There are many different Arduino compatible boards now. LoRa is different than LoRaWAN - this project just uses LoRa devices, and the devices are all under $30.00 U.S. dollars. (most a lot less than that).  

There are several projects like this, actually a lot more then several. There are even a few specifically for Ham radio. Probably one of the better known projects is from Travis Goodspeed and is simply called LoRaHam https://github.com/travisgoodspeed/loraham. This project hasn't been updated in about 2 years. The LoRaHam project is one of the first, if not the first project specifically for amateurs. The project code is easy to read, and it is fairly easy to understand what is going on.  Travis is using Adafruit M0 LoRa Feathers, which are a little more expensive. The drawback was I did not see how it would scale, the other thing that I thought about was these are IoT devices, meaning you could set up a sensor network, but there was no provision for adding devices that were not "chat" devices.  A more recent ham radio project called LoRaMaDor by Elvis Pfutzenreuter https://github.com/elvis-epx/LoRaMaDoR. Was inspired by LoRaHam, this project is still active, and I've contributed some small python scripts to the project. This project was designed more as a terminal program. It uses ESP32 devices mainly because the sketch uses a lot of memory, and the ESP32s have plenty of memory.  This project uses a better, more scale-able protocol, which I believe is similar to AX.25, thou I'm not 100% sure about that. The main draw back to the LoRaMaDor is how big the sketch when it's compiled. It appears that it was designed to work with just the ESP32 hardware. I did not try it with a LoRa M0 because I don't own one. The other problem is the sketch is just confusing to someone who may not be a programmer. I have been programming for a while, but I only consider myself to be an intermediate programmer, and while I understood some of it, other parts just were confusing. Maybe in another 5 years I'll understand.

I am also aware of the Hamshield for the Arduino, and aware there is a "chat" sketch for that device, that can be found here: https://github.com/EnhancedRadioDevices/LoRaChat. I don't know anything about this device except that it at a slightly higher power output than the "cheap" IoT devices. 

My project which tentatively I am calling LoRaAPS or LoRa Amateur Packet Service. The name may change based on feedback, or it may not. I was inspired by both of the above projects, and APRS.

What makes this project different, I attempted to create something that is as simple to understand as LoRaHam, but with "real" protocols that can be expanded on, and scaled. I am treating the device as an IoT device, like it was designed to be. The protocol uses json, the gateway uses a MQTT broker. The device will beacon every 10 minutes (like LoRaMaDor devices), It uses some of the same settings as LoRaMaDor, but isn't directly compatible with that system. By using some common IoT systems, interfacing to the devices should be fairly easy and straight forward. 

The sketches were written to able to add or change LoRa hardware as needed. Currently the hardware I have is Two LoRa32u4 devices, based on the Arduino Leonardo, Two TTGO T-Beam v1 with Oled display and GPS, and 4 TTGO ESP32-Oled. 

LoRaAPS uses Repeaters (digipeaters), pagers, and gateway devices, the roll of the device depends on which sketch is loaded onto the device. However LoRa32u4 devices seem to lean to being digipeaters. 

Roles:

DIGIPEATER - These devices will repeat any packet that they get, there is limited checking, if a message is addressed to the DIGIPEATER it will not be repeated. IE: Our digitpeater's callsign is KD8BXP-10.  If we send a message to KD8BXP-10 and if the digipeater hears the message, the message will stop there. If these are hooked to a computer, you can use a terminal program to send messages, but currently there is no way to see a message addressed to the digipeater. Sketch is called "relay"

GATEWAYS - These devices will also repeat any packet they hear. Packets not addressed to them will also be sent to a MQTT broker. Packets that are addressed to them will stop, and not travel. ESP32 devices appear to be perfect for this role, because they can connect to a WiFi network. Like the digipeater there is a limited serial interface, for sending messages, they will not show messages addressed to them. GATEWAYS add a few more limited checks for messages, mostly so a message they have already send doesn't get resent either OTA or back to MQTT.

PAGER - these are mobile devices, because the "network" is relatively small/new, they will also digipeat any packet they hear. Devices with some type of screen work well for this role, ESP32 devices with OLEDs. These devices will show messages addressed to them on the screen. If using an ESP32 a bluetooth terminal can be used to send messages, or if you connect them to a computer a serial console can be used.  These devices will also show "BEACON", "CQ", "WX", and "BLT" special calls. 

SERIAL PORT:  Settings: All roles can send messages via the serial port if connected to a computer, PAGERs using ESP32 devices can also send messages with a bluetooth terminal.  Serial Port Settings: 8-N-1 9600baud, the newline code is used to indicate you are ready to send.  You may also want to turn on "ECHO" for your terminal program, just to make it easier to see what you are typing. For Bluetooth you must make sure the BT terminal program you are using can send a "newline" code.  *Using PUTTY or the Arduino Serial Console is probably recommended* 

CALLSIGNS: All devices need to have a callsign with two digit ssid,00 to 99, which gives 100 possible devices. Callsigns must be unique, otherwise problems will arise, missed messages, or messages not sent, and could create other issues. 

SPECIAL CALLS: "BEACON", "CQ", "WX", "BLT".  "BEACON" this is sent every 10 minutes as long as the device is on, the message it sends can be changed within limits (Remember to keep the message short if you want to change it) by Default it will send "DE: KD8BXP-00 LoRaAPS v1 pager" or something similar to that. "CQ" can be used to see if anyone is around that wants to chat, like a normal "CQ", to use just send "CQ some message".  "WX" weather information, this is for future expansion, but could be used to ask a sensor for information, or used to make general weather reports. (*Still a work in progress, however it can be used now). "BLT" is like a bulletin on the APRS system, this could be used to make a general short anouncement, Like "BLT KD8BXP-20 Gateway going off line" or "BLT Net tonight at 7pm on W8BLV 146.610 repeater". I considered adding "GPS", "NET" and "CHAT" - but thought between "BLT" and "CQ" that could cover "NET" and "CHAT", and I couldn't figure out what I really thought "GPS" would be used for.

ROUTING: Routing is based off APRS "hops". APRS recommends 3 hops, the path for APRS is recommended to be WIDE1-1, WIDE2-2. What this means is the first station that hears a packet will mark off the first WIDE1-1, causing the path to become WIDE1-1*, WIDE2-2. The next station will mark off one of the hops from WIDE2-2, causing the path to look like this WIDE1-1*, WIDE2-1, finally the third station that hears the packet will retransmit and mark off the last hop. The next station to hear the packet will not retransmit.  This project uses a count down, when the counter reaches 0, the packet will no longer be transmitted. Because all the devices also act as digipeaters the use of a specific call was not needed (IE: WIDE1-1). On each hop the station that repeats the packet will also add it's call to the packet. (* Not implimented yet * Ability to display the path the packet took)

PACKET: When sending a message to someone you may see something like this:  
{"T":"KD8BXP-00","F":"KD8BXP-02","M":"This is the message","R":3,"P":["KD8BXP-02","NOCALL1","NOCALL2","NOCALL3"]}
The above is a json string that is sent OTA, and if coming from a GATEWAY to an MQTT broker. Json  stands for JavaScript Object Notation, it's lightweight format for storing and transporting data. It is common to find websites that use JSON, it has also become a defacto standard for the IoT. For machines this is easy to read and understand, while for humans it may look like it doesn't make much since, it is actually pretty easy to read simple json. Originally the JSON for this project was going to look a little different, I had to make some changes because of limits of how much data could be sent over LoRa. IF we look at what I had originally thought, it might be a little easier for a human to read.
{"TO":"KD8BXP-00","FROM":"KD8BXP-02","MESSAGE":"This is the message","RT":3,"PATH":["KD8BXP-02","NOCALL1","NOCALL2","NOCALL3"]}

So, hopefully this makes a little more since. JSON is generally in pairs, so in the above example. "TO" and "KD8BXP-00" are together. This message is meant for the device that has the callsign "KD8BXP-00". "FROM" and "KD8BXP-02" are a pair, so KD8BXP-02 sent the message, "MESSAGE" well this should be self explanatory, it's the message that has been sent. "RT" and 3, this is the re-transmit counter, and will be updated with each hop. Finally "PATH" as you can see PATH is associated with a group as signified from the [ ] braces. But the idea is the same, as the simple example. "PATH" each time the packet is re-transmitted, the callsign of the transmitting station will be added, first spot will be the original call (yes, "FROM" and the first call in the PATH are duplicated.) NOCALL1 will be replaced by the first hop, NOCALL2 by the 2nd hop, and NOCALL3 by the 3rd and final hop.

To save some space when sending OTA, everything was shorten to the first character of what it represents. It does make it a little harder for humans to read, but when you know what they mean, it is still readable. "T" = "TO", "F" = "FROM", "M" = "MESSAGE", "R" = "RT", "P" = "PATH". For the TTGO T-Beam, another field was added for "C" = "COORDINATES" or latitude, and longitude. It will look something like the "PATH" field.  {"C":[38.39899,-84.4994]}. The TTGO T-Beam will add it's latitude and longitude to each packet sent. But currently no other device uses this information.

SENDING A MESSAGE: While all the various roles can be used to send a message, only a pager can currently display a message received. Because of this, I recommend only using the pagers to send messages. The easy way is to connect the pager to a computer and use a terminal program, see above for settings. Sending a message is "callsign-ssid message", a message of upto 77 characters can be sent. IE:
"KD8BXP-00 This is my test message"

With out the quotes, If you forget the ssid, or if the message is too long you will get an error message in the serial console. (*BT currently has no feedback, an updated version will be coming soon). CALLSIGNS can be entered in low case, or upper case as desired. This makes it simple to create programs that can interface with the devices. IE: python script, or other types of programs.

ARDUINO SKETCHES: I am not going to go into how to install the correct "board core" for your device, there hundreds of tutorials on that subject. I'm also not going to go into how to install a library, again, there are a lot of tutorials for this as well. I do want to say which libraries are required and were to find those libraries. 

  1. LoRa https://github.com/sandeepmistry/arduino-LoRa
  2. ArduinoJson (please use version 5.13.5) https://github.com/bblanchon/ArduinoJson
  3. TimedAction https://playground.arduino.cc/Code/TimedAction/
  4. SSD1306Ascii https://github.com/greiman/SSD1306Ascii
  5. PubSubClient https://github.com/knolleary/pubsubclient
  6. (for the T-Beam Device) TinyGPS++ https://github.com/mikalhart/TinyGPSPlus
  7. (for the T-Beam Device) axp20x https://github.com/lewisxhe/AXP202X_Library

Sketch names: gatewayLoRa32, pagerLoRa32, pagerT-Beam, relay32u4, relayLoRa32

GatewayLoRa32 - This is written for the ESP32 devices (LoRa32-Oled) the display is not used, this will gate messages to a MQTT Broker. see GATEWAYS above. 

 PagerLoRa32, PagerT-Beam - these are written for the ESP32 based boards, with OLED display support. The T-Beam version also adds GPS support. See PAGERS above.

Relay32u4, RelayLoRa32 - These are digipeaters, Relay32u4 was written for the LoRa32u4 boards, while RelayLoRa32 is written for the ESP32 device, no OLED display, these will just relay (digipeat) see DIGIPEATER above.

The sketches are written in such away that if you do not have a ESP32 or LoRa32u4 or T-Beam, you should be able to adapt the sketches to work with what you do have. * Please share if you have other hardware, let's extend the project and make the network better. *

LORA SETTINGS: LoRa settings should be considered for each area, the only thing to consider is frequency to use. The current project uses 433mhz hardware, with a spreading factor of 9, coding rate of 5, tx power of 17, and bandwidth of 62500. Every device using the network has to match, frequency, and the above. These can be local settings however, and don't have to be used globally. But if you want to communicate with another ham in your own town, the settings do have to match each other. 

MQTT BROKER: MQTT is a lightweight messaging protocol for small sensors and mobile devices, optimized for high-latency or unreliable networks. Originally designed by IBM in 1999, it was used to monitor oil pipelines. The goal was to have a protocol that is bandwidth-efficient, and uses little battery power. This is a publish/subscribe with topic hierarchy. IE: Device 1 publishes a message "Light ON" to topic "LIGHTS", Device 2 (The light) subscribes to the topic "LIGHTS" and acts on the message "Light ON". For this project, the gateway subscribes to the same topic it publishes too, this is a little strange, but not unheard of, our gateway will filter out it's own messages, it will also filter out invalid JSON.  The project currently uses a public MQTT broker, because of this anyone can subscribe or publish to the topic (an attempt was made to make the topic hard to remember, but not impossible). Also because this connection could be seen by the public consider every message sent to be in the clear, don't send sentive information. 

Because MQTT is a common and standardized protocol, phone apps, python scripts, or custom written software can be used to expand on the network, and make the network more useful.

* IF this project gains lots of users, it may make more since to more to a private broker, at which time the project would need to gain funds to fund that broker. 

** A Raspberry PI could also be used as a broker, this might be good for keeping information within a local network, and not gated out to the world. If you choose to go this route, you might also consider using a MQTT bridge to bridge to more/other networks. http://www.steves-internet-guide.com/mosquitto-bridge-configuration/

RECOMMENDED SETUP: At this time the "network" is small, very small. So I would recommended setting up at least one gateway, one relay (digipeater), and one pager. Good antennas are almost a must for this project, but they will work with the stock "antenna".  DIGIPEATERS should be on higher ground,  while a gateway can sit on your desk to make it easier to connect it to the internet. PAGERs can either be mobile or stationary. SSIDs - there is no hard or fast rule to this, I would recommend using higher numbers for your GATEWAY and DIGIPEATERS, leaving the lower number ssids for the PAGERS.  IE: GATEWAY - KD8BXP-70, DIGIPEATER - KD8BXP-80, PAGER KD8BXP-00, or PAGER KD8BXP-01. If you plan on deploying some sensors you may want those to be in the 50 or 60 range. This are just recommendations, and are subject to change based on feedback, and other factors.

Sept 12, 2020 - v.5, documentation written. github repository created, code not yet published.