Field Data Collection System

The Field Data Collection System was desgined to collect data from multiple sensors, bundeled into so-called FDCS Nodes, which can be spread out across a large area. Each Node logs the recoreded data onto its own SD Card and may send the data wirelessly over radio to a so-called FDCS Receiver, which in-turn uploads it to the cloud. The system allows for easy and flexible deployment of a wide variety of sensors in a remote environement, with the nearest Internet access point beeing 5 to 10km away.

Table of Contents

• Components / Terminology
• Node Configuration
  • Logging Devices
  • External Controllers
  • Storage
  • Connectivity
  • Other
• Receiver Configuration
  • Nodes
  • Connectivity
  • Other
• Physical Setup
  • Connecting controller
  • Analog Sensors
  • I2C Sensors
• Software Setup
  • Computer Software
  • Firmware
  • Uploading the code
  • Configuration
• Finally there

---

Components / Terminology

• FDCS Node (aka. node): A microcontroller (RPi Pico) on which multiple devices are set up. A Node has an SD Card slot and can transmitt data to the receiver, over LoRa when a device "logs" something.
• External Controller (aka. controller): A microcontroller (Arduino Nano) that may be connected to an FDCS Node with a wired connection using the Serial protocol. External Controllers act like an extension of a Node, allowing for more logging devices to be set up (thorugh the Node). Useful when having two groups of spaciallly detached sensors.
• FDCS Receiver (aka. receiver): A microcontroller (Rpi Pico) that is connected to a WiFi Network and receives data from nodes, uploading it to a cloud service like "Thigspeak". Only one Receiver exists in an ecosystem.
• LoRa: The radio technology used for communication between nodes and the receiver
• Logging Device (aka. device): The software representation of a physical sensor connected to a node.
  • Logging Devices may log raw or slightly processed sensor data.
  • The term Logging Device usually refers to a class and the functionality that it provides, whereas device typically refers to instances of those classes aka. an entry in the loggingDevices JSON array of the configuration file. You can set up multiple devices of the same type (with the same id).
  • Logging refers to saving collected data onto the nodes SD Card and (if configured) sending it to the Receiver.
  • The log format is DEVICE_ALIAS,DATETIME,DATA1,DATA2,.... Currently all supported logging devices log one or two data (e.g. BMP390 logs pressure and temperature).
• Onboard Logging Device: A device, interfacing sensors that are directly connected to a node and not through a controller.
  • The IDs from 01 to 19 are reserved for such devices.
• Serial Logging Device: A Logging device that is set up on a controller and not on the node itself.
  • The IDs from 20 to 39 are reserved for serial devices and their class names are denoteted with a trailing capital S.
  • All Serial logging devices can only log periodically at the specified interval, since the node must first request a log through the external controller.
• Overlays: Overlays may be set up on logging devices. They act like a mask on the data that is beeing logged, processing it further and/or making it more readable. Examples:
  • A GenericDigitalInput device that logs 0 or 1 may be used with the DigitalOnOff Overlay to map those values to On and Off respectivly.
  • A GenericAnalogInputS or CircularRunningAverageS device may be used with the ZTSWindDirection Overlay to translate analog readings into wind direction, if the reading was provided by a sensor, the overlay was designed for.
• Ecosystem: A collection of multiple nodes (with or without external controllers) and one receiver.

---

Node Configuration

The Node configuration is divided into the following sections: Logging Devices, External Controllers, Connectivity and Other. You can find complete examples under pico-dev/config-examples.

• In path settings sd/ may be used to reference files stored on the sd card.
• The keywords required, optional and arbitrary are used in this documentation, with the last one signifiying that the setting is not used by the system.

Logging Devices

In the LoggingDevices array, zou specify the devices that you have connected either to the node or an external controller and want to use. A list of all available Logging Devices can be found under lookup/LoggingDevices.json.

Parameters

• id (required):

  • Specifies the Logging Device this device should be an instance of.
  • For example, the ID "01" is used for setting up a DHT-11 device.
  • Note that an id does not uniquely identify a device, as it is possible to have multiple instances of one logging device. Unique identification is achieved though alias.

• alias (required): A short string without white-spaces that uniquely identifies the device and is used in its log messages.

• info (arbitrary): User information to help identify the device (e.g., mounting location, purpose, etc.).

• overlay (optional): Specifies what overlay to use, either by providing a name as string or an object with a required name and other parameters. A list of all available overlays and the parameters they accept can be found under lookup/overlays.json. A missing parameter could cause an Exception on start-up or a hard-coded default value might be used instead.

• constructor_args (required): Specifies all configuration parameters for initializing the device.

  • Some logging devices have a set of default parameters defined in lookup/LoggingDevices.json.
  • interval: (required for periodic logging devices): Time between two logs in seconds. If set to 0 the device never logs unless logs are triggered by external events.
  • controller: (required for serial logging devices): id of the conteroller the device should be set up on.
  • Other parameters could be: pin, sda, scl, bus, resolution, type, etc. Consult docs for specific device or lookup/LoggingDevices.json.
  • Missing or falsly initialized parameters could result in an Exception on start-up.

Example

Here's an example of how a logging device might be configured in config/config.json:

{
    "LoggingDevices": [
        {
            "id": "01",
            "alias": "Room1_DHT11",
            "info": "Temperature Sensor",
            "constructor_args": {
                "interval": 600, //Logs each 100 minutes
                "pin": 5
            }
        },
        { //Serial device on controller "01"
            "id": "24",
            "alias": "24-rain",
            "info": "Tipping bucket rain gauge",
            "overlay": {
                "name": "TippingBucketRainGauge",
                "mmPerPulse": 3.5
            },
            "constructor_args": {
                "interval": 360,
                "controller": "01",
                "pin": 2,
                "trigger": "FALLING"
            }
        }
    ],
    //other sections...
}

External Controllers

In the controllers array, controllers connected to the device may be configured, assingning a controller id a settings object.

Parameters

• type (arbitrary): Controller type (eg. Arduino Nano).
• info (arbitrary): User provided info.
• triggerPin (optional): In their current implementation external controllers are sleeping and must be woke up through a third line. If specified, this line will be triggered through the provided pin (of the node).
• retries (optional): How often to retry sending a command if communication fails.
• uart (required): UART Settings
  • tx_pin (required), rx_pin (required), bus (required), timeout and more.

Example

{
  "controllers": {
        "01": {
            "type": "Arduino Nano",
            "info": "next to solar panel",
            "triggerPin": 10,
            "uart": {
                "bus": 0,
                "tx_pin": 12,
                "rx_pin": 13,
                "timeout": 500
            },
            "retries": 2
        }
    },
    //other sections...
}

Storage

Important note for SD Card selcetion: The system only supports SDHC SD-Cards up to 8GB and non-SDHC SD Cards up to 2GB. While other SD Cards might initially appear to work they might cause issues in the future!

• log_file_path: (optional) The path for the log.csv file, containing all device log messages. Default: sd/log.csv.
• error_log_file_path: (optional) The path for the error-log.txt files, capturing system messages and exceptions. Default: sd/error-log.txt.

Connectivity (Node)

• device_id (required for lora): An integer ranging from 0 to 255 uniquely identifying a node within one ecosystem
• lora (required): Adjust the following parameters depending on the environment
  • enable (optional): Enables logging over LoRa if set to true.
  • frequncy (required): Frequency of the LoRa module (usually 355.0 MHz)
  • bandwidth (required): Higher -> faster data rate, Lower -> longer range (Typical values: 125, 250 or 500 kHz)
  • spreading_factor (required): Higher -> longer range + less interfearence, Lower -> faster data rate (Typical values: 7 to 12)
  • coding_rate (required): Higher -> more reliable, Lower -> faster (Typical values 5 to 8)

Other (Node)

• external_rtc (optional): Use external rtc if set to true.

---

Receiver Configuration

The Receiver configuration is divided into the following sections: Nodes, Connectivity and Other.

Nodes

In the nodes array you specify all nodes that should part of your ecosystem and report to the receiver. You can individually choose if and over which service a node should upload the logging data of all its devices. Currently only Thingspeak is supported.

Paramters

• id (required): The node's node_id as specified in it's configuration file.
• info (arbitrary): User provided info (e.g. node location, purpose).
• thingspeak (optional): Thingspeak is an online service for real-time, ulimited data logging and visualization. This parameter allows for configuring a Thingspeak Channel for each node seperatly. Channels must have at least 4 fields enabled to account for devices that log 2 or more things. Please refer to the Thingspeak documentation for further information on how to set up channels.
  • channel_id (optional): This setting is optional as it is not required for writing to a channel.
  • write_key (required): An API Key associated with the channel issued with writing privilages.

Example

{
    "nodes": [
        {
            "id": 9,
            "info": "Next to olive trees",
            "thingspeak": {
                "channel_id": "5397853",
                "write_key": "0HIZG9HY5AUEBMKT"
            }
        }
    ],
    //other sections...
}

Connectivity (Receiver)

• lora (required): Same settings as for node. Important: The settings of node(s) and receiver must match.
• wifi (required):
  • ssid and password

Other (Receiver)

• lcd_display: if you have connected a lcd display via I2C Pins 0 and 1 you can use it to display logs and errors by setting this to true. Useful for debugging.

---

Physical Setup

This section guides you on how to wire up nodes, external controllers and sensors. Pinout diagrams can be found under docs/diagrams.

• In the configuration file only the numerical value of the pin is specified (without the leading D or A for Arduinos).

Connecting Node to external controller

Follow steps 1. and 2. Faulty connection will raise a ControllerException.

1. Connect the rx_pin (specified under controller->uart->rx_pin) of the node to the TX-labled pin of the external controller and the tx_pin to the RX-labled pin.
2. Connect the triggerPin of the node to pin D2 of the controller.

Analog Sensors

For analog sensors you have to use pins A0 to A7 on Arduinos / pins 26 to 28 on Picos. (Tipp: Avoid long wires if you want accurate and consistant results)

I2C Sensors

For I2C Sensors connect the scl and sda pins. For RPi Pico you must specify scl, sda and an additional bus parameter (see pico-pinout-diagram). You can use the pico-dev/tools/i2c_scanner.py tool to find out your device's address.

---

Software setup

This table shows what harware is supported for each FDCS component.

Software	Supported hardware
External Controller	Arduino Nano, Arduino Uno (not tested), etc.
Node	Raspberry Pi Pico W and WH, Raspberry Pi Pico 2 (not tested)
Receiver	Raspberry Pi Pico W and WH. (Wifi required)

Note: If you want to use Arduino Hardware that has not been tested you may have to adjust some parameters in the defines.h file.

This is a guide on how to set up the respective component software on Arduinos / Picos.

Computer Software

1. Thonny IDE: For flashing firmware and code onto the Raspberry Pi Picos (both for nodes and the receiver) you will need to install the Thonny IDE.
2. Arduino IDE: For uploading code onto Arduinos (used as external controllers) install the Arduino IDE.

Firmware

1. Raspberry Pi Pico: Connect the Pico to your computer , while pressing down on the BOOTSEL button. Locate the button at the lower - right corner of the Editor. Clicking on it will reveal a menu. Click on "Install Micropyhton" and in the new window select the model you are using before flashing the device.
2. Arduino: Firmware is pre-installed.
3. Python: To use setup tools, for fast and easy configuration, you will need a Python installation on your machine.

Uploading the code

After completing the above steps clone this repository to your device.

• For Raspberry Pi Picos: Connect the pico via USB. In the Thonny IDE under the file view, select the root folder of the component you want the Pico to be (pico-dev for nodes or pico-receiver for receiver), and click "Upload to /" and wait for the process to finish. And that's it! (Note: You may have to disconnect external controllers for your computer to recognize the Pico)
• For Arduinos: Connect the Arduino via USB. In the Arduino IDE open the .ino file that corresponds to the wanted compontent (arduino-mstr/arduino-mstr.ino for external controller)

Configuration

• Node: Create a config.json file and refer to Configration (Node) to set up logging devices and more! (Important: If you are using multiple nodes dont forget to give them distict node_ids.). Now place the file either at the root of the SD Card or upload it to the root of the Pico filesystem with Thonny. The Pico will search for it in exactly this order.
• RTC (Node): To set the correct time for a node, insert a battery in the Real Time Clock module (if present) and run pico-dev/tools/rtc_configurator.py after connecting the node to your computer.
• Receiver: Create a config.json file and refer to Configration (Receiver) to set up nodes, wifi, thingspeak, etc. Upload it to the root of the Pico filesystem. You can use the pico-receiver/tools/thingspeak-fields-setup.py tool, to setup Thingspeak fields, based on an existing Node Configuration file.

---

Finally there

You can now power on the components all together. The built-in LED next to the USB port should rapidly flash three times on start-up. If it flashes again shortly after, an exception was thrown (5 times for fatal exception likely caused by faulty configuration / SD Card failure, 3 times if one or more devices failed to initialize). Exceptions are recorded in the error_log.txt file created automatically under the path specified in error_log_path (if possible) or at the Pico root. Extract the file for further information. If all went well you should be able to:

• View logs on Thingspeak (if set up)
• View logs on the SD Card
• View logs on the display of the receiver (if set up)