init v2

.github/workflows/main.yml

@@ -1,11 +1,11 @@
-name: ci
+name: build
 on:
   push:
     tags:
       - 'v*'
     paths-ignore:
     - 'README.md'
-    #datasets
+    - datasets/**
 
 env:
   BUILD_VERSION: "0.1.0"
@@ -33,14 +33,19 @@ jobs:
         with:
           username: ${{ secrets.DOCKERHUBUNAME }}
           password: ${{ secrets.DOCKERHUBTOKEN }} 
+      - 
+        name: Lint with flake8
+        run: |
+          pip install flake8
+          flake8 . --max-line-length=130
       -
         name: Run Buildx
         run: |
           docker buildx build --push \
-          --tag hvalev/dht22mqtt:latest \
-          --tag hvalev/dht22mqtt:${BUILD_VERSION} \
-          --tag ghcr.io/hvalev/dht22mqtt:latest
-          --tag ghcr.io/hvalev/dht22mqtt:${BUILD_VERSION}
+          --tag hvalev/dht22mqtt-homeassistant:latest \
+          --tag hvalev/dht22mqtt-homeassistant:${BUILD_VERSION} \
+          --tag ghcr.io/hvalev/dht22mqtt-homeassistant-docker:latest
+          --tag ghcr.io/hvalev/dht22mqtt-homeassistant-docker:${BUILD_VERSION}
           --platform linux/arm/v7,linux/arm64 .
       - 
         name: Docker Hub Description

Dockerfile

@@ -1,5 +1,5 @@
 FROM python:3.9.1-alpine3.12
-COPY requirements.txt dht22mqtt.py ./
+COPY requirements.txt dht22mqtt.py gpiomapping.py ./
 RUN apk add gcc musl-dev && \
     pip3 install -r requirements.txt --no-cache-dir && \
     pip3 cache purge && \

README.md

@@ -1,11 +1,68 @@
-# dht22mqtt
+# dht22 temperature/humidity sensor in a docker container
+![build](https://github.com/hvalev/dht22mqtt-homeassistant/workflows/build/badge.svg)
+![Docker Pulls](https://img.shields.io/docker/pulls/hvalev/dht22mqtt-homeassistant)
+![Docker Stars](https://img.shields.io/docker/stars/hvalev/dht22mqtt-homeassistant)
+![Docker Image Size (latest by date)](https://img.shields.io/docker/image-size/hvalev/dht22mqtt-homeassistant)
 
-#full|essential
-#log2file|log2stdout
+This docker container enables you to use the DHT11, DHT22 or AM2302 temperature and humidity sensors with a GPIO enabled device such as raspberry pi and relay its readings to an MQTT broker. Additionally, it integrates with home assistants' [auto-discovery](https://www.home-assistant.io/docs/mqtt/discovery/) feature. Discovery automatically detects the presence of the sensor and makes it available for visualizations, automations, etc. Finally, the container implements a robust outlier detection scheme, which filters outliers before they are sent to the MQTT broker (and subsequently home assistant) resulting in clean and consistent graphs.
 
-#Things to write in the description
-#debug both-> make sure to output to stout that they need to mount a docker volume to access files -> mount docker volume folder:/log/
+## How to run it
+The following docker run command or docker-compose service will get you up and running with the minimal configuration.
 
-#Aknowledgement
-#TODO https://github.com/jumajumo/dht22-docker-arm/blob/master/publish.py
-#TODO https://forum.dexterindustries.com/t/solved-dht-sensor-occasionally-returning-spurious-values/2939/5
+```docker run --device=/dev/gpiomem:/dev/gpiomem -e topic=zigbee2mqtt -e device_id=dht22 -e broker=192.168.X.X -e pin=4 hvalev/dht22mqtt-homeassistant```
+
+```
+version: "3.8"
+services:
+  dht22mqtt:
+    image: hvalev/dht22mqtt-homeassistant
+    container_name: dht22mqtt
+    devices:
+      - /dev/gpiomem:/dev/gpiomem
+    environment:
+      - topic=zigbee2mqtt
+      - device_id=dht22
+      - broker=192.168.X.X
+      - pin=4
+```
+```/dev/gpiomem:/dev/gpiomem``` is required to access the GPIO and communicate with your DHT22 sensor. If it doesn't work, you can try to run the container in priviledged mode ```priviledged:true```.
+
+## Parameters
+The container offers the following configurable environment variables:</br>
+```topic``` - MQTT topic to submit to. Default: ```zigbee2mqtt```. </br>
+```device_id``` - Unique identifier for the device. Default: ```dht22```. *If you have multiple, you could use something like ```bedroom_dht22```.* </br>
+```broker``` - MQTT broker ip address. Default: ```192.168.1.10```. </br>
+```pin``` - GPIO data pin your sensor is hooked up to. Default is ```4```. </br>
+```poll``` - DHT22 sampling rate in seconds. Default is ```2```. [*For further information: DHT11/DHT22/AM2302 spec sheet.*](https://lastminuteengineers.com/dht11-dht22-arduino-tutorial/) </br> 
+```device_type``` - Sensor type. Possible values are ```dht11``` or ```dht22```, which also works for AM2302. Default: ```dht22``` </br>
+```unit``` - Measurement unit for temperature. Either ```C```elsius or ```F```ahrenheit. Default: ```C```. </br>
+```mqtt_chatter``` - Controls how much information is relayed over to the MQTT broker. Possible *non-mutually exclusive* values are ```essential|ha|full```. Default: ```essential|ha```. </br>
+&emsp;&emsp;```essential``` - Enables basic MQTT communications. </br>
+&emsp;&emsp;```ha``` - Enables home assistant discovery. </br>
+&emsp;&emsp;```full``` - Enables sending information about the outlier detection algorithm internals over to the MQTT broker. </br>
+
+```logging``` - Logging strategy. Possible values are ```log2stdout|log2file```. Default is ```None```. </br>
+&emsp;&emsp;```log2stdout``` - Forwards logs to stdout, inspectable through ```docker logs dht22mqtt```. </br>
+&emsp;&emsp;```log2file``` - Logs temperature and humidity readings to files timestamped at containers' start. </br>
+
+*If you end up using ```log2file```, make sure to add this volume in your docker run or docker-compose commands ```- ~/dht22mqttlog:/log``` to be able to access the logs from your host os.* </br> 
+*If you want to run this container to simply record values to files with no MQTT integration, you need to explicitly set ```mqtt_chatter``` to a blank string. In that case, you can also omit all MQTT related parameters from your docker run or compose configurations.*
+
+## Outlier detection scheme
+To detect outliers, I'm using the [68–95–99.7 rule](https://en.wikipedia.org/wiki/68%E2%80%9395%E2%80%9399.7_rule), where I'm using 3 standard deviations from the mean. 
+In order to have a scheme, which is adaptive to rapid changes, I'm using two [FILO](https://everythingcomputerscience.com/discrete_mathematics/Stacks_and_Queues.html) stacks of length 10 for temperature and humidity.
+This allows the algorithm to adapt to gradual changes in received sensor readings, while discarding implausible ones.
+Furthermore, when reading new measurements, only those which are not contained in the stack are added in, in order to prevent the stack from homogenizing.
+This approach ensures that the std calculation will never return 0.
+Finally, the algorithm keeps track on successive outlier detections and flushes the stack when 3 such readings are detected. This prevents the stack from fitting on outliers if we presume that we get abnormal values less than half the time. 
+Then again, perhaps the temperature really is 97°C.
+Most of the time, outliers detected for temperature and humidity are heavily correlated.
+As such a conservative approach is implemented, where only non-outliers for both temperature and humidity and forwarded.
+
+## Visualize your own data and/or contribute!
+If you'd like visualize and check your own data (provided you have used the ```-logging=log2file``` option) to perhaps tweak the sampling rate (or stack size, if you're building it yourself), you can take a look at the ```dht22mqtt_visualize.py``` script. As long as you change the filename to your own, it will plot your data at the original sampling rate and simulate sparser ones for comparison. I'm also open to the idea of accepting pull requests with novel datasets **with interesting features** to this repository, where the outlier detection scheme fails. This can help me adapt and tune the algorithm. If you'd like to do that, please crop the part where the algorithm fails with some pre- and postamble data and avoid sending me weeks worth of temperature and humidity. Ideally, the pre- and postamble should be only a couple of hours in length. 
+
+## Aknowledgements
+The following resources have been very helpful in getting this project up and running: </br>
+https://github.com/jumajumo/dht22-docker-arm/blob/master/publish.py </br>
+https://forum.dexterindustries.com/t/solved-dht-sensor-occasionally-returning-spurious-values/2939/5

dht22mqtt.py

@@ -2,44 +2,41 @@
 from datetime import datetime
 import time
 import os
-import sys
 import statistics
 import csv
-
-import board
 import adafruit_dht
-
-import RPi.GPIO as GPIO
-
+# import RPi.GPIO as GPIO
+from gpiomapping import gpiomapping
 import paho.mqtt.client as mqtt
 
-#Begin
+# Begin
 dht22mqtt_start_ts = datetime.now()
 
 ###############
 # MQTT Params
 ###############
-mqtt_topic = os.getenv('topic','zigbee2mqtt/')
-mqtt_device_id = os.getenv('device_id','dht22')
-mqtt_brokeraddr = os.getenv('broker','192.168.1.10')
-if not mqtt_topic.endswith('/'): mqtt_topic = mqtt_topic + "/"
+mqtt_topic = os.getenv('topic', 'zigbee2mqtt/')
+mqtt_device_id = os.getenv('device_id', 'dht22')
+mqtt_brokeraddr = os.getenv('broker', '192.168.1.10')
+if not mqtt_topic.endswith('/'):
+    mqtt_topic = mqtt_topic + "/"
 mqtt_topic = mqtt_topic + mqtt_device_id + '/'
-mqtt_homeassistant = os.getenv('ha_enabled','True')
 
 ###############
 # GPIO params
 ###############
-#TODO how to get pin to board mapping -> GPIO needed
-#TODO check if we can use the GPIO test https://github.com/kgbplus/gpiotest
+# TODO check if we can use the GPIO test https://github.com/kgbplus/gpiotest to autodetect pin
+# Problems with multiple sensors on the same device
 dht22mqtt_refresh = int(os.getenv('poll', '2'))
 dht22mqtt_pin = int(os.getenv('pin', '4'))
+dht22mqtt_device_type = str(os.getenv('device_type', 'dht22')).lower()
 dht22mqtt_temp_unit = os.getenv('unit', 'C')
 
 ###############
-# Logging params
+# MQTT & Logging params
 ###############
-dht22mqtt_mqtt_chatter = os.getenv('mqtt_chatter','full|essential')
-dht22mqtt_logging_mode = os.getenv('logging','log2file|log2stdout')
+dht22mqtt_mqtt_chatter = str(os.getenv('mqtt_chatter', 'essential|ha|full')).lower()
+dht22mqtt_logging_mode = str(os.getenv('logging', 'None')).lower()
 dht22mqtt_sensor_tally = dict()
 
 ###############
@@ -54,58 +51,67 @@ dht22_stack_size = 10
 dht22_std_deviation = 3
 dht22_error_count_stack_flush = 3
 
+
 ###############
 # Logging functions
 ###############
-def log2file(filename,params):
+def log2file(filename, params):
     if('log2file' in dht22mqtt_logging_mode):
-        ts_filename = dht22mqtt_start_ts.strftime('%Y-%m-%dT%H:%M:%SZ')+'_'+filename+".csv"
-        with open("/log/"+ts_filename,"a+") as file:
+        ts_filename = dht22mqtt_start_ts.strftime('%Y-%m-%dT%H-%M-%SZ')+'_'+filename+".csv"
+        with open("/log/"+ts_filename, "a+") as file:
             w = csv.DictWriter(file, delimiter=',', lineterminator='\n', fieldnames=params.keys())
             if file.tell() == 0:
                 w.writeheader()
             w.writerow(params)
 
+
 def log2stdout(timestamp, msg):
     if('log2stdout' in dht22mqtt_logging_mode):
         print(datetime.fromtimestamp(timestamp).strftime('%Y-%m-%dT%H:%M:%SZ'), str(msg))
 
+
 ###############
 # Polling & Processing functions
 ###############
 def getTemperatureJitter(temperature):
-    return getTemperature(temperature-0.3),getTemperature(temperature+0.3)
+    return getTemperature(temperature-0.3), getTemperature(temperature+0.3)
+
 
 def getTemperature(temperature):
     if(dht22mqtt_temp_unit == 'F'):
         temperature = temperature * (9 / 5) + 32
     return temperature
 
+
 def getHumidity(humidity):
     return humidity
 
+
+###############
+# Polling & Processing functions
+###############
 def processSensorValue(stack, error, value, value_type):
-    #flush stack on accumulation of errors
+    # flush stack on accumulation of errors
     if(error >= dht22_error_count_stack_flush):
         stack = []
         error = 0
-    
-    #init stack
+
+    # init stack
     if(len(stack) <= dht22_error_count_stack_flush):
         if(value not in stack):
             stack.append(value)
-        #use jitter to bootstrap temperature stack
+        # use jitter for bootstrap temperature stack
         if(value_type == 'temperature'):
             low, high = getTemperatureJitter(value)
             stack.append(low)
             stack.append(high)
         return stack, error, None
-    
-    #get statistics
+
+    # get statistics
     std = statistics.pstdev(stack)
     mean = statistics.mean(stack)
 
-    #compute if outlier or not
+    # compute if outlier or not
     if(mean-std*dht22_std_deviation < value < mean+std*dht22_std_deviation):
         outlier = False
         if(value not in stack):
@@ -114,16 +120,17 @@ def processSensorValue(stack, error, value, value_type):
     else:
         outlier = True
         error += 1
-    
-    #remove oldest element from stack
+
+    # remove last element from stack
     if(len(stack) > 10):
         stack.pop(0)
     return stack, error, outlier
 
+
 ###############
 # MQTT update functions
 ###############
-def updateEssentialMqtt(temperature,humidity,detected):
+def updateEssentialMqtt(temperature, humidity, detected):
     if('essential' in dht22mqtt_mqtt_chatter):
         if(detected == 'accurate'):
             payload = '{ "temperature": '+str(temperature)+', "humidity": '+str(humidity)+' }'
@@ -133,6 +140,24 @@ def updateEssentialMqtt(temperature,humidity,detected):
             client.publish(mqtt_topic + "detected", str(detected), qos=1, retain=True)
         client.publish(mqtt_topic + "updated", str(datetime.now()), qos=1, retain=True)
 
+
+def registerWithHomeAssitant():
+    if('ha' in dht22mqtt_mqtt_chatter):
+        ha_temperature_config = '{"device_class": "temperature",' + \
+                                ' "name": "'+mqtt_device_id+'_temperature",' + \
+                                ' "state_topic": "'+mqtt_topic+'value",' + \
+                                ' "unit_of_measurement": "°'+dht22mqtt_temp_unit+'",' + \
+                                ' "value_template": "{{ value_json.temperature}}" }'
+        ha_humidity_config = '{"device_class": "humidity",' + \
+                             ' "name": "'+mqtt_device_id+'_humidity",' + \
+                             ' "state_topic": "'+mqtt_topic+'value",' + \
+                             ' "unit_of_measurement": "%",' + \
+                             ' "value_template": "{{ value_json.humidity}}" }'
+        client.publish('homeassistant/sensor/'+mqtt_device_id+'Temperature/config', ha_temperature_config, qos=1, retain=True)
+        client.publish('homeassistant/sensor/'+mqtt_device_id+'Humidity/config', ha_humidity_config, qos=1, retain=True)
+        log2stdout(datetime.now().timestamp(), 'Registering sensor with home assistant success...')
+
+
 def updateFullSysInternalsMqtt():
     if('full' in dht22mqtt_mqtt_chatter):
         client.publish(mqtt_topic + "sys/temperature_stack_size", len(dht22_temp_stack), qos=1, retain=True)
@@ -141,6 +166,7 @@ def updateFullSysInternalsMqtt():
         client.publish(mqtt_topic + "sys/humidity_error_count", dht22_hum_stack_errors, qos=1, retain=True)
         client.publish(mqtt_topic + "updated", str(datetime.now()), qos=1, retain=True)
 
+
 def updateFullSensorTallyMqtt(key):
     if('full' in dht22mqtt_mqtt_chatter):
         if key in dht22mqtt_sensor_tally:
@@ -150,89 +176,90 @@ def updateFullSensorTallyMqtt(key):
         client.publish(mqtt_topic + "sys/tally/" + key, dht22mqtt_sensor_tally[key], qos=1, retain=True)
         client.publish(mqtt_topic + "updated", str(datetime.now()), qos=1, retain=True)
 
+
 ###############
 # Setup dht22 sensor
 ###############
 log2stdout(dht22mqtt_start_ts.timestamp(), 'Starting dht22mqtt...')
-dhtDevice = adafruit_dht.DHT22(board.D4, use_pulseio=False)
+if(dht22mqtt_device_type == 'dht22' or dht22mqtt_device_type == 'am2302'):
+    dhtDevice = adafruit_dht.DHT22(gpiomapping[dht22mqtt_pin], use_pulseio=False)
+elif(dht22mqtt_device_type == 'dht11'):
+    dhtDevice = adafruit_dht.DHT11(gpiomapping[dht22mqtt_pin], use_pulseio=False)
+else:
+    log2stdout(datetime.now().timestamp(), 'Unsupported device '+dht22mqtt_device_type+'...')
+    log2stdout(datetime.now().timestamp(), 'Devices supported by this container are DHT11/DHT22/AM2302')
 
 log2stdout(datetime.now().timestamp(), 'Setup dht22 sensor success...')
 
 ###############
 # Setup mqtt client
 ###############
-client = mqtt.Client('DHT22', clean_session=True, userdata=None)
+if('essential' in dht22mqtt_mqtt_chatter):
+    client = mqtt.Client('DHT22', clean_session=True, userdata=None)
 
-client.will_set(mqtt_topic + "state", "OFFLINE", qos=1, retain=True)
+    # set last will for a disgraceful exit
+    client.will_set(mqtt_topic + "state", "OFFLINE", qos=1, retain=True)
 
-#keep alive for 60 times the refresh rate
-client.connect(mqtt_brokeraddr, keepalive=dht22mqtt_refresh*60)
+    # keep alive for 60 times the refresh rate
+    client.connect(mqtt_brokeraddr, keepalive=dht22mqtt_refresh*60)
 
-client.loop_start()
+    client.loop_start()
 
-client.publish(mqtt_topic + "type", "sensor", qos=1, retain=True)
-client.publish(mqtt_topic + "device", "dht22", qos=1, retain=True)
+    client.publish(mqtt_topic + "type", "sensor", qos=1, retain=True)
+    client.publish(mqtt_topic + "device", "dht22", qos=1, retain=True)
 
-client.publish(mqtt_topic + "env/pin", dht22mqtt_pin, qos=1, retain=True)
-client.publish(mqtt_topic + "env/brokeraddr", mqtt_brokeraddr, qos=1, retain=True)
-client.publish(mqtt_topic + "env/refresh", dht22mqtt_refresh, qos=1, retain=True)
-client.publish(mqtt_topic + "env/logging", dht22mqtt_logging_mode, qos=1, retain=True)
-client.publish(mqtt_topic + "env/mqtt_chatter", dht22mqtt_mqtt_chatter, qos=1, retain=True)
+    client.publish(mqtt_topic + "env/pin", dht22mqtt_pin, qos=1, retain=True)
+    client.publish(mqtt_topic + "env/brokeraddr", mqtt_brokeraddr, qos=1, retain=True)
+    client.publish(mqtt_topic + "env/refresh", dht22mqtt_refresh, qos=1, retain=True)
+    client.publish(mqtt_topic + "env/logging", dht22mqtt_logging_mode, qos=1, retain=True)
+    client.publish(mqtt_topic + "env/mqtt_chatter", dht22mqtt_mqtt_chatter, qos=1, retain=True)
 
-client.publish(mqtt_topic + "sys/dht22_stack_size", dht22_stack_size, qos=1, retain=True)
-client.publish(mqtt_topic + "sys/dht22_std_deviation", dht22_std_deviation, qos=1, retain=True)
-client.publish(mqtt_topic + "sys/dht22_error_count_stack_flush", dht22_error_count_stack_flush, qos=1, retain=True)
+    client.publish(mqtt_topic + "sys/dht22_stack_size", dht22_stack_size, qos=1, retain=True)
+    client.publish(mqtt_topic + "sys/dht22_std_deviation", dht22_std_deviation, qos=1, retain=True)
+    client.publish(mqtt_topic + "sys/dht22_error_count_stack_flush", dht22_error_count_stack_flush, qos=1, retain=True)
 
-client.publish(mqtt_topic + "updated", str(datetime.now()), qos=1, retain=True)
+    client.publish(mqtt_topic + "updated", str(datetime.now()), qos=1, retain=True)
 
-log2stdout(datetime.now().timestamp(), 'Setup mqtt client success...')
+    log2stdout(datetime.now().timestamp(), 'Setup mqtt client success...')
 
-client.publish(mqtt_topic + "state", "ONLINE", qos=1, retain=True)
+    client.publish(mqtt_topic + "state", "ONLINE", qos=1, retain=True)
 
-if('essential' in dht22mqtt_mqtt_chatter and mqtt_homeassistant == 'True'):
-    ha_temperature_config = '{"device_class": "temperature", "name": "'+mqtt_device_id+'_temperature", "state_topic": "'+mqtt_topic+ \
-                            'value", "unit_of_measurement": "°C", "value_template": "{{ value_json.temperature}}" }'
-    ha_humidity_config =    '{"device_class": "humidity", "name": "'+mqtt_device_id+'_humidity", "state_topic": "'+mqtt_topic+ \
-                            'value", "unit_of_measurement": "%", "value_template": "{{ value_json.humidity}}" }'
-    client.publish('homeassistant/sensor/'+mqtt_device_id+'Temperature/config', ha_temperature_config, qos=1, retain=True)
-    client.publish('homeassistant/sensor/'+mqtt_device_id+'Humidity/config', ha_humidity_config, qos=1, retain=True)
-    log2stdout(datetime.now().timestamp(), 'Registering sensor with home assistant success...')
+    registerWithHomeAssitant()
 
 log2stdout(datetime.now().timestamp(), 'Begin capture...')
 
+
 while True:
     try:
         dht22_ts = datetime.now().timestamp()
         temperature = getTemperature(dhtDevice.temperature)
         humidity = getHumidity(dhtDevice.humidity)
 
-        temp_data = processSensorValue(dht22_temp_stack, 
-                                       dht22_temp_stack_errors, 
+        temp_data = processSensorValue(dht22_temp_stack,
+                                       dht22_temp_stack_errors,
                                        temperature,
                                        'temperature')
         dht22_temp_stack = temp_data[0]
         dht22_temp_stack_errors = temp_data[1]
         temperature_outlier = temp_data[2]
 
-        hum_data = processSensorValue(dht22_hum_stack, 
-                                      dht22_hum_stack_errors, 
+        hum_data = processSensorValue(dht22_hum_stack,
+                                      dht22_hum_stack_errors,
                                       humidity,
                                       'humidity')
         dht22_hum_stack = hum_data[0]
         dht22_hum_stack_errors = hum_data[1]
         humidity_outlier = hum_data[2]
-        
-        #Since the intuition here is that errors in humidity and temperature are moderately correlated,
-        #so let's skip mqtt-ing if we detect an outlier in either of them. Otherwise mqtt it.
-        #explicitly do a boolean comparison, because not None is actually True
+
+        # Since the intuition here is that errors in humidity and temperature readings
+        # are heavily correlated, we can skip mqtt if we detect either.
         detected = ''
-        if(temperature_outlier == False and humidity_outlier == False):
+        if(temperature_outlier is False and humidity_outlier is False):
             detected = 'accurate'
-            
         else:
             detected = 'outlier'
-            
-        updateEssentialMqtt(temperature,humidity,detected)
+
+        updateEssentialMqtt(temperature, humidity, detected)
         updateFullSysInternalsMqtt()
         updateFullSensorTallyMqtt(detected)
 
@@ -247,11 +274,11 @@ while True:
         time.sleep(dht22mqtt_refresh)
 
     except RuntimeError as error:
-        #DHT22 throws errors often. Does not mean it's not working.
+        # DHT22 throws errors often. Keep reading.
         detected = 'error'
-        updateEssentialMqtt(None,None,detected)
+        updateEssentialMqtt(None, None, detected)
         updateFullSensorTallyMqtt(error.args[0])
-        
+
         data = {'timestamp': dht22_ts, 'error_type': error.args[0]}
         log2stdout(dht22_ts, data)
         log2file('error', data)
@@ -260,12 +287,14 @@ while True:
         continue
 
     except Exception as error:
-        client.disconnect()
+        if('essential' in dht22mqtt_mqtt_chatter):
+            client.disconnect()
         dhtDevice.exit()
         raise error
 
-#Graceful exit
-client.publish(mqtt_topic + "state", "OFFLINE", qos=2, retain=True)
-client.publish(mqtt_topic + "updated", str(datetime.now()), qos=2, retain=True)
-client.disconnect()
-dhtDevice.exit()
+# Graceful exit
+if('essential' in dht22mqtt_mqtt_chatter):
+    client.publish(mqtt_topic + "state", "OFFLINE", qos=2, retain=True)
+    client.publish(mqtt_topic + "updated", str(datetime.now()), qos=2, retain=True)
+    client.disconnect()
+dhtDevice.exit()

dht22mqtt_visualize.py

@@ -19,42 +19,46 @@ dht22_error_count_stack_flush = 3
 
 dht22mqtt_temp_unit = 'C'
 
+
 ###############
 # Polling & Processing functions
 ###############
 def getTemperatureJitter(temperature):
-    return getTemperature(temperature-0.3),getTemperature(temperature+0.3)
+    return getTemperature(temperature-0.3), getTemperature(temperature+0.3)
+
 
 def getTemperature(temperature):
     if(dht22mqtt_temp_unit == 'F'):
         temperature = temperature * (9 / 5) + 32
     return temperature
 
+
 def getHumidity(humidity):
     return humidity
 
+
 def processSensorValue(stack, error, value, value_type):
-    #flush stack on accumulation of errors
+    # flush stack on accumulation of errors
     if(error >= dht22_error_count_stack_flush):
         stack = []
         error = 0
-    
-    #init stack
+
+    # init stack
     if(len(stack) <= dht22_error_count_stack_flush):
         if(value not in stack):
             stack.append(value)
-        #use jitter for bootstrap temperature stack
+        # use jitter for bootstrap temperature stack
         if(value_type == 'temperature'):
             low, high = getTemperatureJitter(value)
             stack.append(low)
             stack.append(high)
         return stack, error, None
-    
-    #get statistics
+
+    # get statistics
     std = statistics.pstdev(stack)
     mean = statistics.mean(stack)
 
-    #compute if outlier or not
+    # compute if outlier or not
     if(mean-std*dht22_std_deviation < value < mean+std*dht22_std_deviation):
         outlier = False
         if(value not in stack):
@@ -63,21 +67,23 @@ def processSensorValue(stack, error, value, value_type):
     else:
         outlier = True
         error += 1
-    
-    #remove oldest element from stack
+
+    # remove oldest element from stack
     if(len(stack) > 10):
         stack.pop(0)
     return stack, error, outlier
 
+
 ###############
 # Dataset processing
 ###############
-def timestampToSeconds(timestamp_begin, timestamp):    
+def timestampToSeconds(timestamp_begin, timestamp):
     b = datetime.fromtimestamp(timestamp_begin/1000)
     e = datetime.fromtimestamp(timestamp/1000)
     return (e-b).total_seconds()
 
-def generatePlots(dataset,data_type):
+
+def generatePlots(dataset, data_type):
     plot_rows = 3
     plot_columns = 4
     reduce_rate = 1
@@ -87,17 +93,15 @@ def generatePlots(dataset,data_type):
             freq = dataset['timestamp'].mean()/len(temp_dataset.index)
             print('generating plot at frequency s='+str(freq)+'...')
             temp_dataset = processDataset(temp_dataset)
-            axes[r,c].set_title(data_type + ' at sampling frequency '+str(round(freq,2))+' (s)')
-            sns.scatterplot(ax=axes[r, c], data=temp_dataset, x='timestamp', y=data_type,hue='type',s=10)
-            #visualize resets
+            axes[r, c].set_title(data_type + ' at sampling frequency '+str(round(freq, 2))+' (s)')
+            sns.scatterplot(ax=axes[r, c], data=temp_dataset, x='timestamp', y=data_type, hue='type', s=10)
+            # visualize stack flushes
             resets = temp_dataset[temp_dataset['reset'] == 'True']
-            #xposition = [0.3, 0.4, 0.45]
-            for key,row in resets.iterrows():
-                plt.axvline(x=row['timestamp'], color='k', alpha=1, linewidth=0.3)#, linestyle='--'
-            
-            
+            for key, row in resets.iterrows():
+                plt.axvline(x=row['timestamp'], color='k', alpha=1, linewidth=0.3)
             reduce_rate += 1
 
+
 def processDataset(dataset):
     dht22_temp_stack = []
     dht22_temp_stack_errors = 0
@@ -105,56 +109,57 @@ def processDataset(dataset):
     dht22_hum_stack_errors = 0
     dataset.loc[:, 'type'] = ''
     dataset.loc[:, 'reset'] = ''
-    
-    for key,row in dataset.iterrows():
+
+    for key, row in dataset.iterrows():
         temperature = row['temperature']
         humidity = row['humidity']
-        
-        temp_data = processSensorValue(dht22_temp_stack, 
-                                      dht22_temp_stack_errors, 
-                                      temperature,
-                                      'temperature')
+
+        temp_data = processSensorValue(dht22_temp_stack,
+                                       dht22_temp_stack_errors,
+                                       temperature,
+                                       'temperature')
         dht22_temp_stack = temp_data[0]
         dht22_temp_stack_errors = temp_data[1]
         temperature_outlier = temp_data[2]
-    
-        hum_data = processSensorValue(dht22_hum_stack, 
-                                      dht22_hum_stack_errors, 
+
+        hum_data = processSensorValue(dht22_hum_stack,
+                                      dht22_hum_stack_errors,
                                       humidity,
                                       'humidity')
         dht22_hum_stack = hum_data[0]
         dht22_hum_stack_errors = hum_data[1]
         humidity_outlier = hum_data[2]
-        
+
         dataset.at[key, 'temperature_outlier'] = temperature_outlier
         dataset.at[key, 'humidity_outlier'] = humidity_outlier
-        
+
+        # record outlier detection source
         if(temperature_outlier and humidity_outlier):
-            dataset.at[key,'type'] = 'both outlier'
+            dataset.at[key, 'type'] = 'both outlier'
         elif(temperature_outlier):
-            dataset.at[key,'type'] = 'temperature outlier'
+            dataset.at[key, 'type'] = 'temperature outlier'
         elif(humidity_outlier):
-            dataset.at[key,'type'] = 'humidity outlier'
+            dataset.at[key, 'type'] = 'humidity outlier'
         else:
-            dataset.at[key,'type'] = 'accurate'
-        
+            dataset.at[key, 'type'] = 'accurate'
+        # record reset pivots
         if(dht22_temp_stack_errors >= 3):
-            dataset.at[key,'reset'] = 'True'
+            dataset.at[key, 'reset'] = 'True'
         if(dht22_hum_stack_errors >= 3):
-            dataset.at[key,'reset'] = 'True'
+            dataset.at[key, 'reset'] = 'True'
 
     return dataset
-        
+
 
 dataset_dir = 'datasets/'
 plots_dir = 'plots/'
 dataset = pd.read_csv(dataset_dir+'dataset.csv')
-dataset['timestamp'] = np.vectorize(timestampToSeconds)(dataset['timestamp'][0],dataset['timestamp'])
+dataset['timestamp'] = np.vectorize(timestampToSeconds)(dataset['timestamp'][0], dataset['timestamp'])
 print('formatted timestamps into seconds...')
-fig, axes = plt.subplots(3, 4, figsize=(50,25))
-generatePlots(dataset,'temperature')
+fig, axes = plt.subplots(3, 4, figsize=(50, 25))
+generatePlots(dataset, 'temperature')
 plt.savefig(plots_dir+'temperature.png')
 plt.clf()
-fig, axes = plt.subplots(3, 4, sharex=True, figsize=(50,25))
-generatePlots(dataset,'humidity')
-plt.savefig(plots_dir+'humidity.png')
+fig, axes = plt.subplots(3, 4, sharex=True, figsize=(50, 25))
+generatePlots(dataset, 'humidity')
+plt.savefig(plots_dir+'humidity.png')

docker-compose.yml

@@ -0,0 +1,19 @@
+version: "3.8"
+services:
+  dht22mqtt:
+    build: .
+    container_name: dht22mqtt
+    devices:
+      - /dev/gpiomem:/dev/gpiomem
+    environment:
+      - topic=zigbee2mqtt
+      - device_id=dht22
+      - broker=192.168.1.10
+      - pin=4
+      - poll=2
+      - device_type=dht22
+      - unit=C
+      - mqtt_chatter='essential|ha|full'
+      - logging=log2stdout|log2file
+    volumes:
+      - ~/dht22mqttlog:/log

gpiomapping.py

@@ -0,0 +1,30 @@
+#!/usr/bin/python3
+import board
+gpiomapping = {
+    2: board.D2,
+    3: board.D3,
+    4: board.D4,
+    5: board.D5,
+    6: board.D6,
+    7: board.D7,
+    8: board.D8,
+    9: board.D9,
+    10: board.D10,
+    11: board.D11,
+    12: board.D12,
+    13: board.D13,
+    14: board.D14,
+    15: board.D15,
+    16: board.D16,
+    17: board.D17,
+    18: board.D18,
+    19: board.D19,
+    20: board.D20,
+    21: board.D21,
+    22: board.D22,
+    23: board.D23,
+    24: board.D24,
+    25: board.D25,
+    26: board.D26,
+    27: board.D27
+}