Shed Cam (updated 1/7/2018)

Got involved with a "grow the biggest sunflower" competition, traditionally not a lot of technology used, but as we all know everything is better when you add some electronics. Requirements:

In the past i thought that all USB web cameras were roughly the same, judging their abilities on resolution i.e. the more pixels the better. This is definitely not true for the variable environment of a shed. Tried four different web cameras, they all had small problems with focusing, but the game breaker was light levels. Cheap and cheerful cameras can't handle too bright or overcast i.e. in all but ideal lighting levels they would white-out or black-out, as shown in figure 1. Tried to auto-correct the white balance / brightness with image-magick, some limited success (Link ), but didn't really solve the problem i.e. GIGO, Garbage In, Garbage Out. In the end used an old PS3 USB camera, gave significantly better performance in variable light levels, figure 2, note the high quality mounting bracket :).

Update: next time you go to the cinema make sure you save your 3D glasses. The polarised 'lens' make a cheap anti-glare filter. I found that most web cameras are better at handling the too dark, rather than the too bright. Cut out the len and mount in front the webcam, i just gaffer taped top and bottom. Did try both lens, rotated on the horizontal (0-90) and vertical (180) axis, to see if i could block all of the light, do confess got some strange effects e.g. a purple tint. Therefore, not sure quite how these are polarised, but a single lens does help in very bright light conditions e.g. early morning when the sun is low in the sky.

Figure 1 : Image quality

Figure 2 : USB camera

To capture a still image i used fswebcam, (Link ) ( Local ). Combined with a little (dodgy) shell script (below) and your normal crontab (Link ) you can capture an image every 2 minutes. Note, remember to put the full path or edit the PATH variable in crontab, otherwise you may not find the required shell script, or the normal command line executables you are expecting. Would of liked to have had a full night vision camera and recorded 24hour, 7 days a week, but lacked the IR illumination. Note, most web cameras will operate in the IR spectrum e.g. if you point a TV remote control at the camera you can see the IR LED illuminating on the display. Therefore, if you have enough IR LEDs and enough electrical power to fully illuminate the scene, you can see in the dark. Will perhaps look into this later (Link )( Local). Otherwise, went with plan B, only take pictures during 'day light' hours, for my camera roughly 3.00AM to 11:00PM.

#!/bin/sh

if test -f /home/pi/Stop-frame/count.txt
then
    COUNT=`cat /home/pi/Stop-frame/count.txt`
else
    echo 0 > /home/pi/Stop-frame/count.txt
    COUNT=0
fi

if test $COUNT -gt 999
then
    FILENAME=pic_$COUNT.jpg
elif test $COUNT -gt 99
then
    FILENAME=pic_0$COUNT.jpg
elif test $COUNT -gt 9
then
    FILENAME=pic_00$COUNT.jpg
else
    FILENAME=pic_000$COUNT.jpg
fi 

#echo $FILENAME

fswebcam -r 640x480 --no-banner /home/pi/Stop-frame/$FILENAME

COUNT=$(($COUNT + 1))
echo $COUNT > /home/pi/Stop-frame/count.txt

Update: made a few small changes to fswebcam, increased the frame rate of the camera (--fps) and skipped a number of captured images (-S). The aim was to see if the camera's auto exposure/brightness controls would give a better image after a few images had been captured, rather than using the first image. Tried increasing the number of images skipped (1 - 100) i.e. allowing the camera to settle down for a few seconds. This seemed to help a little with the cheap cameras, but to be honest didn't see that much improvement for the PS3 camera.

fswebcam -d /dev/video1 --fps 15 -S 8 --no-banner -r 640x480 /home/pi/Stop-frame/$FILENAME

Inserting the '0's into the image file name is not strictly required, but it does simplify video generation i.e. makes sure pictures remain in the correct order. Looked around for a nice way of generating a video from the still images (approx 600). Tried a number of different techniques with varying degrees of success. Finally found mencoder which just worked straight out of the box (Link ) (Link ) ( Local ). The video is automatically generate at midnight using a crontab entry to trigger the shell script below. The key option is the fps i.e. how long do you want the final video to be, i set this to 10 frames per second, making the video about a minute long.

#!/bin/sh

DATE=`date | tr -s " "`
MONTH=`echo $DATE | cut -d' ' -f2`
DAY=`echo $DATE | cut -d' ' -f3`
TIME=`echo $DATE | cut -d' ' -f4`
YEAR=`echo $DATE | cut -d' ' -f6`

#echo $TIME $DAY $MONTH $YEAR

FILENAME="video_$DAY$MONTH$YEAR.mp4"
#echo $FILENAME

cd /home/pi/Stop-frame

mencoder "mf://*.jpg" -mf fps=10 -o $FILENAME -ovc lavc -lavcopts vcodec=mpeg4:vbitrate=4800

Its surprising how quickly you can fill up the Raspberry Pi SD card, each image is approx 200K, each video is approx 35M. In my original version i kept all the old images in a backup directory just in case i needed to regenerate the video, a month later you get a crashed Pi. Once the video is generated the clean up script below is triggered. This moves the pictures and video to a time-stamped folder, resetting the system for the next day. It also timestamps one picture (taken around 11:00AM) so that i can make a time lapse video when we get to the end (Winter). At present i'm manually transferring these backups to a file server every two weeks, will look at automating this later.

#!/bin/sh

DATE=`date`| tr -s " "
MONTH=`echo $DATE | cut -d' ' -f2`
DAY=`echo $DATE | cut -d' ' -f3`
TIME=`echo $DATE | cut -d' ' -f4`
YEAR=`echo $DATE | cut -d' ' -f6`

PICNAME="pic_$DAY$MONTH$YEAR.jpg"
LOGNAME="sensor_log_$DAY$MONTH$YEAR.csv"
DIRNAME="backup_$DAY$MONTH$YEAR"

#echo $FILENAME

cp /home/pi/Stop-frame/pic_0250.jpg /home/pi/Stop-frame/backup/$PICNAME

cd /home/pi/Stop-frame

mkdir /home/pi/Stop-frame/backup/$DIRNAME
mv *.mp4 *.jpg *.csv *.txt  /home/pi/Stop-frame/backup/$DIRNAME

Update: as with all software just when you have everything working the OS goes and changes the libraries it supports. Unfortunately mencoder is no longer supported on the Raspberry Pi. Therefore, had to more over to avconv. To install this software i followed this link (Link), which basic is:

sudo apt-get install mpv libav-tools

Avconv is installed in libav, I'm guessing there are some libs needed in mpv, but all seems to work. To convert the jpg into a mp4 i used:

avconv -i pic_%04d.jpg -s 320x240 -r 25 -vcodec libx264 -crf 20 -g 15 $FILENAME

Went for a smaller (scale -s 320x240), shorter 25 frames per second (rate -r 25) video as i was uploading to Twitter which has a 30sec limit. Obviously you notice the change in speed i.e. approximately twice as fast, but you don't really see the difference in size when played within the browser. The crf and g options not a clue, did have a quick read around something to do with compression

In addition to taking pictures and making videos i wanted a data logger to record the weather in the garden, from a practical point of view as the Raspberry pi is in the shed its easier to record the weather in the shed than the garden, however, the shed acts as a mini green house, so you do need to go to the effort of cabling sensors to the outside. Typical weather sensors available for the Raspberry Pi are:

The BMP085 atmospheric pressure sensor (Link ) and the DHT11 humidity sensor (Link )are easy to interface to the Pi as they use digital serial buses (I2C and adhoc GPIO) i.e. no additional analogue to digital converters needed. Information on how to interface these to the Raspberry Pi using Python programs can be found here, DHT11 (Link ) ( Local ), BMP085 (Link ) ( Local ). Both of these sensors also contain a temperature sensor. To avoid water damage to these components (and Pi) these are mounted in the shed (shown in figure 5), which makes their readings a little off i.e. there is some air flow through the shed, but humidity levels will be lower than outside.

Figure 5: Pressure sensor (left), Humidity sensor (right)

The other sensors use simple potential divider type circuits shown in figure 6, LDR, thermistor and simple contacts. Variable resistors and fixed resistors are mounted on the patch board (figure 7, values vary). These circuits produce an analogue voltage output, therefore, they need to be converted into a digital representation. The Raspberry Pi doesn't have on-bard analogue to digital converters (ADC), however, these can be easily attached to the inter IC communications (I2C) serial bus. An example of the one i used can be found here (Link ), very cheap and also contained light and temperature sensors which can be used for the shed. The ADC used on this board is the PCF8591 (Link ) ( Local ), very simple to use ,this web site has a nice graphical test program written in Python (Link ) ( Local )( Code ). The rain sensor (figure 8) uses another purchased module (Link ), this gives an analogue output (connected to an ADC channel) and a ON/OFF digital output i.e. if the analogue signal crosses a defined voltage threshold (adjusted with a Pot). Long term not the best rain sensor to use, quite a lot of oxidisation on the sensor, requiring the threshold voltage to be constantly adjusted.

Figure 6: Sensor circuits

Figure 7: Bought in modules ADC (left back), Rain sensor (left front), sensor patch board (right)

Figure 8: Rain sensor (front), light & temperature sensors (brass tube, back)

Figure 9: System

Using the previous example Python code i wrote a small program to read all the sensors and print each value to the screen separated by commas. Then using the shell script below i create a comma separated variable (csv) log file i.e. redirected stdout to a file. Again this shell script was called in the crontab every two minutes.

#!/bin/sh

DIR="/home/pi/Sensors"
TIME=`date | tr -s " " | cut -d' ' -f4`

if test -f $DIR/count.txt
then
    COUNT=`cat $DIR/count.txt`
else
    echo 0 > $DIR/count.txt
    echo `/bin/date` > $DIR/sensor-log.csv
    echo `/bin/date` > $DIR/sensor-date-log.csv
    COUNT=0
fi

SENSORS=`python $DIR/read-sensors.py`

echo "$COUNT, $SENSORS" >> $DIR/sensor-log.csv
echo "$TIME, $SENSORS" >> $DIR/sensor-date-log.csv
#echo "$COUNT, $SENSORS"

COUNT=$(($COUNT + 1))
echo $COUNT > $DIR/count.txt

At the end of each day the spreadsheet of sensor values is converted into a graphical plot using GNUplot. This software is a little tricky to use, but once you get your head around the command line syntax its not too bad (well maybe). The different sensor values are plotted in four graphs i.e. two sensors per plot. To maximize the plotted area the data is scaled by defining the expected MAX and MIN sensor values i.e. normal range.

The GNUplot script and shell is given below:

set term png  size 1024, 720
set output 'plot.png'
set multiplot layout 4,1
set key out vert
set key right

# 1 Time index
# 2 BMP Temp
# 3 BMP Pressure
# 4 DHT Temp
# 5 DHT Humidity
# 6 Chan 0 : light
# 7 Chan 1: NU
# 8 Chan 2: temp
# 9 Chan 3: NU
# 10 Chan 4: Rain analogue
# 11 Chan 5: temp
# 12 Chan 6: light
# 13 Chan 7: NU
# 14 Rain digital

set xdata time
set timefmt "%H:%M:%S"
set xrange ["3:0:0":"23:0:0"]
set xtics format "%H:%M"

set title 'Shed Sensors'
set xlabel "Time"
plot 'sensor-log.csv' u 1:2 t 'Temp' w line, 'sensor-log.csv' u 1:3 t 'Pressure' w line
plot 'sensor-log.csv' u 1:6 t 'Light' w line,   'sensor-log.csv' u 1:5 t 'Humidity' w line

set title 'Garden Sensors'
set xlabel "Time"
plot 'sensor-log.csv' u 1:11 t 'Temp' w line, 'sensor-log.csv' u 1:12 t 'Light' w line                                   
plot 'sensor-log.csv' u 1:10 t 'Humidity' w line, 'sensor-log.csv' u 1:14 t 'Rain' w line                                         

unset multiplot

Figure 9: Weather plot

#!/bin/sh

DATE=`date`
MONTH=`echo $DATE | cut -d' ' -f2`
DAY=`echo $DATE | cut -d' ' -f3`
TIME=`echo $DATE | cut -d' ' -f4`
YEAR=`echo $DATE | cut -d' ' -f6`

FILENAME="backup_$DAY$MONTH$YEAR.csv"

cd /home/pi/Gnuplot

cp /home/pi/Sensors/sensor-date-log.csv /home/pi/Gnuplot/sensor-log.csv

gnuplot plot2.gnu
convert plot.png plot.jpg

mv /home/pi/Gnuplot/sensor-log.csv $FILENAME
mv -f /home/pi/Gnuplot/$FILENAME backup

To tweet a daily photo (taken at 11:00AM) and the days sensor plot i used the Twython Python package (Link )(Link )(Link )(Local ). The daily video is uploaded to flickr using the Python code flickr-uploader (Link). Again these programs are trigger during the night by crontab when the Raspberry Pi is not taking photos. Takes quite a long time to upload the video filkr, so to reduce processor load either test to see if this process is finished, or be lazy like me and just schedule it an hour to do the upload.

Update: Twitter uploads have been very reliable, had some problems with Flikr, but i think this was just due to the size of files i was uploading. Anyway, with the updates to Twitter and twython decided to move everything over to Twitter. The Twython package (Link)(Link) is very easy to use allowing you to tweet text, pictures and videos (videos have to be less than 512M and no more than 30sec). An example taken from the documentation is shown below:

import time
from twython import Twython

IMG_WIDTH = "640"
IMG_HEIGHT = "480"
IMG_NAME = "pic.jpg"

# your twitter app keys goes here
apiKey = 'twitter API Key here' 
apiSecret = 'twitter API Secret here' 
accessToken = 'twitter access token here' 
accessTokenSecret = 'twitter access token secret'  

# this is the command to capture the image using pi camera
snapCommand = "raspistill -w " + IMG_WIDTH +  " -h " + IMG_HEIGHT + " -o " + IMG_NAME

api = Twython(apiKey,apiSecret,accessToken,accessTokenSecret)

print "Program running...\n"
photo = open(IMG_NAME, 'rb')

print "Uploading photo to twitter...\n"
media_status = api.upload_media(media=photo)

time_now = time.strftime("%H:%M:%S") # get current time
date_now =  time.strftime("%d/%m/%Y") # get current date

tweet_txt = "Photo captured by @yorksunflowers on " + date_now

print "Posting tweet with picture...\n"
api.update_status(media_ids=[media_status['media_id']], status=tweet_txt)

Final thoughts. The system seems to work ok, had a couple of small problems with my shell script at first, the joys of 'cut' and extra white spaces. Also don't forget to archive photos and videos, its doesn't take long to fill 8G. Things to fix, at present the sensor graphs just show relative changes not absolute values. One of the reasons for this is that i need to calibrate the LDR and Thermistor voltages to light level and temperature value. Temperature should be easy as the Pressure and Humidity sensors have an absolute sensor i.e. put the thermistor in the shed and note the temperature for max and min voltages (assuming it has a linear response). Light level is a little more tricky as i don't have a calibrated light level meter. Finally need to automate the archive of the pics and video, simplest solution will be to upload to Flickr as this has 1T of storage as standard.

Shed Cam: new and improved

Decided to add a few more sensors, you can't have to many pretty graphs. Soil moisture sensors, not sure what units this would be measured in, but an indication of whether the sunflowers need a water. The simplest form of moisture meter is one base on resistance. I know this is not the best idea as again like the rain sensor you get corrosion and electrolysis effects over time, but it is very a simple to build, an easy one to start off with. Used galvanised nails as shown in figure 10, thought this may help with the corrosion issues, filed down to steal then soldered on wires. These two probes + soil form one of the "resistor" elements in a simple potential divider, the other a fixed 250K resistor. The output voltage of this circuit is connected to one of the free ADC channels. I did dig down a bit into the soil about 10cm, to try and get below the "crust", covered with a piece of old guttering so that the sensor would not get directly rained on i.e. so that it measured the moisture of the surrounding soil, rather than surface water. Note, seperation between nails is approximately 2cm, a guess.





Figure 10: Soil moisture probe

Also cleaned up the construction of the hardware a little, moved the components on the breadboard onto veroboard, and added some choc-block to simplify wiring. New hardware construction is shown in figure 11.













Figure 11: New Pi and hardware

The circuit diagram for this new hardware is shown in figure 12. Since last time managed to loose the pressure sensor, so had to swap to a BMP280, found it a bit tricky to find a python library for the Raspberry Pi, eventually found a couple, code is available here: (Link). The fixed resistors for the fence mounted LDR and Thermistor potential dividers were previous implemented as pots, but this was not really needed so this time went for fixed 4K7 ohms. Fixed resistors for soil probes, did a bit of testing, a few 100K seems to work, so went for a 500K variable and a fixed 4K7, set half way i.e. approach 250K ohms. Therefore, can increase/decrease as needed. Power for all modules is now from +5V, as most use open collector type drivers/interfaces i.e. will not drive +5V onto Pi pins. The exception is the rain sensor, so make a cheap and nasty level shifter using a 330 ohm series resistor i.e. uses the GPIO pin's clamping diodes, resistor limits current.

Figure 12: New hardware circuit diagram

Software wise, mostly the same, Twitter has changed its video spec a little, still limited to 512MB, but videos can now be 2 minutes long, therefore reduced the frame rate of the video from 25 fps to 15 fps, which makes each video about a minute long, to be honest 30 seconds was too quick, you missed all the finer details. Also decided to up the resolution to 640x480.

Update 8-6-2018: improved the mounting of the anti-glare filter, 3D printed an enclosure, as shown in figure 13, download link (Link).






Figure 13: Filter

Simple push / friction fit onto the camera, as shown in figure 14.


Figure 14: mounting filter on camera

Creative Commons Licence

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Contact details: email - mike.freeman@york.ac.uk, telephone - 01904 32(5473)

Back