Resetting an SD card after writing a Linux Image from Windows

After writing the Rasbian image to my SD card for a new Raspberry Pi project I found myself wanting to reset the SD card to be used as normal storage so I could install NOOBS, but I got stuck when I couldn't work out how to delete the partitions made by writing the image on the device from Windows.

I found a Windows solution in the way of DISKPART that got the job done.

How to use DISKPART

1. Open a new command window by either searching for "cmd" from the windows menu or pressing the windows key and 'r' and running the command "cmd" and hit enter
2. Enter the command "DISKPART" and another command window will be opened
3. In the new window type "LIST DISK" and hit enter
4. Select the correct disk from the list with the command "SELECT DISK #" replacing the # with the drive number of the SD card. Make sure you get this right or you could ruin your windows installation!
5. Enter the command "CLEAN" to clear the partition table
6. Enter the command "CREATE PARTITION PRIMARY" to reallocate the cleaned space as the primary partition
7. Enter the command "FORMAT FS=FAT32 QUICK" to reformat the SD card

Optionally you can use the command "ASSIGN" to reassign a drive letter to the SD card.

HOME AUTOMATION: ELK/NESS M1 Home Alarm Controller with OpenHAB over Serial

When I installed the NESS M1 home automation controller and home alarm I had plans of integrating it in to my overall setup. I purchased an M1XEP with plans of exposing the alarm control to the network but ran in to setup issues that I was never able to resolve.

When I installed my openHAB controller I looked for a binding for the NESS/ELK but was not able to find one. There are some active conversations on the forums but no available bindings, so I decided to try my hand at getting something going. I'm using a USB to Serial adaptor plugged from the serial port of my home alarm directly in to my Raspberry Pi running openHAB2.

OpenHAB Serial Binding

First thing to do is to install and configure the serial binding. I have already covered this in an older post.

Reading Zone States

The easiest thing to start with is to read the zone state from the NESS alarm controller. With serial configured this is simply a matter of reading the serial input data and interpreting the commands based on the well documented serial protocol for the controller.

Here are the files I have used to mirror the state of the motion sensors on my home alarm by reading the zone states:

alarm.items

 Switch AlarmUpdate  "Alarm Update"  { serial="/dev/ttyUSB0@115200" }  
 String AlarmMessage "Alarm Message" { serial="/dev/ttyUSB0@115200" }  
 Group gAlarm_PIR
 Contact Alarm_PIR_Zone1                                                (gAlarm_PIR)
 Contact Alarm_PIR_Zone2                                                (gAlarm_PIR)
 Contact Alarm_PIR_FF_MasterBedroom "Master Bedroom Motion Sensor [%s]" (gAlarm_PIR)
 Contact Alarm_PIR_Zone4                                                (gAlarm_PIR)
 Contact Alarm_PIR_Nursery "Nursery Motion Sensor [%s]"                 (gAlarm_PIR)
 Contact Alarm_PIR_FF_Stairs "Stairs Motion Sensor [%s]"                (gAlarm_PIR)
 Contact Alarm_PIR_GF_Lounge "Lounge Motion Sensor [%s]"                (gAlarm_PIR)
 Contact Alarm_PIR_GF_Stairs "Basement Stairs Motion Sensor [%s]"       (gAlarm_PIR)

init.rules

 rule "Init"  
 when  
     System started  
 then  
     // let openHAB settle and give other rules the chance to fire  
     createTimer(now.plusSeconds(180)) [|  
         gAlarm_PIR?.members.forEach[g |  
             g.postUpdate(CLOSED)  
         ]  
 end  

alarm.rules

 import java.lang.Integer.*  
 val AlarmMemoryMessage = "AM"  
 val ZoneChangedMessage = "ZC"  
 val NetworkAdapterPing = "XK"  
 val ZoneStatusTable_NormalUnconfigured = "0"  
 val ZoneStatusTable_NormalOpen = "1"  
 val ZoneStatusTable_NormalEOL = "2"  
 val ZoneStatusTable_NormalShort = "3"  
 val ZoneStatusTable_TroubleOpen = "5"  
 val ZoneStatusTable_TroubleEOL = "6"  
 val ZoneStatusTable_TroubleShort = "7"  
 val ZoneStatusTable_ViolatedOpen = "9"  
 val ZoneStatusTable_ViolatedEOL = "A"  
 val ZoneStatusTable_ViolatedShort = "B"  
 val ZoneStatusTable_BypassedOpen = "D"  
 val ZoneStatusTable_BypassedEOL = "E"  
 val ZoneStatusTable_BypassedShort = "F"  
 rule "Interpret Alarm Command"  
 when  
     Item AlarmMessage received update  
 then  
     var lines = AlarmMessage.state.toString.split('\n')  
     lines.forEach[line |  
         if(line.length > 6) {  
             var messageLength = line.substring(0,2)  
             var commandType = line.substring(2,4)  
             var checksum = line.substring(line.length-3, line.length-1)  
             //logInfo("alarm", "Message:" + line + " Len:" + messageLength + " Command:" + commandType)  
             if(commandType == AlarmMemoryMessage) {  
                 var alarmMemoryAreas = line.substring(4, 12)  
             }  
             else if(commandType == ZoneChangedMessage) {  
                 else if(commandType == ZoneChangedMessage) {  
                 var int zoneNumber = Integer::parseInt(line.substring(4,7))  
                 var zoneStatus = line.substring(7,8)  
                 //logInfo("alarm", "Zone:" + zoneNumber + " Status:" + zoneStatus)  
                 //logInfo("alarm", "gAlarm_PIR:" + gAlarm_PIR.members.filter(g | g.name == "Alarm_PIR_GF_Lounge").toString)  
                 var zone = Alarm_PIR_GF_Lounge  
                 var state = CLOSED  
                 // case statement setting the zone item from the message received  
                 switch zoneNumber {  
                     case zoneNumber == 1 : zone = Alarm_PIR_Zone1  
                     case zoneNumber == 2 : zone = Alarm_PIR_Zone2  
                     case zoneNumber == 3 : zone = Alarm_PIR_FF_MasterBedroom  
                     case zoneNumber == 4 : zone = Alarm_PIR_Zone4  
                     case zoneNumber == 5 : zone = Alarm_PIR_FF_Nursery  
                     case zoneNumber == 6 : zone = Alarm_PIR_FF_Stairs  
                     case zoneNumber == 7 : zone = Alarm_PIR_GF_Lounge  
                     case zoneNumber == 8 : zone = Alarm_PIR_GF_Stairs  
                 }  
                 // set the state of the zone from the message received  
                 if(zoneStatus == ZoneStatusTable_NormalUnconfigured) {  
                     state = CLOSED  
                 }  
                 else if(zoneStatus == ZoneStatusTable_NormalOpen) {  
                     state = CLOSED  
                 }  
                 else if(zoneStatus == ZoneStatusTable_NormalEOL) {  
                     state = CLOSED  
                 }  
                 else if(zoneStatus == ZoneStatusTable_TroubleOpen) {  
                     state = OPEN  
                 }  
                 else if(zoneStatus == ZoneStatusTable_TroubleEOL) {  
                     state = OPEN  
                 }  
                 else if(zoneStatus == ZoneStatusTable_TroubleShort) {  
                     state = OPEN  
                 }  
                 else if(zoneStatus == ZoneStatusTable_ViolatedOpen) {\  
                     state = OPEN  
                 }  
                 else if(zoneStatus == ZoneStatusTable_ViolatedEOL) {  
                     state = OPEN  
                 }  
                 else if(zoneStatus == ZoneStatusTable_ViolatedShort) {  
                     state = OPEN  
                 }  
                 else {  
                     state = CLOSED  
                 }  
                 postUpdate(zone, state)  
             }  
             else if(commandType == NetworkAdapterPing) {  
             }  
             else {  
                 //logInfo("alarm", "No code match")  
             }  
         }  
     ]  
 end  

Add the following lines to your sitemap:

 Frame label="Ground Floor Stairs" {  
   Text item=Alarm_PIR_GF_Stairs icon="motion"  
 }  

Testing

View the sitemap and we should see the motion sensor updates coming through

Test the sitemap is working as expected with the command:

 tail -f /var/log/openhab2/openhab.log -f /var/log/openhab2/events.log tail -f /var/log/openhab2/openhab.log -f /var/log/openhab2/events.log  

If everything is working as expected you should see updates coming through:

 ==> /var/log/openhab2/events.log <==  
 2017-04-03 13:44:24.818 [GroupItemStateChangedEvent] - gAlarm_PIR changed from CLOSED to UNDEF through Alarm_PIR_GF_Lounge  
 2017-04-03 13:44:28.191 [ItemStateChangedEvent   ] - AlarmMessage changed from 0AZC007900C2  
 1EAS000000000111111100000000000F  
 0CAM000000007F  
  to 0AZC007200C9  
 1EAS000000001111111100000000000E  
 0CAM000000007F  
 2017-04-03 13:46:26.579 [ItemStateChangedEvent   ] - Alarm_PIR_GF_Lounge changed from OPEN to CLOSED  

Update

There is a bug in openHAB2 that stops icon updates on the sitemap as the states are updated. It's documented here. The solution is to restart the OpenHAB server.

HOME AUTOMATION: OpenHAB2 Serial Binding on Raspberry Pi

I have always wanted to enable my home automation setup with per-room presence detection. With this in mind I purchased and installed an ELK/NESS home alarm with automation capability.

I haven't been able to find any decent OpenHAB binding for the alarm, and after recently giving up on getting my M1XEP serial to network box for my alarm working I've decided to try my hand at rolling my own serial solution. Here I'll discuss how I got the serial configuration up and running.

Installation

Install the binding from the configuration GUI

I'm using a USB to Serial adapter, but other serial devices should also show up as serial devices if they have been recognised by the kernel. You can check this with the command:

 dmesg  

Or to view messages relating to the serial port configuration use the command:

 dmesg | grep tty

This should tell you what port was assigned to your USB to Serial adapter, in my case ttyUSB0

Check which users have access to the serial port with the command:

 ll /dev/ttyUSB0  

Output should look something like:

 crw-rw---- 1 root dialout 188, 0 Mar 27 10:03 /dev/ttyUSB0  

You may need to add the default user to the dialout group:

 sudo adduser openhabian dialout  

It's a good time to check the serial connection to make sure it's working as expected. You can use the GNU screen command to view the serial port from the command line:

 screen /dev/ttyUSB0 115200  

The screen command won't exit with the typical commands like CTRL-X or CRTL-Z, to exit the screen command enter 'CTRL-a' then press '\', then hit 'Y' to exit the screen.

Now that we have the binding installed and confirmed the serial port is working as expected we can continue with the OpenHAB setup

OpenHAB2 Configuration 

I created and 'alarm.items' file to store alarm related configuration info:

 Switch AlarmUpdate "Alarm Update" { serial="/dev/ttyUSB0@115200" }  
 String AlarmMessage "Alarm Message" { serial="/dev/ttyUSB0@115200" }  

and a 'alarm.rules' file with the following:

 ule "Interpret Alarm Command"  
 when  
     Item AlarmMessage received update  
 then  
     logDebug("alarm", "Alarm message updated")  
 end  
   

Check everything is working with the command:

 tail -f /var/log/openhab2/openhab.log -f /var/log/openhab2/events.log  

And you should see log messages like the following:

 2017-03-27 12:08:27.946 [ItemStateChangedEvent   ] - AlarmMessage changed from 0AZC007200C9  
 1EAS000000001111111100000000000E  
 0CAM000000007F  
  to 16XK2708212270117011006C  

Now that the serial binding is working we can move on to interpreting the data!

HOME AUTOMATION: OpenHAB2 on the Raspberry Pi 3

After spending a bunch of time messing with the Pine64 getting it up and running and installing OpenHAB2, I accidentally unplugged power while the device was on and it caused my installation to no longer boot :-( This is not the first time some minor error has cause my device to stop booting but it's going to be the last, I'm moving to a Rapsberry Pi 3.

While the Raspberry Pi is not immune to this type of issue, I already use a Raspberry Pi 1 and have lost power while the system is up over 100 times and I've only ever corrupted one SD card over the last four years. I realise this is not something I can guarantee, but the Pine64 has been nothing but a hassle since it was delivered.

Installing OpenHAB2

Instead of going through the pain of a manual install and configuration of OpenHAB like I had to with my Pine64 I've decided to go with OpenHABian. This image is supposed to be a self configuring installation of OpenHAB on the Raspberry Pi.

As usual I didn't bother to read the instructions and so decided that something must be wrong with the installation when it didn't respond to SSH connection request after I turned it on. I found out the hard way that it takes around 45 minutes for first boot! Not what I was expecting. Once I let it run through the first boot procedure I was able to SSH in with:
    username: openhabian
    password: openhabian

Once the image has booted for the first time I ran the script:

 sudo openhabian-config  

From here I installed all of the packages required for OpenHAB, including OpenHAB2!

OpenHABian on the RPi 3

Bluetooth

By default the bluetooth does not work on the OpenHABian minimal installation, if you try to scan for bluetooth adapters you will get errors like "no default adapter".

I was able to enable the bluetooth on the Raspberry Pi 3 doing the following:

 openhabian@openHABianPi:~$ sudo apt-get install bluetooth bluez blueman  
 openhabian@openHABianPi:~$ sudo reboot  
 openhabian@openHABianPi:~$ hcitool dev  
 Devices:  
     hci0  B8:27:EB:15:64:58  

HOME AUTOMATION: OpenHAB2 on the Pine64

I "bought" a Pine64 Pro with a WiFi/Bluetooth adapter on Kickstarter that I had originally planned to use as the home automation server to replace my RPi 1, but I was never really committed to the installation as the early builds for the Pine64 were pretty unstable.

Things have developed and become more stable since then and with the recent release of OpenHAB2 it seemed like the right time to jump back in.

Installation

I tried a few different builds to get the Bluetooth adapter working with no luck, there seem to be a lot of people complaining about the lack of support for the Bluetooth device on the forums. Because of that I decided to just stick with the most stable build of Linux for the Pine64, which at the time of writing is the Ubuntu image, and use a USB Bluetooth adapter if I needed to integrate Bluetooth in to my OpenHAB installation.

It's worth noting that there is an OpenHAB image for the Pine64 but I was never able to get this to boot so I abandoned it and started with a Ubuntu image.

I downloaded and unzipped the image using 7zip and used Win32 Disk Imager to write the image to the SD card.

I plugged my device into the internet via ethernet for first boot so I could SSH in without having to set up the WiFi network.
Login to your router and find the IP address of your Pine64. Alternatively you could use an app for your phone like Fing Network Discovery Tool to find the device IP address.

I use Putty to SSH in to my devices from my Windows machine. It's a nice little tool that allows a few different connection types including SSH and Serial and ability to save sessions for frequently used devices.
While there is no location for a username you can add one to the saved session by typing the username in to the host name box as follows:

 ubuntu@192.168.1.151  

PuTTY Saved Session

The default password for the Ubuntu image is 'ubuntu'.


Reboot your device with the command:

 sudo shutdown -r now  

Install Java with the following commands:

 echo "deb http://ppa.launchpad.net/webupd8team/java/ubuntu xenial main" | sudo tee /etc/apt/sources.list.d/webupd8team-java.list  
 echo "deb-src http://ppa.launchpad.net/webupd8team/java/ubuntu xenial main" | sudo tee -a /etc/apt/sources.list.d/webupd8team-java.list  
 sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys EEA14886  
 sudo apt-get update  
 sudo apt-get install oracle-java8-installer oracle-java8-set-default  

Now that Java is installed we can install the OpenHAB2 official release:

 wget -qO - 'https://bintray.com/user/downloadSubjectPublicKey?username=openhab' | sudo apt-key add -  
 echo 'deb http://dl.bintray.com/openhab/apt-repo2 stable main' | sudo tee /etc/apt/sources.list.d/openhab2.list  
 sudo apt-get update  
 sudo apt-get install openhab2  

Start OpenHAB and add it to startup with the following commands:

 sudo systemctl start openhab2.service  
 sudo systemctl status openhab2.service  
 sudo systemctl daemon-reload  
 sudo systemctl enable openhab2.service  

If everything installed and started correctly you should be able to navigate to the page:

 http://YOUR.DEVICE.IP:8080  

From here you should be able to follow the beginners guide to get your device going.

If you get stuck and need to work out what is happening with you OpenHAB installation you can view the log file with this command:

 tail -f /var/log/openhab2/openhab.log -f /var/log/openhab2/events.log  

HOME AUTOMATION: Mounting network storage device on Raspberry Pi

I run a NAS storage device to serve up media content as well as a place to backup photos and videos.
I've found it really handy to be able to connect to my network storage from my Raspberry Pi but it took me a while to work out the right configuration for my setup. Here are the steps I followed:

Raspberry Pi 1 Configuration:

Install the cifs-utils package. This is the package we need to connect the the network drive

 sudo apt-get install cifs-utils  

Make a directory where you would like to mount the network storage:

 sudo mkdir /home/pi/NAS  

We are going to use two files to do the network drive mounting, the first file will store the network drive and mounting information and the second will hold the login credentials information:

1. /etc/fstab - network drive information
2. /root/.servercred - hidden file with network credential information

The cifs-utils package creates a file named fstab that holds the network drive information.
Edit the fstab file and add the network location and password:

 sudo nano /etc/fstab  

To begin with you file should look something like this:

 proc           /proc      proc  defaults         0    0  
 /dev/mmcblk0p1 /boot      vfat  defaults         0    2  
 /dev/mmcblk0p2 /          ext4  defaults,noatime 0    1  
 # a swapfile is not a swap partition, so no using swapon|off from here on, use dphys-swapfile swap[on|off] for that  

Add the following line to the bottom of the fstab file:

 proc           /proc      proc  defaults         0    0  
 /dev/mmcblk0p1 /boot      vfat  defaults         0    2  
 /dev/mmcblk0p2 /          ext4  defaults,noatime 0    1  
 # a swapfile is not a swap partition, so no using swapon|off from here on, use dphys-swapfile swap[on|off] for that  
   
 //192.168.1.100/ /home/pi/NAS cifs credentials=/root/.servercred,sec=ntlmv2,iocharset=utf8,file_mode=0777,dir_mode=0777 0  

Hit ctrl-x to exit nano, then type y and hit enter to save the file.
Now we need to create the server credential information:

 sudo nano /root/.servercred  

Add your username and password to this file (including network homegroup)

 username=KLIMOVSKI\andrew  
 password=1234abcd  

If you get the error "sudo: nano: command not found" the you will need to install nano:

 sudo apt-get install nano  

Finally, attempt to mount the network drive by typing:

 sudo mount -a  

If everything worked the command should execute and you should not see any output. You can now test the network drive by navigating to the directory we created earlier.

Setting up a VPN on Android and Raspberry Pi (Private Internet Access - PIA)

There are a number of reason you might want to get access to a VPN, some legitimate and some a little shady like protecting yourself while downloading torrents. My main reason is that I do a bit of travelling and find myself connecting to random free Wi-Fi services and it's always bothered me that someone is probably intercepting any messages that are unencrypted and using my data (man-in-the-middle attack). It also doesn't hurt to be able to access any websites I want, regardless of and government mandates blocks to ISPs.

I decided to go with Private Internet Access as my VPN provider, they seemed to offer a wide range of functionality and multi-platform support. In my case I was focused on an Android app and easy Raspberry Pi configuration.

Note: I paid for my subscription using the website which charged me in USD. After downloading the app I realised it would have been cheaper to sign up through the app, it was advertised about $5 cheaper than I paid in AUD after conversion.

Android:

1. Download the Android app from the app store
2. Enter your username and password from the email you got when you subscribed
3. Done! It's that easy

Raspberry Pi:

I used this description on the PIA forums to get my Raspberry Pi configured but had to make a couple of small changes to get it to work. It's a bit of a process, but here are the steps I followed:

1. Log in to your device, I use the terminal app Putty for Windows to remote in to my device
2. Make sure you device is up to date
• sudo apt-get update
• sudo apt-get upgrade
3. Install openvpn and the network manager to configure the VPN on the Pi
1. sudo apt-get install network-manager-openvpn openvpn
4. Create a new folder for the PIA configuration files and download the PIA openvpn configurations
• sudo mkdir  openvpn-pia
• cd openvpn-pia
• sudo wget https://www.privateinternetaccess.com/openvpn/openvpn.zip
5. Extract the openvpn zip file
• sudo unzip openvpn.zip
• 
6. Try to connect using one of the default openvpn configurations for your region. You'll be asked for your username and password
• sudo openvpn --config ./AU\ Sydney.ovpn
• enter username and password
7. Once the script is running you should see a number of messages printed to the terminal, one of which should be 'Initialization Sequence Completed'. If this doesn't happen you may have mucked up one of the previous steps.
8. The Raspberry Pi will not terminate the script when it is finished, you'll need to do this yourself by pressing 'CTRL -c' to kill the script
9. Assuming everything went well we will now create a config file to store our username and password
• sudo nano /etc/openvpn/login.conf
10. The file created will be blank, enter only your username and password on separate lines as I have below
• p123456 
• AbCdEF123
11. Press 'CTRL -x' to exit and enter 'y'  and 'Enter' to save the file
12.  Change the file permission with the following command
• sudo chmod 400 /etc/openvpn/login.conf
13. Make a copy of the configuration file we have been previously using in the /etc/openvpn/ folder and change the file extension to .conf
• sudo cp AU\ Sydney.ovpn /etc/openvpn/Sydney.conf
14. Edit the .conf file we copied and add the login.conf file to the file at the line auth-user-pass. It should change from the first point to the second as shown below
• sudo nano /etc/openvpn/Sydney.conf
• auth-user-pass [CHANGE THIS LINE]
• auth-user-pass login.conf [THIS IS HOW IT SHOULD LOOK AFTER CHANGE]
• hit 'CTRL -x' to exit and type 'y' and hit 'Enter' to save
15. You now need to copy the ca.rsa.2048.crtfile and the crl.rsa.2048.pem file to the /etc/openvpn folder. You won't see these files in your folder using the standard 'ls' command, you will need to use 'ls -a' to show hidden files
• sudo cp ca.rsa.2048.crt /etc/openvpn/
• sudo cp crl.rsa.2048.pem /etc/openvpn/
16. Navigate to the openvpn folder and test the configuration file we have edited to make sure it is still working as it was in step 7
• cd /etc/openvpn/
• sudo openvpn Sydney.conf
17. If everything went well you should see the 'Initialization Sequence Complete' again
18. Now we need to add the configuration to the startup sequence so the VPN connection is made on boot. We will need to open /etc/default/openvpn file and add an autostart option
• sudo nano /etc/default/openvpn
• add the line 'AUTOSTART="Sydney" to the autostart commands so it looks like:
• #AUTOSTART="all"
• #AUTOSTART="none"
• #AUTOSTART="home office"
• AUTOSTART="Sydney"
19. Done! Everything should be working, but we should test the setup to make sure everything is working on boot
• sudo reboot
• wget http://ipecho.net/plain -O - -q ; echo
20. If everything worked you should see a different IP to the one your ISP has given you

Happy restriction-free browsing