Serial UART connection RPI zero / Arduino

You can connect the RPI and Arduino is several ways, eg. Serial/UART, I2C or USB. With I2C you can connect several device on the same wire. 1:1 communication is now suffiecient so I will go for the Serial/UART connection.

The RPI operates at 3.3V and the arduino I have chosen on 5V. Therefor we should use a Logic Level Converter (LLC) for serial/UART communication. If you are going for I2C please use the I2C converter.

Wiring

There are 12 total pins on the LLC – two parallel rows of six headers. One row contains all of the high voltage (e.g. 5V) inputs and outputs, the other row has all things low voltage (e.g. 3.3V).

The voltage supplied to the HV and GND inputs should be higher than that supplied to the LV side. For example, if you’re interfacing from 5V to 3.3V, the voltage on the HV pin should be 5V, and the voltage on LV sould be 3.3V.

There are four separate data channels on the BD-LLC, each capable of shifting data to and from high and low voltages. These pins are labeled HV1, LV1, HV2, LV2, HV3, LV3, HV4, and LV4. The number at the end of each label designates the channel of the pin, and the HV or LV prefix determines whether it’s on the high or low side of the channel.

A low-voltage signal sent in to LV1, for example, will be shifted up to the higher voltage and sent out HV1. Something sent in HV3 will be shifted down and sent out of LV3. Use as many of these channels as your project requires.

The RPI is the low level side and the Arduino on the high level side. Please note that for the GPIO the RPI uses 3.3V, but still it has pins to provide 5V. I use the RPI 5V pins to provide power to the Arduino Nano board. We can also use the other RPI 5V pin to provide the HV side for the Arduino. We also need to twist the TX/RX, check this cross on the LLC board (blue/yellow crossing).

RPI config
The RPI is not yet configured to use the serial for communication. Please perform the following steps:
You can use “raspi-config” utility to enable the UART:
1. sudo raspi-config
2. Under “Interfacing Options”, choose “Serial”.
3. Say “No” when it asks if you want a login shell over serial.
4. Say “Yes” when asked if you want the hardware enabled.
5. Finish, then accept the offer to reboot.

After this configuration change and the wiring you are ready to check if the serial communication is ready to use. First you can check if ttyS0 is available on the RPI side with the command ls -l /dev/ttyS0, you should see the following output: crw-rw—- 1 root dialout 4, 64 Mar 22 19:16 /dev/ttyS0

The fastest way to check the communication is to load the following program into your arduino:

int x = 0; // variable

void setup() {
Serial.begin(9600); // open the serial port at 9600 bps:
}

void loop() {
// print labels
Serial.print("NO FORMAT"); // prints a label
Serial.print("\t"); // prints a tab

Serial.print("DEC");
Serial.print("\t");

Serial.print("HEX");
Serial.print("\t");

Serial.print("OCT");
Serial.print("\t");

Serial.print("BIN");
Serial.print("\t");

for(x=0; x< 64; x++){ // only part of the ASCII chart, change to suit

// print it out in many formats:
Serial.print(x); // print as an ASCII-encoded decimal - same as "DEC"
Serial.print("\t"); // prints a tab

Serial.print(x, DEC); // print as an ASCII-encoded decimal
Serial.print("\t"); // prints a tab

Serial.print(x, HEX); // print as an ASCII-encoded hexadecimal
Serial.print("\t"); // prints a tab

Serial.print(x, OCT); // print as an ASCII-encoded octal
Serial.print("\t"); // prints a tab

Serial.println(x, BIN); // print as an ASCII-encoded binary
// then adds the carriage return with "println"
delay(200); // delay 200 milliseconds
}
Serial.println(""); // prints another carriage return
}

This arduino program will print out a list of dec, hex and binair numbers on the serial interface of the arduino.

Now enable the RPI side with:
sudo apt-get install minicom
sudo minicom -b 9600 -o -D /dev/ttyS0

To exit minicom: Hit enter, then ctrl-a then q then press “enter”

RPI installation without monitor, keyboard(Headless) and finalizing the config

I just bought a RPI zero and lacking a micro HDMI. Any way when installing a RPI2 or 3 I still do not want to connect a keyboard and monitor. Please perform below steps to make an installation without monitor and keyboard (headless). We will enable SSH login (disabled with newer raspbian images), configure the Wifi and finalize the installation. In the last steps I will expand the file system and add additional security measures, e.g. removing the PI user and setting up a firewall. So please download and write the raspbian image to your SD card.

Enabling Wifi and SSH access is done by writing a file to /boot/ of the SD card.

Enable Wifi: If a wpa_supplicant.conf file is placed into the /boot/ directory, this will be moved to the /etc/wpa_supplicant/ directory the next time the system is booted, overwriting the network settings.
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=your_ISO-3166-1_two-letter_country_code

network={
ssid="your_SSID"
psk="your_password"
key_mgmt=WPA-PSK
}

Location and filename: /boot/wpa_supplicant.conf

Replace your SSID, password and ISO-3166 in the config file. Your_ISO-3166-1_two-letter_country_code is your ISO Country Code (such as NL for Netherlands).

[Problem] Wifi not working with RPI3 B+: With the RPI zero above process worked, with the RPI3 B+ I run into problems with the wifi. I executed: sudo raspi-config, under localisation options selected change wifi country and then sudo rfkill unblock wifi. Now my wifi worked. [/Problem]

Enable SSH Raspbian will look for a file named ssh on the boot partition at startup and will enable sshd, which by default is disabled. So please create an empty file “ssh”

Location and filename: /boot/ssh

Expand file system We do this to make use of all the space present on the SD card as a full partition. All this does is, expand the OS to fit the whole space on the SD card which can then be used as the storage memory for the RPI.

sudo raspi-config
Option 7:Advanced options
A1: Expand file system

Update firmware It’s always good practice to run the latest version of the firmware which also included the latest bug fixes.

sudo rpi-update

Add user I do not want to continue to use the PI user for security reasons, that’s why we should create a new user with sudo rights.

sudo adduser --gecos "" replace_with_username
sudo usermod -aG sudo replace_with_username

–gecos: Set the gecos field for the new entry generated. Adduser will not ask for finger information if this option is given.

-aG add in group
Now it would be nice if only this user could login via SSH and that we disable root login:
sudo nano /etc/ssh/sshd_config
Add, edit, or append to the end of the file the following line, which contains the usernames you wish to allow to log in
AllowUsers add_with_username
PermitRootLogin no
After the change you will need to restart the sshd service using sudo systemctl restart ssh or reboot so the changes take effect.

Disable Pi user login Everybody knows that on a RPI the user PI is available, so for security reasons I would like to delete this PI user. Please login with your new created user.

sudo deluser pi
sudo passwd root

Change hostname You can change the hostname of the RPI (default: raspberry) by editing the hostname file:  sudo nano /etc/hostname

Firewall We still want to block access to the RPI on any ports that are not explicitly enabled.

apt-get install -y ufw
sudo ufw allow ssh
sudo ufw --force enable
sudo ufw status

Now your RPI is ready to rock!

.local domain fails

This post is meant for users who can do a dns query on a local dns record but cannot access this name. E.g. dig is giving you the IP address of the host, but you cannot ping or SSH to it (and it’s not firewall related)

Recently I’ve changed my network setup and moved away from my router as DHCP/DNS server. I installed pi-hole and I’m quite satisfied.

I have several RPI’s and other material (Yún and linkit7688) running and I want to reach them via a host name. You can add the host name in “etc/hosts” file on the RPI-hole (192.168.x.x [name] [name].[local-domain name]). You can ping or ssh from different kinds of OS, but not from Ubuntu.

This is caused mdns (multicast dns, for auto configuration of the .local domain).

Check /etc/nsswitch.conf, and you will see:

hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4
mdns4 is what is doing multicast dns.

If you change this to, you should be able to reach the host:
hosts: files dns

The you have two options:
1.) Remove mdns permanently with sudo apt-get remove libnss-mdns
2.) don’t use .local – use .lan or something instead

I’ve chosen for option 2, due to the fact that I want Ubuntu to work “out of the box” and in my opinion it’s better to make a server side change then updating all the clients of the network.

Easy gauge chart with C3JS

When you produce data (in the field of industrial automation or IoT) you need to be able to visualize your data. I started out with the google charts library. It is really easy to use, but then I found out that you can only use this when connected to the internet. This could cause a potential problem when my customers are experiencing problems with their internet connection and you cannot show their data.

I was always charmed by D3 but compared to google charts it has a bigger learning curve. Then I found C3. C3 makes it easy to generate D3-based charts by wrapping the code required to construct the entire chart. We don’t need to write D3 code any more :-).

Below shows a gauge chart which displays values instead of the ratio (%). The default example is working with %, this actually does the trick to convert to values: format: function(value) { return value; }

<!DOCTYPE html>
<html>
<head>
	<link href="/static/c3.css" rel="stylesheet" type="text/css">
	<script charset="utf-8" src="static/d3.v3.min.js">
	</script>
	<script src="/static/c3.min.js">
	</script>
	<title></title>
</head>
<body>
	<div id="chart"></div>
	<script>
	   var chart = c3.generate({
	   data: {
	       columns: [
	           ['data', 91.4]
	       ],
	       type: 'gauge',
	       onclick: function (d, i) { console.log("onclick", d, i); },
	       onmouseover: function (d, i) { console.log("onmouseover", d, i); },
	       onmouseout: function (d, i) { console.log("onmouseout", d, i); }
	   },
	   gauge: {
	       label: {
	           format: function(value) { return value; },
	//            show: false // to turn off the min/max labels.
	       },
	//    min: 0, // 0 is default, //can handle negative min e.g. vacuum / voltage / current flow / rate of change
	   max: 300, // 100 is default
	   units: ' V',
	//    width: 39 // for adjusting arc thickness
	   },
	   color: {
	       pattern: ['#FF0000', '#F97600', '#F6C600', '#60B044'], // the three color levels for the percentage values.
	       threshold: {
	           unit: 'value', // percentage is default
	//            max: 200, // 100 is default
	           values: [30, 60, 200, 199]
	       }
	   },
	   size: {
	       height: 90
	   }
	});

	setTimeout(function () {
	   chart.load({
	       columns: [['data', 10]]
	   });
	}, 1000);

	setTimeout(function () {
	   chart.load({
	       columns: [['data', 50]]
	   });
	}, 2000);

	setTimeout(function () {
	   chart.load({
	       columns: [['data', 70]]
	   });
	}, 3000);

	setTimeout(function () {
	   chart.load({
	       columns: [['data', 299]]
	   });
	}, 4000);


	</script> 
</body>
</html>

None blocking bi serial communication between a microcontroller and SoC (e.g. RPI/Linkit/Yun)

If the yun bridge is not suffiecient for your, due to the fact that you would like to use bi directional communcation, you could disable the yun bridge and use the serial1 communcation between the MCU and SoC part of the Yun/Linkit. Or you want to setup a serial connection between an arduino and RPI, please see this post for the wiring.

For the software side you should install pyserial with: pip install pyserial

Please use the following code for bidirectional and none blocking communication.

The python part:

import serial
import time

s = serial.Serial("/dev/ttyS0", 57600, timeout=0.1)
s.write("Hello")
arduinodata = s.readline()

while (True):
    if (s.inWaiting()>0): #if incoming bytes are waiting to be read from$
                arduinodata = s.readline()
                print(arduinodata)

The C code:

String incoming_data;

void setup() {
  // put your setup code here, to run once:
  Serial.begin(9600); 
  Serial1.begin(57600);
}

void loop() {
  // put your main code here, to run repeatedly:
  incoming_data  = Serial1.readStringUntil('\n');
  
  if(incoming_data!="") { 
    
    if(incoming_data=="Hello") {
      Serial1.println("World");
      Serial.println("World");
    }
  }
  Serial1.println("Writing in none blocking mode");
  delay(50);
  
}

I would like to thank Michael Kirsch for above code.

Linkit 7688 older IPK installation

Sometimes their is an application missing in a new openwrt release which was present in an older version. Luckily you can still download the older packages. You only need to find the right architecture to now where to download from.

https://downloads.openwrt.org/ choice your release, eg. barrier_breaker/14.07/ and then the Linkit MPU architecture ramips/mt7620a/packages/
You can download it via wget and then opkg install [filename].

Arduino yun MQTT Tutorial

In this tutorial we will dive into the use of MQTT with the arduino yun (or linkit) to control the LED#13.

One of the common protocols used in the IOT world is MQTT. MQTT is a machine-to-machine (M2M)/”Internet of Things” connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is available.

MQTT
The MQTT protocol is based on the principle of publishing messages and subscribing to topics, or “pub/sub”. Multiple clients connect to a broker and subscribe to topics that they are interested in. Clients also connect to the broker and publish messages to topics. Many clients may subscribe to the same topics and do with the information as they please. The broker and MQTT act as a simple, common interface for everything to connect to.

The setup
1.) MQTT broker running on a RPI
2.) Python script on the arduino yun (openwrt)
3.) Sketch for the MCU

MQTT broker
To install the MQTT broker:

apt-get update
apt-get install mosquitto

To test it open two SSH connections. One for subscribing and the other for publishing.

mosquitto_sub -h 192.168.2.40 -t "arduino/led"
mosquitto_pub -h 192.168.2.40 -t "arduino/led" -m "1"
mosquitto_pub -h 192.168.2.40 -t "arduino/led" -m "0"

Result:

1
0

For debugging purposes you can also run mosquitto in a shell. So open up the third SSH connection. Execute “/etc/init.d/mosquitto stop” and then execute “mosquitto”. Output:

1468435883: mosquitto version 1.3.4 (build date 2014-08-22 06:10:51+0000) starting
1468435883: Using default config.
1468435883: Opening ipv4 listen socket on port 1883.
1468435883: Opening ipv6 listen socket on port 1883.
1468435883: New connection from 192.168.x.x on port 1883.

Python
Install the MQTT python client:
pip install paho-mqtt

And then use the following python code:


#!/usr/bin/python
import sys
import paho.mqtt.client as mqtt

sys.path.insert(0, '/usr/lib/python2.7/bridge/')

from bridgeclient import BridgeClient as bridgeclient

from tcp import TCPJSONClient
json = TCPJSONClient('127.0.0.1', 5700)

value = bridgeclient()


# The callback for when the client receives a CONNACK response from the server.
def on_connect(client, userdata, rc):
    print("Connected with result code "+str(rc))
        # Subscribing in on_connect() means that if we lose the connection and
        # reconnect then subscriptions will be renewed.
    client.subscribe("arduino/led")

# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
        print(msg.topic+" "+str(msg.payload))
        json.send({'command':'put', 'key':'led', 'value':msg.payload})

client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message

client.connect("192.168.x.x", 1883, 60)

# Blocking call that processes network traffic, dispatches callbacks and
# handles reconnecting.
# Other loop*() functions are available that give a threaded interface and a
# manual interface.
client.loop_forever()

Normally I would use value.put() instead of using json.send() only I received a “Bad file descriptor” once in a while. As the arduino yun uses JSON via TCP to the bridge library I used this workaround instead.

Sketch
Upload the sketch. memset will set al the items in the array to zero.

#include <Bridge.h>
#include <stdio.h>
 
// Here we will hold the values coming from Python via Bridge.
char ledvalue[2];
 
void setup() {
  // Zero out the memory we're using for the Bridge.
  memset(ledvalue, 0, 2);
   
  // Initialize digital pins 13 as output.
  pinMode(13, OUTPUT); 
 
  // Start using the Bridge.
  Bridge.begin();
}
 
void loop() {
  
  
  // Write current value of D13 to the pin (basically turning it on or off).
  Bridge.get("led", ledvalue, 2);
  int ledD13 = atoi(ledvalue);
  digitalWrite(13, ledD13);
   
  delay(10);  
}

Please note that the delay for retreiving info from the bridge is rather low, causing heavy load on the linux side.

Update [29-12-2017]:
If you receive the following error: TypeError: on_connect() takes exactly 3 arguments (4 given) the following could help you. Paho-mqtt client has been updated and you need to change the following (1. on_connect and subscribe):

def on_connect(client, userdata, rc):
print(“Connected with result code “+str(rc))
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
client.subscribe(“arduino/led”)

Should be changed to:
def on_connect(self, client, userdata, rc):
print(“Connected with result code “+str(rc))
# Subscribing in on_connect() means that if we lose the connection and
# reconnect then subscriptions will be renewed.
self.subscribe(“arduino/led”)

Arduino yun bridge Tutorial

The Yún has two processors on board. One is an ATmega32U4 like on the Arduino Leonardo (we call it MCU from now on). The other is an Atheros 9331 (we call it MPU from now on), running Linux and the OpenWRT wireless stack, which enables the board to connect to WiFi and Ethernet networks. This tutorial also works for the linkit 7688 Duo. To run it on a linkit, please enable the arduino yun bridge software. You can find it in my older posts.

The bridge has quite some options available, for me there are two use case who are practical for me:
1.) Sending sensor data from the MCU to a server
2.) Bi-directional communication between a server and the MCU

Sending sensor data from the MCU to a server
Say for example that you have a sensor which can measure the temperature and partical concentration and you want to upload it to a restfull api webserver. In this case you are only sending data from the MCU to a server. Easiest solution is to use the “runShellCommand”. As the functionname tells us, it runs a command in the shell.

There are two flavours, runShellCommandAsynchronously() and runShellCommand(). runShellCommand() is a blocking function. That is, once you call Process.runShellCommand(), nothing else will happen in your sketch until it has completed. The time depends on the nature of the command you are executing. For a non-blocking alternative, please use runShellCommandAsynchronously().

Now we have a solution to run from your sketch a shell command. Now we have access to the shell, we need a tool to upload our data to the server. We can use Curl for this. Curl is an open source command line tool and library for transferring data with URL syntax. Eg. you can download a webpage, or upload via json data to a restfull api.

#include <Process.h>

void setup(void) {
  Bridge.begin();
}

void loop(void) {
  String curlCmd;

  String resultConcentration;
  String resultTemperature;
 
  resultConcentration=5.5;
  resultTemperature=20.0;
  
  curlCmd = "curl -H \"Content-Type: application/json\" -X POST -d \'{\"concentration\": "+resultConcentration+", \"temperature\": "+resultTemperature+"}\' http://192.168.x.xx:8080/measurement";

  Process database;
  database.runShellCommand(curlCmd);
  delay(1000);
}

You need to include for the bridge communication. “Bridge.begin()” starts Bridge, facilitating communication between the MCU and MPU. This should be called once in setup(). begin() is a blocking function. Once you call Bridge.begin(), nothing else will happen in your sketch until it has completed. This process takes approximately three seconds.
Process is the base class for all Bridge based calls for communicating with the Yun’s shell. Followed by run() or runAsynchronously(). The named process does not start executing until run() is called.

Bi-directional communication between a server and the MCU
For bi-directional communication you need to have a program/script running on the MPU. This will have a connection to a server (Eg. MQTT broler) and read/writes to the bridge. On the MCU you will need to have a sketch which can read/write to the bridge.

Sketch
The put() function allows you to store data on the MPU using a Key/Value structure. The Key field is like a label and you can associate a value to it. The key name must be unique in order to identify the correct value. On the MPU side there is a data store where all the keys and the values are saved.

The datastore is saved in the RAM of the MPU (AR9331), you will lose the datastore when you restart the bridge software on the Linux side (through power cycling, resetting the Linux processor, or uploading a sketch through WiFi or Ethernet). You will not lose the datastore if you reset the MCU (ATMega32u4 processor).

get() allows you to read a key/value item previously saved on the MPU. You can request for a value stored in the datastore by passing get() the Key you want to search for, the support buffer, and its size. The Key is similar to a label, used to identify an associated value. The key name must be unique in order to identify the correct value.

#include <Bridge.h>

unsigned long timer;
unsigned long counter = 0L;

void setup()
{
    Bridge.begin();     // this launches /usr/bin/run-bride on Linino
    timer = millis();
    Serial.begin(9600);
}

void loop()
{
    /* Every 200ms: */
    char bridge_Value[6];
    
    if (millis() - timer > 200) {
        timer = millis();

        Bridge.put("counter", String(counter++));
        Bridge.put("random1", String(random(1, 100)));
        Serial.println(counter);
        Bridge.get("hello", bridge_Value, 6);
        Serial.println(bridge_Value);
    }
}

Python script
For the MPU I’ve chosen to use a python script. You need to import the bridge library. The code is rather self explaining. You can also access the data via the API of the arduino Yun. In our case you can use http://192.168.x.xx/data/get/counter to read the value of counter, or write to counter via http://192.168.x.xx/data/put/counter/4

#!/usr/bin/python
import time
import sys

sys.path.insert(0, '/usr/lib/python2.7/bridge/')

from bridgeclient import BridgeClient as bridgeclient

value = bridgeclient()


while True:
        message = value.get("counter")
        print message
        value.put('hello','world')
        time.sleep(1)
        value.put('hello','you')

The sketch code is based on code from Jan-Piet Mens. The python from alnitak1000. I combined both the bridge.put()/bridge.get() and value.put()/value.get() together.