Python talks to Arduino

This blog post gives an overview of how I created a simple python app for sending data to the firebase database from Arduino. I used Arduino, an array of Sensors, Raspberry Pi and firebase realtime Database. I have included links to the Github repository of this project.

First up, let me explain what the eventual goal was. The goal was to set up a series of low-cost climate/environment modules that capture various types of data like temperature, humidity and various pollutants and gases in the environment. Then take all this data and put it in the cloud where we can eventually build out dashboards, alerts and more. Also, We will be able to show real-time data of a particular place.

This blog post explains how we build one single setup where we create a system comprising an Arduino, an array of sensors, a Python application that can read the data from the Arduino and post that data to the cloud. To automate all the process, We used Raspberry PI booted with Linux(Raspbian) where we will execute a few bash scripts to automate the whole process.

After trying various cloud storage API(s), We didn’t find any convenient for our uses. The main challenge that we were facing was to send data from Arduino to cloud. Either the APIs documentation was not good or the SDK itself.

The Hardware Setup

We used the following:

  • Arduino microcontroller
  • An array of sensors for collecting various parameters.
  • We will have the Raspberry Pi that interfaces with the Arduino to read and transmit off the values but to validate things, for now, the Arduino was powered via Battery source. PI is also powered by same with Linux on it. The communication between the Arduino and the Microprocessor is via serial port communication

Of course, we used a breadboard to connect all this together but I am not going to explain the circuitry here. Our main aim is to show how we used python to get data from Arduino and store it at firebase using python SDK provided by google firebase and how we can automate all the process.

Our data from Arduino looks like this on the output screen

[Add Image]

14:49:00 32, 34, 67, 88.9, 90.5, 33.0
14:49:01 32, 34, 76, 88.7, 91.2, 33.4
14:49:02 33, 34, 78, 88.9, 91.7, 34.5
14:49:03 34, 34, 76, 88.9, 91.8, 34.6

Once this happens, we know that the Arduino setup is looking good and all we need to do now is to write a client program on the PC that interfaces with this Arduino, read the values via the Serial port and then push them to cloud.

Reading data from Arduino using Python

We used pySerial module to encapsulates the access for the serial port. It provides backends for Python running on all major OS. The module named “serial” automatically selects the appropriate backend. You can install it using pip or easy_install.

pip install pyserial

Now you need to figure out which Serial Port on your machine is interfaced to the Arduino. In our case it was ‘/dev/ttyS1’.

class ArdClient:
    port = '/dev/ttyS1'
    baudrate = 9600
    timeout = 0.1

    def __init__(self, port=port, baudrate=baudrate, timeout=timeout):
        self.port = port
        self.baudrate = baudrate
        self.timeout = timeout
  
   def startSerial(self):
        try:
            self.arduino = serial.Serial(self.port, self.baudrate, self.timeout)
            return True
        except:
            return False
        else:
            return False

To check if we are connected to open serial port.

ac = ArdClient('/dev/ttyS1')
if startSerial():
    print("Connected")
else:
    print("Something Wrong")

Once, We get connected to open port, it’s time to read data

def readData(self):
        start = 0
        st = []        
        while True:
            
            # Ignore the last two character             
            data = self.arduino.readline()[:-2]
            if data:
                #Ignore first two lines
                if start <= 2:
                    start += 1
                    pass
            else:
                st.append(data.decode('ASCII'))
                send2firebase(filterData(st))
                st = []

 

Sending to Firebase RealTime Database

Once we get the data, we will send those to firebase cloud database using python SDK provided. To make things easier, Firebase has good documentation available for SDKs that you can use to ease your setup.

The Firebase Admin Python SDK is available via pip.

$ sudo pip install firebase-admin

Setting up firebase realtime database

  1. Setup python admin SDK and initialise the SDK following the steps below

    You can follow the official documentation for  adding and initialising the SDK

    https://firebase.google.com/docs/admin/setup

     

  2. Integrate the Firebase Realtime Database SDKs

    The Firebase Realtime Database is a cloud-hosted database. Data is stored as JSON and synchronized in realtime to every connected client.

    import firebase_admin
    from firebase_admin import credentials
    from firebase_admin import db
    
    # Fetch the service account key JSON file contents
    cred = credentials.Certificate('path/to/serviceAccountKey.json')
    
    # Initialize the app with a service account, granting admin privileges
    firebase_admin.initialize_app(cred, {
        'databaseURL': 'https://databaseName.firebaseio.com'
    })
    
  3. Create Realtime Database References.
    # As an admin, the app has access to read and write all data, 
    # regradless of Security Rules
    
    ref = db.reference()
    
    print(ref.get())
  4. Set Data and Listen for Changes
  5. # Initialise data to 0
    
    def __init__(self, data):
        self.root = fa.createdb(data)
    
    def setRoot(self):
        self.root.set({
        'SensorA': 00,
        'SensorB' : 00,
        'SensorC': 00,
        'SensorD' : 00
        })
  6. Updating Saved Data
def updateData(self, datalist):
    try:
         self.root.update({
            'SensorA': float(datalist[0]),
            'SensorB': float(datalist[1]),
            'SensorC': float(datalist[2]),

            'SensorD': float(datalist[3])
         })
         print("Data Updated")
   except IOError:
       print('Error! Something went wrong.')
   time.sleep(20)

Checking our data

Assuming that all the hardware is wired up and you have your Python code ready, executing the code is straightforward. A sample run of the Python code is shown below:

~ python src/main.py

The final step was to validate if our data was being transmitted successfully and into Firebase.

Visit the Dashboard for your Firebase Database and you should start seeing some entries over there i.e. the Sensor data.

Github Link

The project is hosted at Github

https://github.com/iKshitiz/Arduino-Firebase-Pi-Serial-Client

Advertisements

Python’s SymPy module is really cool

So I was just browsing some code, and I came across a cool module I’d never seen before: SymPy

Basically, SymPy is a Python library for symbolic mathematics. It aims to become a full-featured computer algebra system (CAS).

What is symbolic mathematics?

Symbolic computation deals with the computation of mathematical objects symbolically. This means that the mathematical objects are represented exactly, not approximately, and mathematical expressions with unevaluated variables are left in symbolic form. Symbolic computation is handling non-numerical values, this means symbols like in algebra. Variables are defined as

In simple word, “Variables are defined as Symbols in Symbolic Computation instead of defining variables as numerical values ”

This will be more clear from an example from SymPy official documentation.

Let us define a symbolic expression, representing the mathematical expression x+2xy+2y.

>>> from sympy import symbols
>>> x, y = symbols('x y')
>>> expr = x + 2*y
>>> expr
x + 2*y

Instead of evaluating to something by convention, the expression remains as just, x+2*y

>>> x*expr
x*(x + 2*y)

Here, we might have expected x(x+2y) to transform into x^2+2xy, but instead, we see that the expression was left alone. This is a common theme in SymPy.

 

The Power of Symbolic Computation

The real power of a symbolic computation system such as SymPy is the ability to do all sorts of computations symbolically.

SymPy can simplify expressions, compute derivatives, integrals, and limits, solve equations, work with matrices, and much, much more, and do it all symbolically. It includes modules for plotting, printing (like 2D pretty printed output of math formulas,), code generation, physics, statistics, combinatorics, number theory, geometry, logic, and more.

Examples from official SymPy tutorial

Solve x^2 – 2 = 0

>>> solve(x**2 - 2, x)
[-√2, √2
Compute  sin(x2)d
>>> integrate(sin(x**2), (x, -oo, oo))
√2⋅√π
─────
  2

Installation

To install SymPy run:

sudo pip install SymPy

If you already have Anaconda and want to update SymPy to the latest version, use:

conda update sympy

After installation, it is best to verify that your freshly-installed SymPy works. To do this, start up Python and import the SymPy libraries:

$ python
>>> from sympy import *

From here, execute some simple SymPy statements like the ones below:

>>> x = Symbol('x')
>>> limit(sin(x)/x, x, 0)
1
>>> integrate(1/x, x)
log(x)
 I am looking forward to using this library for mathematical computation for my IoT projects.

Currently, I am going through the tutorial and documentation to get familiar with using the software.

References:

Github: Introduction to contribution

Tutorial: http://docs.sympy.org/latest/tutorial/index.html

Documentation: http://docs.sympy.org/

Download: Releases 

 

Writing a Simple terminal based Dice Game in Python

Last week, We were given a task and that was to show a project (Game, app or script), where we had to use tools that we have learned so far.

I was busy this weekend shifting my hostel and I forgot to complete the task. So I decided to write a simple Dice Game.

Coming to the code of the game. This is a terminal based game without using any graphics. You will roll a die and If Six comes, You will win the Game :). I used random module to generate random numbers. When user will roll the die, randit function generates a random number between 1,6. You will have a better luck if you get a Six.

Since writing code was not only part of the task, but to use tools taught during the session. So I used following tools for my development

  • Vim – Code Editor
  • Git
  • Github
  • Markdown
  • Python

I created Python Small Projects repo on GitHub and pushed code there.

I will write more scripts or silly games to that repo as I am practicing Python these days. I will also add python scripts, that I wrote a few days back and already using.

Please Look at Code and Repo and If find error and improvement, Create and Send a pull request.

Github Repo Link: https://github.com/iKshitiz/Python_Small_Projects

Switched to Fedora as my Main OS & It’s going Great!

After having used Windows as the Main operating system for about the past two years of Engineering, I’m finally switched to Fedora as my main OS. So far the Fedora experience has been going excellent.

What is Fedora?

Fedora is a Unix-like operating system based on the Linux kernel and GNU programs (a Linux distribution), developed by the community-supported Fedora Project and sponsored by the Red Hat company.

Basically, Fedora is a popular open source Linux-based operating system.

How I came to know about Fedora?

I saw fedora OS for the first time in my college lab during ‘Basic concepts of operating systems like UNIX, MS DOS class’. I was not familiar with Linux at that time. Later, when I joined dgplug, They introduced us to Linux and suggested to use Fedora for all work during the session.

Switching to Fedora finally

After using fedora for more than a year, I was impressed by its simplicity, flexibility and easy to use considering myself as a newbie to Linux. It offers pure GNOME with clean Desktop.

I think Fedora is best OS if you want to move to Linux from Windows. It is easy to use and Desktop Environment provides smooth and is easy for users, especially for those who are moving from windows OS. Because It’s not easy to switch directly to the terminal based environment.

I still use Windows for gaming and some other’s work. I am trying to find alternatives to those app or software in Fedora. My machine is running on dual boot currently, Fedora 26 and Windows 10.

I’ve run Fedora for last one years with a consistently reliable experience, and I look forward to what the next one brings. My most used Linux applications are the GNOME Terminal, Mozilla Firefox, Google Chrome, Mozilla Thunderbird, Gedit, and ViM.

I will write more about Fedora in my next posts. 🙂

Looking forward to exploring more “How does a computer works”

 

Who are hackers and What is Hacking ?

I was reading “Free as in Freedom” suggested by Kushal on the #dgplug channel on IRC. We also had a session on “History of Free Software Movement” on Monday”. I bookmarked the page at that time and thought to give a read in free time.

After reading the book, I was surprised and disgraced at the same time that we have no knowledge of the history of this computer science world and even we never tried to know.

So,  I will start with short Introduction of the book. This book is “biographical snapshots of GNU project founder Richard Stallman with the political, social and economic history of the free software movement. This book examines one man’s 20-year attempt to codify and communicate the ethics of 1970s era “hacking” culture in such a way that later generations might easily share and build upon the knowledge of their computing forebears. The book documents Stallman’s personal evolution from teenage misfit to the prescient adult hacker to political leader and examines how that evolution has shaped the free software movement. ”

‘Hacker’, Whenever we read or find out this term, the image that comes to our head is “a person who secretly gets access to a computer system in order to get information, cause damage”. This is what we use to see in movies or media. This is definition by most of the dictionary we follow. But that’s not the fact.

According to Richard Stallman hack means “Playful cleverness.” Hacking meant playful brilliance.

Hackers amaze people with their intelligence and innovation. They gave life to novel ideas which people thought were impossible to do; to wrought into reality.

To understand the meaning of the word “hacker,” and to understand the hacker ethic culture, one should read this book.

This book beautifully explains how the terms cracking’,security breaking’ and prank’ mixed up and create a misunderstanding.  How the word ‘Hacking’ got the new definition as the time changed. That’s why writer of the book finished his writing with these lines

Using the term “cracking” rather than “hacking,” when you mean “security breaking,” shows respect for Stallman and all the hackers and helps preserve something which all computer users have benefited from: the hacker spirit.

After reading the book I can say “I am proud to be a hacker”

References:

  1. Kushal Das’s Article on “Hacker Ethic and Free software movement”
  2. Free as in Freedom

Free software movement & Hacker culture

I will start this post with a quote by Kushal Das in between the session yesterday.

“The history is important, Learn about it!”

We had a session on IRC channel dgplug last night and the topic was “History of hacking and free software movement”.

The session started with this simple recent tweet by Gnome and a few replies

In this session, Kushal gave an inspiring talk on this topic, He told us about

  • TX-O computer at MIT lab,
  • How ‘Hacker Ethic’ word came into existence
  • The rise of Free Software Movement.
  • PDP – I computer
  • Richard Stallman
  • Founding FSF
  • Launching GNU
  • Open Source Initiative (OSI) by Bruce Perens and Eric S. Raymond.

Whenever we use Linux, or an open source software, Everything that we see today in Open source world, It all started with Free Software movement, which is about the freedom of users.

He also suggested us some books and resources to read about “History of hacking and free software movement”

  1. Free as in Freedom
  2. Hackers: Heroes of Computer Revolution
  3. A documentary “Revolution OS”

You can check IRC logs of dgplug here.

Dotfiles: Customize your system

I heard this term first during dgplug Summer training in 2016 when sayan was taking a class on vim. Currently, I am setting up all of my dotfiles for a Fedora configuration. (Read more about Fedora here )

What are dotfiles?

Dotfiles are plain text configuration files on Unix-like systems for things like our shell in.bash_profile,  our editor in .vimrc,.gitignore,  and many others. They are called “dotfiles” as they typically are named with a leading . making them hidden files on your system.

Why Dotfiles?

Dotfiles are used to customize your system. these are not regular documents, and by default are hidden in directory listings.

Dotfiles basically contain the preferred setup of your computer. They usually come with a setup procedure so you can easily install everything again when you need to start from a fresh system. dotfiles also really useful for syncing preferences across multiple devices.

Configuration for my Fedora machine

I set up dotfiles for my vim configuration, bash profile and git configuration, these are the tools(packages) that I use regularly. I am currently using limited data package of Airtel prepaid connection, which is too costly to afford.  I will upload all my dotfiles to GitHub repository next month when I will have a broadband connection.