Stephen Smith's Blog

Musings on Machine Learning…

Playing with Julia 1.0 on the Raspberry Pi

leave a comment »

Introduction

A couple of weeks ago I saw the press release about the release of version 1.0 of the Julia programming language and thought I’d check it out. I saw it was available for the Raspberry Pi, so I booted up my Pi and installed it. Julia has been in development since 2012, it was created by four MIT professors as an open source project for mathematical computing.

Why Julia?

Most people doing data science and numerical computing use the Python or R languages. Both of these are open source languages with huge followings. All new machine learning projects need to integrate to these to get anywhere. Both are very productive environments, so why do we need a new one? The main complaint about Python and R is that these are interpreted languages and as a result are very slow when compared to compiled languages like C. They both get around this by supporting large libraries of optimized code written in C, C++, Assembler and Fortran to give highly optimized off the shelf algorithms. These work great, but if one of these doesn’t apply and you need to write Python loops to process a large data set then it can get really frustrating. Another frustration with Python is that it doesn’t have a built in array data type and relies on the numpy and pandas libraries. Between these you can do a lot, but there are holes and strange differences between the two systems.

Julia has a powerful builtin array type and most of the array manipulation features of numpy and pandas are built in to the core language. Further Julia was created from scratch around powerful new just in time (JIT) compiler technology to provide both the speed of development of an interpreted language combined with the speed of a compiled language. You don’t get the full speed of C, but it’s close and a lot better than Python.

The Julia language borrows a lot of features from Python and I find programming in it quite similar. There are tuples, sets, dictionaries and comprehensions. Functions can return multiple values. For loops work very similarly to Python with ranges (using the : built into the language rather than the range() function).

Julia can call C functions directly (meaning you can get pointers to objects), and this allows many wrapper objects to have been created for other systems such as TensorFlow. This is why Julia is very precise about the physical representation of data types and the ability to get a pointer to any data.

Julia uses the end keyword to terminate blocks of code, rather than Pythons forced indentation or C’s semicolons. You can use semicolons to have multiple statements on one line, but don’t need them at the end of a line unless you want it to return null.

Julia has native built in support of most numeric data types including complex numbers and rational numbers. It has types for all the common hardware supported ints and floats. Then it also has arbitrary precision types build around GNU’s bignum library.

There are currently 1906 registered Julia packages and you can see the emphasis on scientific computing, along with machine learning and data science.

The creators of Julia always keep performance at the top of mind. As a result the parallelization support is exceptional along with the ability to run Julia code on CUDA NVidia graphics cards and easily setup clusters.

Is Julia Ready for Prime Time?

As of the time of this writing, the core Julia 1.0 language has been released and looks quite good. Many companies have produced impressive working systems with the 0.x versions of Julia. However right now there are a few problems.

  • Although Julia 1.0 has been released, most of the add on packages haven’t been upgraded to this version yet. In the first release you need to add the Pkg package to add other packages to discourage people using them yet. For instance the library with GPIO support for the Pi is still at version 0.6 and if you add it to 1.0 you get a syntax error in the include file.
  • They have released the binaries for all the versions of Julia, but these haven’t made them into the various package management systems yet. So for instance if you do “sudo apt install julia” on a Raspberry Pi, you still get version 0.6.

Hopefully these problems will be sorted out fairly quickly and are just a result of being too close to the bleeding edge.

I was able to get Julia 1.0 going on my Raspberry Pi by downloading the ARM32 files from Julia’s website and then manually copying them over the 0.6 release. Certainly 1.0 works much better than 0.6 (which segmentation faults pretty much every time you have a syntax error). Hopefully they update Raspbian’s apt repository shortly.

Julia for Machine Learning

There is a TensorFlow.jl wrapper to use Google’s TensorFlow. However the Julia group put out a white paper dissing the TensorFlow approach. Essentially TensorFlow is a separate programming language that you use from another programming language like Python. This results in a lot of duplication and forces the programmer to operate in two different paradigms at once. To solve this problem, Julia has the Flux machine learning system built natively in Julia. This is a fairly powerful machine learning system that is really easy to use, reducing the learning curve to getting working models. Hopefully I’ll write a bit more about Flux in a future article.

Summary

Julia 1.0 looks really promising. I think in a month or so all the add-on packages should be updated to the 1.0 level and all the binaries should make it out to the various package distribution repositories. In the meantime, it’s a good time to learn Julia and you can accomplish a lot with the core language.

I was planning to publish a version of my LED flashing light program in Julia, but with the PiGPIO package not updated to 1.0 yet, this will have to wait for a future article.

 

Advertisements

Written by smist08

August 31, 2018 at 7:34 pm

TensorFlow on the Raspberry Pi and Beyond

leave a comment »

Introduction

You’ve been able to use TensorFlow on a Raspberry Pi for a while, but you’ve had to build it yourself. With TensorFlow 1.9, Google added native support, so you can just use pip3 to install precompiled binaries and be up and running in no time. Although you can do this, general TensorFlow usage on the Raspberry Pi is slow. In this article I’ll talk about some challenges to running TensorFlow on the Raspberry Pi and look at some useful cases that do work. I’ll also compare some operations against my Intel i3 based laptop and the rather beefy servers available through Google’s subsidiary Kaggle.

Installing TensorFlow on a Pi

I saw the press release about how easy it was to install TensorFlow on a Raspberry Pi, so I read the TensorFlow install page for the Pi, checked the prerequisites, and followed the instructions. All I got was strange unhelpful error messages about how there was no package for my version of Python. The claim on the TensorFlow web page is that Python 3.4 or greater is required and I was running 3.4.2, so all should be good. I installed all the prerequisites and dependencies from the TensorFlow script and those all worked, including TensorBoard. But no luck with TensorFlow itself.

After a bit of research, it appeared that the newest version of Raspbian is Stretch, but I was running Jessie. I had assumed that since my operating system was updating that it would have installed any newer version of Raspbian. That turns out to not be true. The Raspberry people were worried about breaking things, so didn’t provide an automatic upgrade path. Their recommendation is to just install a new image on a new SD card. I could have done that, but I found instructions on the web on how to upgrade from Jessie to Stretch. I followed the instructions available here, and it all worked fine.

To me, this is really annoying since I wasted quite a bit of time on this. I don’t understand why Raspbian didn’t at least ask if I wanted to upgrade to Stretch offering the risks and trade-offs. At any rate now I know, not to trust “sudo apt-get dist-upgrade”, it doesn’t necessarily do what it claims.

After I upgraded to Stretch, doing a “sudo pip3 install TensorFlow” worked quickly and I was up and running.

Giving TensorFlow a Run

To try out TensorFlow on my Raspberry Pi, I just copied the first TensorFlow tutorial into IDLE (the Python IDE) and gave it a run.

import tensorflow as tf
mnist = tf.keras.datasets.mnist

(x_train, y_train),(x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(),
  tf.keras.layers.Dense(512, activation=tf.nn.relu),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

model.fit(x_train, y_train, epochs=5)
model.evaluate(x_test, y_test)

This tutorial example trains on the MNINST dataset which is a set of handwritten digits and then evaluates the test set to see how accurate the model is. This little sample typically achieves 98% accuracy in identifying the digits. The dataset has 60,000 images for training and then 10,000 for testing.

I set this running on the Raspberry Pi and it was still running hours later when I went to bed. My laptop ran this same tutorial in just a few minutes. The first time you run the program, it downloads the test data, on the Pi this was very slow. After that it seems to be cached locally.

Benchmarking

To compare performance, I’ll look at a few different factors. The tutorial program really has three parts:

  1. Downloading the training and test data into memory (from the local cache)
  2. Training the model
  3. Evaluating the test data

Then I’ll compare the Raspberry Pi to my laptop and the Kaggle virtual environment, both with and without GPU acceleration.

 

Load Time Fit Time Eval Time
Raspberry Pi 3.6 630 4.7
I3 Laptop 0.6 95 0.5
Kaggle w/o GPU 1.7 68 0.6
Kaggle with GPU 1.1 44 0.6

 

Keep in mind that my Raspberry Pi is only a 3 and not the newer slightly faster 3 B+. The GPU in the Kaggle environment is the NVIDIA Tesla K80. The server is fairly beefy with 16GB of RAM. The Kaggle environment is virtual and shared, so performance does vary depending on how much is going on from other users.

Results

As you can see the Raspberry Pi is very slow fitting a model. The MNINST data is fairly compact as these things go and represents a relatively small data set. If you want to fit a model and only have a Raspberry Pi, I would recommend doing it in a Kaggle environment from an Internet browser. After all it is free.

I think the big problem is that the Raspberry Pi only has 1Gig of RAM and will be swapping to the SD Card which isn’t the greatest in performance. My laptop has 4Gig RAM and a good SSD Hard Drive. I suspect these are more key than comparing the Intel i3 to the ARM Cortex processor.

So why would you want TensorFlow on the Raspberry Pi then? The usage would be to run pre-trained models for specific applications. For instance perhaps you would want to make a smart door camera. The camera could be hooked up to a Raspberry Pi and then a TensorFlow image recognition model could be run to determine if someone approaching the door should be admitted, and if so, send a signal from a GPIO pin to unlock the door.

From above you might think that evaluation is still too slow on a Raspberry Pi. However, x_test which we are evaluating actually contains 10,000 test images. So performing 10,000 image evaluations in under 5 seconds is actually pretty decent.

A good procedure would be to train the models on a more powerful computer or in the cloud, then run the model on the Pi to create some sort of smart device utilizing the Pi’s great I/O capabilities.

Summary

The Raspberry Pi with its great DIY interface abilities combined with its ability to run advanced machine learning AI applications provides a great platform to develop smart devices. I look forward to seeing all sorts of new smart projects appearing on the various Raspberry Pi project boards.

Written by smist08

August 17, 2018 at 12:09 am

Posted in Uncategorized

Erlang on the Raspberry Pi

leave a comment »

Introduction

Erlang is a rather interesting programming language with a number of rather unique properties. If you are used to procedural languages like C or its many variants, then Erlang will appear very different. Erlang was originally developed by Ericsson to program telephone switches. The name is either based on Ericsson Language or is named after Agner Krarup Erlang depending on who you ask. In Erlang there are no loops, all loops are accomplished via recursion. Similarly once a variable is assigned, it can never be changed (is immutable). A lot of the function execution is controlled via pattern matching. Beyond the basic language the Erlang system is used to create highly available, scalable fault tolerant systems. You can even swap code on the fly without stopping the system. Erlang is still used in many telephony type applications, but it is also used to create highly reliable and scalable web sites. The best known being WhatsApp. Plus Facebook, Yahoo and many others have services implemented in Erlang serving massive numbers of users.

In this article we’ll begin to look at the basic language and consider an implementation of our flashing LED program for the Raspberry Pi implemented in pure Erlang. ERLang runs on most operating systems and is open source now. It is easy to add to the Raspberry Pi and is a very good system if you want to make use of a cluster of Raspberry Pis.

How to Run the Program

Erlang doesn’t come pre-installed with Raspbian, but it’s easy to add with the command:

sudo apt-get install erlang

After this completes you are good to go.

Erlang includes a terminal based command system where you can compile modules and call functions. To run the programs included here, you need to save the first file as lights.erl and the second as gpio.erl. Then you can compile and execute the program as indicated in the screenshot of my terminal window below:

Some things to note:

  1. Erlang is case sensitive and case has meaning. For instance variables start with a capital letter and functions with a lowercase letter.
  2. Punctuation is very important. The periods end a statement and in the erl shell will cause it to execute. If you miss the period, nothing will happen until you enter one (it assumes you have more text to enter). Similarly inside functions , and ; have a specific meaning that affects how things run.
  3. You do everything by calling functions both in the shell and in the Erlang language. So the c() function compiles a module (produces a .beam file). The q() function exits the shell after terminating all programs. lights:flashinglights() is our exported entry point function to run the program with. You can also call things like ls() to get a directory listing or cd() to change directories or pwd() to find out where you are. Remember to enter any lines with a period to terminate the line.
  4. To access the gpio /sys files, erl must be run from sudo. You could fix the file system permissions, but this seems easy enough.

Flashing LED Program

Below is my main module to flash the lights. Unlike the C or Fortran version of this program, there is no loop, since loops don’t exist in Erlang. Instead it uses recursion to accomplish the same things. (Actually in the Erlang tutorials there are design patterns on how to accomplish while or for loops with recursion). Notice that once a variable is assigned it can never be changed. But you accomplish the same thing with recursion by passing a modified version of the variable into a function. Similarly you can preserve variables using function closures, but I don’t do that here. I included edoc comments which are Erlang version of JavaDoc. Otherwise this is just intended to give a flavour for the language without going into too much detail.

 

%% @author Stephen Smith
%% @copyright 2018 Stephen Smith
%% @version 1.0.0
%% @doc
%% A erlang implementation of my flashing lights program
%% for the Raspberry Pi.
%% @end

-module(lights).
-export([flashlights/0]).
-author('Stephen Smith').

flashlights() ->
    Leds = init(),
    flash(Leds, 10).

init() ->
    L0 = gpio:init(17, out),
    L1 = gpio:init(27, out),
    L2 = gpio:init(22, out),
    {L0, L1, L2}.

flash(Leds, Times) when Times > 0 ->
    gpio:write(element(1, Leds), 1),
    timer:sleep(200),
    gpio:write(element(1, Leds), 0),
    gpio:write(element(2, Leds), 1),
    timer:sleep(200),
    gpio:write(element(2, Leds), 0),
    gpio:write(element(3, Leds), 1),
    timer:sleep(200),
    gpio:write(element(3, Leds), 0),

    flash(Leds, Times-1);

flash(Leds, Times) when Times =< 0 ->
    true.

Erlang GPIO Library

Rather than write the file access library for the GPIO drivers myself, doing a quick Google search revealed several existing ones including this one from Paolo Oliveira.

 

%% @author Paolo Oliveira <paolo@fisica.ufc.br>
%% @copyright 2015-2016 Paolo Oliveira (license MIT)
%% @version 1.0.0
%% @doc
%% A simple, pure erlang implementation of a module for 
%% <b>Raspberry Pi's General Purpose
%% Input/Output</b> (GPIO), using the standard Linux kernel
%% interface for user-space, sysfs,
%% available at <b>/sys/class/gpio/</b>.
%% @end

-module(gpio).
-export([init/1, init/2, handler/2, read/1, write/2, stop/1]).
-author('Paolo Oliveira <paolo@fisica.ufc.br>').

%% API

% @doc: Initialize a Pin as input or output.
init(Pin, Direction) ->
  Ref = configure(Pin, Direction),
  Pid = spawn(?MODULE, handler, [Ref, Pin]),
  Pid.

% @doc: A shortcut to initialize a Pin as output.
init(Pin) ->
  init(Pin, out).

% @doc: Stop using and release the Pin referenced as file descriptor Ref.
stop(Ref) ->
  Ref ! stop,
  ok.

% @doc: Read from an initialized Pin referenced as the file descriptor Ref.
read(Ref) ->
  Ref ! {recv, self()},
  receive
    Msg ->
      Msg
  end.

% @doc: Write value Val to an initialized Pin referenced as the file descriptor Ref.
write(Ref, Val) ->
  Ref ! {send, Val},
  ok.

%% Internals

configure(Pin, Direction) ->
  DirectionFile = "/sys/class/gpio/gpio" ++ integer_to_list(Pin) ++ "/direction",

  % Export the GPIO pin
  {ok, RefExport} = file:open("/sys/class/gpio/export", [write]),
  file:write(RefExport, integer_to_list(Pin)),
  file:close(RefExport),

  % It can take a moment for the GPIO pin file to be created.
  case filelib:is_file(DirectionFile) of
      true -> ok;
      false -> receive after 1000 -> ok end
  end,

  {ok, RefDirection} = file:open(DirectionFile, [write]),
  case Direction of
    in -> file:write(RefDirection, "in");
    out -> file:write(RefDirection, "out")
  end,
  file:close(RefDirection),
  {ok, RefVal} = file:open("/sys/class/gpio/gpio" ++ integer_to_list(Pin) ++ "/value", [read, write]),
  RefVal.

release(Pin) ->
  {ok, RefUnexport} = file:open("/sys/class/gpio/unexport", [write]),
  file:write(RefUnexport, integer_to_list(Pin)),
  file:close(RefUnexport).

% @doc: Message passing interface, should not be used directly, it is present for debugging purpose.
handler(Ref, Pin) ->
  receive
    {send, Val} ->
      file:position(Ref, 0),
      file:write(Ref, integer_to_list(Val)),
      handler(Ref, Pin);
    {recv, From} ->
      file:position(Ref, 0),
      {ok, Data} = file:read(Ref, 1),
      From ! Data,
      handler(Ref, Pin);
    stop ->
      file:close(Ref),
      release(Pin),
      ok
   end.

%% End of Module.

Summary

Erlang is a very interesting language. If you are interested in functional programming or how to create highly scalable reliable web servers, then Erlang is definitely worth checking out. We only looked at a quick introduction to the language, really by example. There are lots of good documentation, tutorials and samples just a Google search away. Perhaps in a future article we’ll look at processes and messages and how to build such a highly scalable server.

 

Written by smist08

February 18, 2018 at 11:59 pm

Posted in raspberry pi

Tagged with , ,

Refurbishing an Old Laptop

with one comment

Introduction

My wife typically used an HP laptop running Windows 10. This laptop is probably about three years old now. Unfortunately she dropped it off the bed onto a cement floor (she claims one of the dogs pushed it off the bed, but I’m suspicious). She actually broke its fall with her foot which is very bruised now, and then it hit the floor after her foot. The laptop then stopped working. She took the laptop to a coffee shop and spilled coffee on it. This didn’t help. She then gave it to me to see if I could get it working. I took it apart to see if it was just a matter of some wires coming loose. Sadly this wasn’t the case. This blog article recounts my attempts to disassemble and repair the laptop, which I successfully accomplished. I’m writing this article on said laptop, while my wife now uses my MacBook Air.

Diagnosing the Problem

On bootup the laptop displayed a text message prompt saying no boot device present. To me this meant that the screen and processor were still working and that the likely problem was the hard drive. After all the harddrive is mechanical with moving parts and likely to be damaged by a shock such as hitting the floor (or a foot). So my first thought was to take it apart and ensure the cables connecting the harddrive were in place.

Opening the Case

It turns out an HP laptop is held together by a lot of screws and many of the screws are hidden behind plastic plugs, the laptop’s feet and various other sneaky bits of plastic. Plus there is a bead of glue around the case that needed to be broken. My first attempt to remove all the screws didn’t allow me inside. Fortunately for doing DIY repairs, there is YouTube. Doing a quick search revealed lots of videos on opening HP laptops and showed a lot of complaining about how HP makes this unnecessarily complex. Anyway watching one of the videos revealed to me where to look for the additional screws (behind some nondescript covers) and allowed me to get the case open.

I removed the hard drive, made sure all the cables were snug and put it back together. And the same result. So I assumed the harddrive was either physically broken or the boot sectors had somehow been erased. So I thought I may as well try installing Ubuntu Linux and repurpose the laptop as a Linux machine. After all running Windows 10, the laptop has been getting slower and slower, so giving up on Windows 10 seemed like a good idea.

Installing Ubuntu Linux

I downloaded an ISO image of Ubuntu Linux for desktop computers. And then installed this onto a USB key to make a bootable USB with Linux. I did this on a really old AMD Windows 7 laptop. Its slow but it’s always been reliable and since I’ve uninstalled nearly everything from it, it works fairly well. Anyway Ubuntu has really good instructions on how to do this at their website. You just download the ISO image and then use the Rufus utility program to copy it to the USB making it bootable.

I then put it in the damaged laptop and booted it up. At this point I could run Ubuntu Linux from the USB or run the Ubuntu installation program. That both of these worked gave me more confidence that the only problem was the laptop’s hard drive. Anyway I went ahead and tried to install Ubuntu Linux, which goes quite a long way before it finally tries to format the hard drive. This failed. So I concluded the harddrive was physically damaged and useless.

Fortunately I have a couple of other laptop hard drives lying around from even older laptops which I could try. The first one I tried didn’t work, so I guess that one is toast as well. But the second one I tried worked. The Ubuntu installation program was able to format the drive and the installation finished successfully. My impression is that installing Linux is now even easier than installing Windows. It didn’t ask too many questions. The screens were simple and easy to understand.

Now I had the laptop restored. Even using the old harddrive, my impression is that the laptop is much faster now running Ubuntu. Of course now I don’t need to run all that Windows stuff that slows computers down so much (like Windows Defender).

Adding Programs

Ubuntu, like Raspbian is based on Debian Linux so anything I’ve blogged about installing things on my Raspberry also works pretty much the same. Plus Ubuntu comes with Firefox, LibreOffice and a number of other useful programs already installed. I was able to quickly add Chrome, Macchanger, Gimp and Tensorflow. For amateur radio there is a program called Chirp that can be used to program most radios. I never got Chirp to talk to my radio from my MacBook due to supposed driver incompatibilities with the USB to serial cable. But when I added Chirp to Ubuntu, it just worked and communicated with my radio no problem, first try.

With my MacBook I use an external drive to hold my photo library since its so large. I use one that is formatted for the MacOS filesystem, so I wasn’t sure it would work connected to Ubuntu, but when I plugged it in, Ubuntu recognized it and automatically mounted it with no problems. I imagine this is since both operating systems are Unix based and their file systems are compatible. I’ve never tried to make this work with Windows, because the instructions are rather daunting and require quite a bit of extra software to be installed.

Summary

The end result of all this is that I have a new refurbished laptop which I’m now using as my regular laptop running Ubuntu Linux. My wife is happily continuing writing her novels on the MacBook Air. And at least for now we don’t need to buy a new laptop. Perhaps we will down the road if we see a really good deal, but it isn’t an emergency. I never feel rushed to buy a new computer since they keep getting better all the time, plus prices are bit high right now because of the run on graphics cards by people doing BitCoin mining. Might be nice to let this run its course first.

 

Written by smist08

February 1, 2018 at 9:48 pm

Posted in Life

Tagged with , , ,

Kali Linux on the Raspberry Pi

leave a comment »

Introduction

Raspbian is the main operating system for the Raspberry Pi, but there are quite a few alternatives. Raspbian is based on Debian Linux and there has been a good uptake on the Raspberry Pi which means that most Linux applications have ARM compiled packages available through the Debian APT package manager. As a consequence it’s quite easy to create a Raspberry Pi Linux distribution, so there are quite a few of them. Kali Linux is a specialist distribution that is oriented to hackers (both black and white hat). It comes with a large number of hacking tools for gaining access to networks, compromising computers, spying on communications and other fun things. One cool thing is that Kali Linux has a stripped down version for the Raspberry Pi that is oriented towards a number of specialized purposes. However with the apt-get package manager you can add pretty well anything that has been left out.

If you watch the TV show Mr. Robot (highly recommended) then you might notice that all the cool in-the-know people are running Kali Linux. If you want to get a taste of what it’s all about and you have a Raspberry Pi then all you need is a free micro-SD card to give it a try.

Are You a Black or White Hat?

If you are a black hat hacker looking to infiltrate or damage another computer system, then you probably aren’t reading this blog. Instead you are somewhere on the darknet reading much more malicious articles than this one. This article is oriented more to white hat hackers or to system administrators looking to secure their computing resources. The reason it’s important for system administrators to know about this stuff is that they need to know what they are really protecting against. Hackers are very clever and are always coming up with new hacks and techniques. So it’s important for the good guys to know a bit about how hackers think and to have defenses and protections against the imaginative attacks that might come their way. This now includes the things the bad guys might try to do with a Raspberry Pi.

Anyone that is responsible for securing a network or computer has to test their security and certainly one easy way to get started is to hit it with all the exploit tools included with Kali Linux.

Why Kali on the Pi?

A lot of hacking tasks like cracking WiFi passwords take a lot of processing power. Cracking WPA2 passwords is usually done on very powerful computers with multiple GPU cards and very large dictionary databases. Accomplishing this on a Raspberry Pi would pretty much take forever if it could actually do it. Many hacking tasks are of this nature, very compute intensive.

The Raspberry Pi is useful due to its low cost and small size. The low cost makes it disposable, if you lose it then it doesn’t matter so much and the small size means you can hide it easily. So for instance one use would be to hide a Raspberry Pi at the site you are trying to hack. Then the Raspberry Pi can monitor the Wifi traffic looking for useful data packets that can give away information. Or even leave the Pi somewhere hidden connected to a hardwired ethernet connection. Then Kali Linux has tools to get this information to external sources in a secretive way and allows you to remotely control it to direct various attacks.

Many companies build their security like eggs with a hard to penetrate shell and often locating a device on their premises can bypass their main security protections. You can then run repeated metasploit attacks looking for weaknesses from the inside. Remember your security should be more like an onion with multiple nested layers, so getting through one doesn’t give an attacker access to everything.

Installing Kali Linux

The Kali Linux web site includes a complete disk image of the Raspberry Pi version. You just need to burn this to a micro-SD card using a tool like ApplePi Baker. They you just put the micro-SD in your Raspberry Pi, turn it on and off you go. However there are a few necessary steps to take before you really start:

  1. The root password is toor, so change this first time you boot up.
  2. The Kali Linux instructions point out you need to refresh your SSH certificates since otherwise you get the ones included with the image. The download page has instructions on how to do this.
  3. The image is configured for 8Gig so if you have a larger SD card then you need to repartition it to get all the free space. I used the GParted program for this which I got via “apt-get install gparted”. Note that to use apt-get you need to connect to WiFi or the Internet. Another option is to get Raspbian’s configuration program and use that, it works with most variants of Debian Linux and allows you to do some other things like setup overclocking. You can Google for the wget command to get this.
  4. Update the various installed programs via “apt-get update” and “apt-get upgrade”. (If you aren’t still logged on as root you need to sudo these).

Now you are pretty much ready to play. Notice that you are in a nice graphical environment and that the application menu is full of hacking tools. These aren’t as many hacking tools as the full Kali distribution, but these are all ones that work well on the Raspberry Pi. They als limited the number so you can run off a really cheap 8 Gig micro-SD card.

I see people say you should stick to command line versions of the tools on the Pi due to its processing power and limited memory, but I found I could add the GUI versions and these worked fine. For instance the nmap tool is installed, but the zenmap graphical front end isn’t. Adding zenmap is easy via “apt-get install zenmap” and then this seems to work fine. I think the assumption is that most people will use the Raspberry version of Kali headless (meaning no screen or keyboard) so it needs to be accessed via remote control software like secure shell which means you want to avoid GUIs.

Summary

Installing Kali Linux on a micro-SD card for your Raspberry Pi si a great way to learn about the various tools that hackers can easily use to try and penetrate, spy on or interfere with people’s computers. There are quite a few books on this as well as many great resources on the Web. Kali’s website is a great starting point. Anyway I find it quite eye opening the variety of readily tools and how easy it is for anyone to use them.

Written by smist08

January 16, 2018 at 3:01 am

Flashing LEDs in Assembler

leave a comment »

Introduction

Previously I wrote an article on an introduction to Assembler programming on the Raspberry Pi. This was quite a long article without much of a coding example, so I wanted to produce an Assembler  language version of the little program I did in Python, Scratch, Fortran and C to flash three LEDs attached to the Raspberry Pi’s GPIO port on a breadboard. So in this article I’ll introduce that program.

This program is fairly minimal. It doesn’t do any error checking, but it does work. I don’t use any external libraries, and only make calls to Linux (Raspbian) via software interrupts (SVC 0). I implemented a minimal GPIO library using Assembler Macros along with the necessary file I/O and sleep Linux system calls. There probably aren’t enough comments in the code, but at this point it is fairly small and the macros help to modularize and explain things.

Main Program

Here is the main program, that probably doesn’t look structurally that different than the C code, since the macro names roughly match up to those in the GPIO library the C function called. The main bit of Assembler code here is to do the loop through flashing the lights 10 times. This is pretty straight forward, just load 10 into register r6 and then decrement it until it hits zero.

 

@
@ Assembler program to flash three LEDs connected to the
@ Raspberry Pi GPIO port.
@
@ r6 - loop variable to flash lights 10 times
@

.include "gpiomacros.s"

.global _start             @ Provide program starting address to linker

_start: GPIOExport  pin17
        GPIOExport  pin27
        GPIOExport  pin22

        nanoSleep

        GPIODirectionOut pin17
        GPIODirectionOut pin27
        GPIODirectionOut pin22

        @ setup a loop counter for 10 iterations
        mov         r6, #10

loop:   GPIOWrite   pin17, high
        nanoSleep
        GPIOWrite   pin17, low
        GPIOWrite   pin27, high
        nanoSleep
        GPIOWrite   pin27, low
        GPIOWrite   pin22, high
        nanoSleep
        GPIOWrite   pin22, low

        @decrement loop counter and see if we loop
        subs    r6, #1      @ Subtract 1 from loop register setting status register
        bne     loop        @ If we haven't counted down to 0 then loop

_end:   mov     R0, #0      @ Use 0 return code
        lsl     R0, #2      @ Shift R0 left by 2 bits (ie multiply by 4)
        mov     R7, #1      @ Service command code 1 terminates this program
        svc     0           @ Linus command to terminate program

pin17:      .asciz  "17"
pin27:      .asciz  "27"
pin22:      .asciz  "22"
low:        .asciz  "0"
high:       .asciz  "1"

 

GPIO and Linux Macros

Now the real guts of the program are in the Assembler macros. Again it isn’t too bad. We use the Linux service calls to open, write, flush and close the GPIO device files in /sys/class/gpio. Similarly nanosleep is also a Linux service call for a high resolution timer. Note that ARM doesn’t have memory to memory or operations on memory type instructions, so to do anything we need to load it into a register, process it and write it back out. Hence to copy the pin number to the file name we load the two pin characters and store them to the file name memory area. Hard coding the offset for this as 20 isn’t great, we could have used a .equ directive, or better yet implemented a string scan, but for quick and dirty this is fine. Similarly we only implemented the parameters we really needed and ignored anything else. We’ll leave it as an exercise to the reader to flush these out more. Note that when we copy the first byte of the pin number, we include a #1 on the end of the ldrb and strb instructions, this will do a post increment by one on the index register that holds the memory location. This means the ARM is really very efficient in accessing arrays (even without using Neon) we combine the array read/write with the index increment all in one instruction.

If you are wondering how you find the Linux service calls, you look in /usr/include/arm-linux-gnueabihf/asm/unistd.h. This C include file has all the function numbers for the Linux system calls. Then you Google the call for its parameters and they go in order in registers r0, r1, …, r6, with the return code coming back in r0.

 

@ Various macros to access the GPIO pins
@ on the Raspberry Pi.

@ R5 is used for the file descriptor

.macro  openFile    fileName
        ldr         r0, =\fileName
        mov         r1, #01     @ O_WRONLY
        mov r7,     #5          @ 5 is system call number for open
        svc         0
.endm

.macro  writeFile   buffer, length
        mov         r0, r5      @ file descriptor
        ldr         r1, =\buffer
        mov         r2, #\length
        mov         r7, #4 @ 4 is write
        svc         0
.endm

.macro  flushClose
@fsync syscall
        mov         r0, r5
        mov         r7, #118    @ 118 is flush
        svc         0

@close syscall
        mov         r0, r5
        mov         r7, #6      @ 6 is close
        svc         0
.endm

@ Macro nanoSleep to sleep .1 second
@ Calls Linux nanosleep entry point which is function 162.
@ Pass a reference to a timespec in both r0 and r1
@ First is input time to sleep in seconds and nanoseconds.
@ Second is time left to sleep if interrupted (which we ignore)

.macro  nanoSleep
        ldr         r0, =timespecsec
        ldr         r1, =timespecsec
        mov         r7, #162    @ 162 is nanosleep
        svc         0
.endm

.macro  GPIOExport  pin
        openFile    gpioexp
        mov         r5, r0      @ save the file descriptor
        writeFile   \pin, 2
        flushClose
.endm

.macro  GPIODirectionOut   pin
        @ copy pin into filename pattern
        ldr         r1, =\pin
        ldr         r2, =gpiopinfile
        add         r2, #20
        ldrb        r3, [r1], #1 @ load pin and post increment
        strb        r3, [r2], #1 @ store to filename and post increment
        ldrb        r3, [r1]
        strb        r3, [r2]
        openFile    gpiopinfile
        writeFile   outstr, 3
        flushClose
.endm

.macro  GPIOWrite   pin, value
        @ copy pin into filename pattern
        ldr         r1, =\pin
        ldr         r2, =gpiovaluefile
        add         r2, #20
        ldrb        r3, [r1], #1    @ load pin and post increment
        strb        r3, [r2], #1    @ store to filename and post increment
        ldrb        r3, [r1]
        strb        r3, [r2]
        openFile    gpiovaluefile
        writeFile   \value, 1
        flushClose
.endm

.data
timespecsec:   .word   0
timespecnano:  .word   100000000
gpioexp:    .asciz  "/sys/class/gpio/export"
gpiopinfile: .asciz "/sys/class/gpio/gpioxx/direction"
gpiovaluefile: .asciz "/sys/class/gpio/gpioxx/value"
outstr:     .asciz  "out"
            .align  2          @ save users of this file having to do this.
.text

Makefile

Here is a simple makefile for the project if you name the files as indicated. Again note that WordPress and Google Docs may mess up white space and quote characters so these might need to be fixed if you copy/paste.

model: model.o
    ld -o model model.o

model.o: model.s gpiomacros.s
    as -ggdb3 -o model.o model.s

clean:
    rm model model.o

 

IDE or Not to IDE

People often do Assembler language development in an IDE like Code::Blocks. Code::Blocks doesn’t support Assembler language projects, but you can add Assembler language files to C projects. This is a pretty common way to do development since you want to do more programming in a higher level language like C. This way you also get full use of the C runtime. I didn’t do this, I just used a text editor, make and gdb (command line). This way the above program has no extra overhead the executable is quite small since there is no C runtime or any other library linked to it. The debug version of the executable is only 2904 bytes long and non debug is 2376 bytes. Of course if I really wanted to reduce executable size, I could have used function calls rather than Assembler macros as the macros duplicate the code everywhere they are used.

Summary

Assembler language programming is kind of fun. But I don’t think I would want to do too large a project this way. Hats off to the early personal computer programmers who wrote spreadsheet programs, word processors and games entirely in Assembler. Certainly writing a few Assembler programs gives you a really good understanding of how the underlying computer hardware works and what sort of things your computer can do really efficiently. You could even consider adding compiler optimizations for your processor to GCC, after all compiler code generation has a huge effect on your computer’s performance.

Written by smist08

January 7, 2018 at 7:08 pm

Spectre Attacks on ARM Devices

leave a comment »

Introduction

I predicted that 2018 would be a very bad year for data breaches and security problems, and we have already started the year with the Intel x86 specific Meltdown exploit and the Spectre exploit that works on all sorts of processors and even on some JavaScript systems (like Chrome). Since my last article was on Assembler programming and most of these type exploits are created in Assembler, I thought it might be fun to look at how Spectre works and get a feel for how hackers can retrieve useful data out of what seems like nowhere. Spectre is actually a large new family of exploits so patching them all is going to take quite a bit of time, and like the older buffer overrun exploits, are going to keep reappearing.

I’ve been writing quite a bit about the Raspberry Pi recently, so is the Raspberry Pi affected by Spectre? After all it affects all Android and Apple devices based on ARM processors. The main Raspberry Pi operating system is Raspbian which is variant of Debian Linux optimized for the Pi. A recent criticism of Raspbian is that it is still 32-Bit. It turns out that running the ARM in 32-bit mode eliminates a lot of the Spectre attack scenarios. We’ll discuss why this is in the article. If you are running 64-Bit software on the Pi (like running Android) then you are susceptible. You are also susceptible to the software versions of this attack like those in JavaScript interpreters that support branch prediction (like Chromium).

The Spectre hacks work by exploiting how processor branch prediction works coupled with how data is cached. The exploits use branch prediction to access data it shouldn’t and then use the processor cache to retrieve the data for use. The original article by the security researchers is really quite good and worth a read. Its available here. It has an appendix at the back with C code for Intel processors that is quite interesting.

Branch Prediction

In our last blog post we mentioned that all the data processing assembler instructions were conditionally executed. This was because if you perform a branch instruction then the instruction pipeline needs to be cleared and restarted. This will really stall the processor. The ARM 32-bit solution was good as long as compilers are good at generating code that efficiently utilize these. Remember that most code for ARM processors is compiled using GCC and GCC is a general purpose compiler that works on all sorts of processors and its optimizations tend to be general purpose rather than processor specific. When ARM evaluated adding 64-Bit instructions, they wanted to keep the instructions 32-Bit in length, but they wanted to add a bunch of instructions as well (like integer divide), so they made the decision to eliminate the bits used for conditionally executing instructions and have a bigger opcode instead (and hence lots more instructions). I think they also considered that their conditional instructions weren’t being used as much as they should be and weren’t earning their keep. Plus they now had more transistors to play with so they could do a couple of other things instead. One is that they lengthed the instruction pipeline to be much longer than the current three instructions and the other was to implement branch prediction. Here the processor had a table of 128 branches and the route they took last time through. The processor would then execute the most commonly chosen branch assuming that once the conditional was figured out, it would very rarely need to throw away the work and start over. Generally this larger pipeline with branch prediction lead to much better performance results. So what could go wrong?

Consider the branch statement:

 

if (x < array1_size)
    y = array2[array1[x] * 256];


This looks like a good bit of C code to test if an array is in range before accessing it. If it didn’t do this check then we could get a buffer overrun vulnerability by making x larger than the array size and accessing memory beyond the array. Hackers are very good at exploiting buffer overruns. But sadly (for hackers) programmers are getting better at putting these sort of checks in (or having automated or higher level languages do it for them).

Now consider branch prediction. Suppose we execute this code hundreds of times with legitimate values of x. The processor will see the conditional is usually true and the second line is usually executed. So now branch prediction will see this and when this code is executed it will just start execution of the second line right away and work out the first line in a second execution unit at the same time. But what if we enter a large value of x? Now branch prediction will execute the second line and y will get a piece of memory it shouldn’t. But so what, eventually the conditional in the first line will be evaluated and that value of y will be discarded. Some processors will even zero it out (after all they do security review these things). So how does that help the hacker? The trick turns out to be exploiting processor caching.

Processor Caching

No matter how fast memory companies claim their super fast DDR4 memory is, it really isn’t, at least compared to CPU registers. To get a bit of extra speed out of memory access all CPUs implement some sort of memory cache where recently used parts of main memory are cached in the CPU for faster access. Often CPUs have multiple levels of cache, a super fast one, a fast one and a not quite as fast one. The trick then to getting at the incorrectly calculated value of y above is to somehow figure out how to access it from the cache. No CPU has a read from cache assembler instruction, this would cause havoc and definitely be a security problem. This is really the CPU vulnerability, that the incorrectly calculated buffer overrun y is in the cache. Hackers figured out, not how to read this value but to infer it by timing memory accesses. They could clear the cache (this is generally supported and even if it isn’t you could read lots of zeros). Then time how long it takes to read various bytes. Basically a byte in cache will read much faster than a byte from main memory and this then reveals what the value of y was. Very tricky.

Recap

So to recap, the Spectre exploit works by:

  1. Clear the cache
  2. Execute the target branch code repeatedly with correct values
  3. Execute the target with an incorrect value
  4. Loop through possible values timing the read access to find the one in cache

This can then be put in a loop to read large portions of a programs private memory.

Summary

The Spectre attack is a very serious new technique for hackers to hack into our data. This will be like buffer overruns and there won’t be one quick fix, people are going to be patching systems for a long time on this one. As more hackers understand this attack, there will be all sorts of creative offshoots that will deal further havoc.

Some of the remedies like turning off branch prediction or memory caching will cause huge performance problems. Generally the real fixes need to be in the CPUs. Beyond this, systems like JavaScript interpreters, or even systems like the .Net runtime or Java VMs could have this vulnerability in their optimization systems. These can be fixed in software, but now you require a huge number of systems to be patched and we know from experience that this will take a very long time with all sorts of bad things happening along the way.

The good news for Raspberry Pi Raspbian users, is that the ARM in the older 32-Bit mode isn’t susceptible. It is only susceptible through software uses like JavaScript. Also as hackers develop these techniques going forwards perhaps they can find a combination that works for the Raspberry, so you can never be complacent.

 

Written by smist08

January 5, 2018 at 10:42 pm