Posts Tagged ‘raspbian’
The Race for 64-Bit Raspberry Pi 4 Linux
Introduction
When the Raspberry Pi 4 was announced and shipped this past June, it caught everyone by surprise. No one was expecting a new Pi until next year sometime, if we were lucky. The Raspberry Pi 4 has updated faster components, including an updated ARM processor and USB 3.0. Raspbian, the official version version of Linux for the Pi was updated to be based on Debian Buster and shipped before the official Debian Buster actually shipped. However, Raspbian is still 32-bit, where the Raspberry foundation say this is so they only have to support one version of Linux for all Raspberry Pi devices.
Others in the Linux community, have then figured out how to run 64-bit Linux’s on the Raspberry Pi. For instance there are 64-bit versions of Ubuntu Mate, Ubuntu Server and Kali Linux. These work on the Raspberry Pi 3, but due to changes in the Raspberry architecture, didn’t work on the Raspberry Pi 4 when it shipped. We still don’t have official 64-bit releases, but we are reaching the point where the test builds are starting to work quite well.
Why 64-Bit?
To be honest, 64-bit Linux never ran very well on the Raspberry Pi 3. 64-bit Linux and 64-bit programs requires quite a bit more memory than their 32-bit equivalents. Each memory address is now 64-bits instead of 32-bits and there is a tendency to use 64-bit integers rather than 32-bit integers. The ARM processor instructions are 32-bits in both 32-bit and 64-bit mode, so programs tend to be about the same size, though 64-bit doesn’t have use of the 16-bit ARM thumb instructions. The Raspberry Pi 3 is limited to 1Gig of memory, that can just barely run a 64-bit Linux, and tends to run out of memory quickly as you run programs, like web browsers. The Raspberry Pi 4 now supports up to 4Gig of memory and that is sufficient to run 64-bit Linux along with a respectable number of programs. Plus the Raspberry Pi 4 has faster access to the SDCard and USB 3, so you can attach an even faster external drive, so if you do get swapping, it isn’t as painful.
In spite of these limitations, there are reasons to run 64-bit. The main one is that you can get better performance, especially if you actually need to work with 64-bit integers. Further the 64-bit instruction set has been optimised to work better with the execution pipeline, so you don’t get as many stalls when you perform jumps. For instance in 32-bit ARM, there is no function return instructions, so people use regular branches, pop the return address from the stack directly into the program counter or do a number of other tricks. As a result, function returns causes the execution pipeline to be flushed. In 64-bit, the pipeline knows about return instruction and knows where to get the next few instructions.
If 64-Bit Worked on the Pi 3, What’s the Problem?
If we had 64-bit working for the Pi 3, why doesn’t it just work on the Pi 4? There are a few reasons for this. The first obstacle was that Raspberry changed the whole Pi boot process. The Raspberry Pi 3 booted using the GPU. When it started the Pi 3’s GPU runs a program that knows how to read the boot folder on an SDCard and will load this into memory and then start the ARM CPU to run what it loaded into memory. The Raspberry Pi 4 now has a slightly larger EEPROM, this contains ARM code that executes on startup and then loads a further step from the SDCard. The volunteers with the other Linux distributions had to figure out this new process and adapt their code to fit into it. Sadly, the original EEPROM program didn’t provide a good way to do this, so the Linux volunteers have been working with Raspberry to get the support they need in newer versions of the EEPROM software. The most recent version seems to be working reliably finally.
The Raspberry Pi 4 then has all new hardware, so new drivers are required for bluetooth, wifi and everything else. To keep the price down, Raspberry uses older standard components, so there are drivers already written for all these devices. It’s just a matter of including the correct drivers and providing default configurations that work and settings dialogs if anything might need user input. This is all being worked on in parallel, and the consensus is that we are already in a better place than we were for the Pi 3.
It’s All Open Source so Why not Copy from Rasbian?
The Raspbian kernel is open source so anyone can look at that source code, but the EEPROM firmware is not open source. This can be reverse engineered, but that takes time. The Raspberry Pi foundation has been quite helpful in supporting people, but that is no substitute for reading the source code. This again shows the importance of open source BIOS.
Development got off to a slow start, because the Raspberry Pi foundation didn’t give anyone a heads up that this was coming. The developers of Ubuntu Mate had to order their Raspberry Pi 4’s just like everyone else when the announcement happened. This meant no one really got started until into July.
In spite of claiming up and down that they will never produce a 64-bit version of Raspbian, the Raspberry Pi foundation has produced a test Raspbian 64-bit Linux kernel. This then tests out that the Raspberry Pi firmware will support 64-bits and that all the device drivers are available. I couldn’t get this kernel to work, but it is proving very helpful for other developers. It also makes people excited that maybe Raspbian will go 64-bit sooner than later.
How Are We Doing?
The first distribution to get all this going is Gentoo Linux. They have a very smart developer Sakaki who provided the first image that actually worked. This then led to Arch and Majaro Linux releases based on Gentoo. This was a good first step, though these distributions are more for the DIY crowd due to their preference to always installing software from source code.
Next James Chambers put together a guide and images to allow you to install Ubuntu Server 64-bit on the Pi 4. Ubuntu Server is character based, but installing a desktop is no problem. The main limitation of this release is that you need a hardwired Internet connection to start. You can’t start with Wifi as the Wifi software isn’t installed with the base image. If you do have a wired Internet connection, getting it installed and installing the desktop is quite straightforward and works well. Once you have the desktop installed, then you can configure Wifi and ditch the ethernet cable.
The changes required for the Raspberry Pi 4 are being submitted to the standard Linux kernel for version 5.4. When this ships it will have available drivers for the Pi 4 hardware and official support for the Broadcom chips used in the Pi. Version 5.3 of the Linux kernel just shipped and added support for the NVidia Jetson Nano. Hopefully the wait for Linux 5.4 won’t be too long.
Summary
I’ve been running the 64-bit version of Ubuntu Linux Server, with the Xubuntu desktop for a few days now and it works really well on my Raspberry Pi 4 with 4Gig of RAM. Performance is great and everything is working. I’ve installed various software, including CubicSDR which works great. This is the first time I’ve been happy with Software Defined Radio running on a Pi.
I look forward to the official releases, and given the state of the current builds, think we are getting quite close.
Raspberry Pi 4 as a Desktop Computer
Introduction
The Raspberry Pi Foundation is promoting the Raspberry Pi 4 as a full desktop computer for only $35. I’ve had my Raspberry Pi 4 for about a month now and in this article we’ll discuss if it really is a full desktop computer replacement. This partly depends on what you use your desktop computer for. My answer is that the $35 price is misleading, you need to add quite a few other things to make it work well.
Making the Raspberry Pi 4 into a Decent Desktop
The Raspberry Pi has always been a barebones computer. You’ve always needed to add a case, a keyboard, a mouse, a monitor, a power supply, a video cable and a microSD card. Many people already have these kicking around, so they don’t need to buy them when they get their Pi. For instance, I already had a keyboard and monitor. The Raspberry Pi 4 even supports two monitors.
Beyond the bare bones, you need two more things for a decent desktop, namely:
With these, it starts to feel like you are playing with a regular desktop computer. You now have enough RAM to run multiple programs and any good SSD will greatly enhance the performance of thee system, only using the microSD card to boot the Pi.
The Raspberry Pi 3 is a great little computer. Its main limitation is that if you run too many programs or open too many browser tabs, it bogs down and you have a painful process of closing windows (that aren’t responding well), until things pick up again. Now the Raspberry Pi 4 with 4GB of RAM really opens up the number of things you can do at once. Running multiple browser tabs, LibreOffice and a programming IDE are no problem.
The next thing you run into with the Raspberry Pi 4 is the performance of the SD card. Since I needed a video cable and a new case, I ordered a package deal that also included a microSD card containing Raspbian. Sadly, these bundled microSD cards are the cheapest, and hence slowest available. Having Raspbian bundled on a slow card is just a waste. Switching to a Sandisk Extreme 64GB made a huge difference. The speed was much better. When buying a microSD card watch the speed ratings, often the bigger cards (64GB or better) are twice as fast as the smaller cards (32GB or less). With a good microSD card the Raspberry Pi 4 can read and write microSD twice as fast as a Raspberry Pi 3.
I’ve never felt I could truly trust running off a microSD card. I’ve never had one fail, but people report problems all the time. Further, the performance of microSD cards is only a fraction of what you can get from good SSDs. The Raspberry Pi 4 comes with two USB 3 ports which have a theoretical performance ten times that of the microSD port. If you shop around you will find M.2 and SATA SSDs for prices less than those of microSD cards. I purchased a Kingston A1000 M.2 drive which was on sale cheap because the A2000 cards just started shipping. I had to get an M.2 USB caddy to contain it, but combined this was less than $100 and USB caddies are always useful.
Unfortunately, you can’t boot the Raspberry Pi 4 directly off a USB port yet. The Raspberry Pi foundation say this is coming, but not quite here yet. What you can do is have the entire root file system on the USB drive, but the boot partition must be on a microSD card. Setting up the SSD was easier than I thought it would be. I had to partition it, format it, copy everything over to the SSD and then edit /boot/config.txt to say where the root of the main file system is.
With this done, I feel like I’m using a real desktop computer. I’m confident my data is being stored reliably, the performance is great.
Overheating
The Raspberry Pi 4 uses more power than previous Pis. This means there is more heat to dissipate. The case I received with my Pi 4 didn’t have any ventilation holes and would get quite hot. I solved the problem by removing the top of the case. This let enough heat out that I could run fine for most things. People report that when using a USB SSD that the USB controller chip will overheat and the data throughput will be throttled. I haven’t run into this, but it is something to be aware of.
I installed Tensorflow, Google’s open source AI toolkit. Training a data model with Tensorflow does make my Pi 4 overheat. I suspect Tensorflow is keeping all four CPU cores busy and producing a maximum amount of heat. This might drive me to add a cooling fan. I like the way the Pi runs so quietly, with no fan, it makes no noise. I might try using a small fan blowing down on the Pi to see is that helps.
Summary
Is the Raspberry Pi 4 a complete desktop computer for $35? No. But if you get the 4GB model for $55 and then add a USB 3 SSD, then you do have a good workable desktop computer. The CPU power of the Pi has been compared to a typical 2012 desktop computer. But for the cost that is pretty good. I suspect the Wifi/Lan and SSD are quite a bit better than that 2012 computer.
Keep in mind the Raspberry Pi runs Linux, which isn’t for everyone. A typical low cost Windows desktop goes for around $500 these days. You can get a refurbished one for $200-$300. A refurbished desktop can be a good inexpensive option.
I like the Raspberry Pi, partly because you are cleanly out of the WinTel world. No Windows, no Intel. The processor is ARM and the operating system is Raspbian based on Debian Linux. A lot of things you do are DIY, but I enjoy that. With over 25 million Raspberry Pis sold worldwide, there is a lot of community support and you join quite an enthusiastic thriving group.
Raspberry Pi 4 First Impressions
Introduction
I’ve received my Raspberry Pi 4B with 4GB or RAM a few weeks ago. I’ve been using it to write my forthcoming book on Raspberry Pi Assembly Language Programming, so I thought I’d give a few of my first impressions. The biggest change for the Raspberry Pi 4 is the support for three memory sizes, 1GB, 2GB and 4GB. This overcomes the biggest complaint against the Raspberry Pi 3, that it bogs down too quickly as you run browser tabs and multiple windows.
Some of the other hardware improvements are:
- Dual 4K monitor support with dual micro-HDMI ports.
- Two of the four USB ports are USB-3.
- The ethernet is now gigabit and the WiFi is faster.
- A 1.5GHz quad-core 64-bit ARM Cortex-A72 CPU.
- The SDRAM is now LPDDR4.
- The GPU is upgraded to Broadcom’s VideoCore VI.
- Hardware HEVC video support for 4Kp60 video.
On paper, this makes the Raspberry Pi 4 appear far superior to its predecessors, In this article, I’ll discuss what is much better and a few of the drawbacks. This release will squash a lot of the compatible Pi competitors, but I’ll compare it to my NVidia Jetson Nano and mention a few places where these products still surpass the Pi.
Raspbian Buster
At the same time the Raspberry Pi Foundation released the Raspberry Pi 4, they also released the new “Buster” version of Raspbian, the Debian Linux derived operating system tailored specifically to the Raspberry Pi. On the day this was announced, I ordered my Raspberry Pi 4, then went and downloaded the new Buster release, then installed it on my Raspberry Pi 3B.
If you have a Raspberry Pi 4, then you must run the Buster release as older versions of Raspbian don’t have support for the newer hardware. If you are running an older Pi then you can keep running the older version or upgrade as you like.
Is it 64-Bits?
The first rumour that was squashed was that Raspbian would move to 64-bits. This didn’t happen. Raspbian is a 32-bit operating system. The Raspberry Pi Foundation says it will stay this way for the foreseeable future. The first reason is that the Raspberry Pi 1 and Raspberry Pi Zero use a much older ARM processor that doesn’t support 64-bits. The Raspberry Pi Foundation still supports and sells these models and they are quite popular due to their low price. They don’t want to support two operating systems, so they stick to one 32-bit version that will run on every Raspberry Pi ever made. Perhaps other hardware vendors should look at this level of support for older models.
Even though 32-bit implies a 32-bit virtual address space for processes, which limits an individual process to 4GB of memory, the ARM SoC used in the Pi has memory access hardware for 48-bit addresses. This allows the operating system to give each process a different 4GB address space, so if Raspberry Pi models with more than 4GB of memory are released, Raspbian can utilize this memory.
Another problem with going to 64-bits is that all the previous Raspberry Pi models, and one version of the Raspberry Pi 4 only have 1GB of RAM. This isn’t sufficient to run a 64-bit operating system. You can do it, but the operating system takes all the RAM, and once you run a program or two, everything bogs down. This is due to all addresses and most integers becoming 64-bits, and hence twice as large. A definite nice feature of Raspbian is that it can run effectively in only 1GB or memory.
Based on Debian Buster
Raspbian is notorious for lagging behind the mainstream releases of Linux. The benefit of this is that Raspbian has always been very stable and reliable. It works well and avoids the problems that happen at the bleeding edge. The downside is that it can contain security vulnerabilities or bugs that are fixed in the newer versions.
With Buster, Raspbian released its version ahead of Debian releasing the main version. Linus Torvalds himself was involved in moving the Pi up to a newer version of the Linux kernel. His concern is that as other hardware platforms adopt proprietary software like UEFI firmware, with government mandated backdoors, that the benefits of open source are being lost. The Raspberry Pi, including its firmware are all open source and there is a feeling in the open source community that this is the future to fight for.
Some Software Not Ported Yet
As a result of the move to Buster, some software that Raspberry users are accustomed to is missing. The most notable case is Mathematica. A port of this is underway and it is promised to be included in a future upgrade.
I had problems with CubicSDR, a Software Defined Radio (SDR) program. It could detect my SDR USB device, but didn’t run properly, just displaying a blank screen when receiving.
Heat Dissipation
The Raspberry Pi 4 uses more power than previous models. It requires a USB-C power adapter which means you can’t use a power adapter from previous models. I bought my Pi 4 from buyapi.ca and got the bundle with a case, power adapter, heat sinks and micro-HDMI cable. I needed the cables. The case is their Raspberry Pi 3 case, with the holes for the cables moved for the slightly different Pi 4 configuration. The case lacked any ventilation holes and the Pi would throttle due to overheating fairly easily. My solution was to run it with the top of the case removed. This seems to provide enough air circulation that I haven’t seen any overheating since.
Some people claim the Raspberry Pi 4 requires a fan for cooling, but that hasn’t been my experience. I think the cases need properly thought out ventilation and that is all that is needed. I think a bigger heatsink like the one included with the NVidia Jetson Nano would be warranted as well. I don’t like fans and consider the quietness of the Pi as one of its biggest features.
Cons
All this sounds great, but what are the downsides of the Raspberry Pi 4?
All New Cables
I purchased an NVidia Jetson Nano and to run it, I just unplugged the cables from my Raspberry Pi 3 and plugged them into the Jetson and away it went. Not new cables required.
The Raspberry Pi required a new USB-C power supply and a lot has been made of how you can’t use Apple laptop power supplies, but I think the real issue is you can’t use an older Pi power supply, even if it can provide sufficient power.
To support dual monitors, the Pi went to micro-HDMI ports to fit both connectors. This means you need either new cables or at least micro- to regular-HDMI adapters. The NVidia Jetson supports dual monitors but annoyingly with two different cables, HDMI and a DisplayPort cable. At least the cables are the same for the two video ports.
Otherwise all my USB devices that I was using with the Raspberry Pi 3 seem to work with the Pi 4.
SDCard Bottleneck
They have improved the data transfer speed to and from the microSD card with the Pi 4, but this is still a bottleneck. I would have loved it if they had added a M.2 SSD interface to the board. You can improve on the microSD card speed by using a USB 3 external SSD. The problem is that you can’t boot from this USB 3 drive. You can copy the root filesystem over to the drive and run mostly from the USB and although I haven’t tried it yet, people report this is an improvement.
Raspberry Pi promote the 4 as a desktop computer replacement and it definitely has the processing power. However, I don’t think this really holds up without something better than running off a microSD card. The Raspberry Pi Foundation say they will add boot from USB support in a future firmware update, but it isn’t there yet. Although the speed of USB 3 is better than the microSD interface, it still isn’t nearly as good as you can obtain with M.2 and a good new SSD.
No 64-Bits Yet
The Raspberry Pi Foundation, caught everyone by surprise with their release. This included the people that maintain alternate operating systems for the Raspberry Pi. There is a good Ubuntu Mate 64-bit version that runs on the Raspberry Pi 3. It is slow and you can’t run many programs, but it does work and you can experiment with things like ARM 64-bit Assembly programming.
The person that maintains this had to order his Raspberry Pi 4, like everyone else and hasn’t produced a Pi 4 version yet. It would have been nice if the Raspberry Pi Foundation had seeded some early models to the people that develop alternate operating systems for the Pi.
As of this writing, Raspbian is the only operating system that runs on the Raspberry Pi, but hopefully the others won’t take too long to modify what they need to.
The Raspberry Pi 4 with 4GB is the first Raspberry Pi that has the power to run a true 64-bit operating system, so it would be nice to play with.
Cost
The Raspberry Pi 4 is still dirt cheap, $35 for the 1GB model and $55 for the 4Gig model. This upgrade is a bit more expensive since you need a new power adapter, new video cables and a new case as well. I think the extra $20 for the extra memory is well worth it.
Compared to the NVidia Jetson Nano
The Raspberry Pi 4 blows most of the current crop of Pi clones out of the water. One exception is the NVidia Jetson Nano. This single board computer has 4GB of memory and runs full 64-bit Ubuntu Linux and as a consequence feels more powerful than the Pi 4.
The Pi 4 has a more powerful ARM CPU, but the Jetson has 4 USB-C ports and a 128 core CUDA GPU. The CUDA GPU is used by software like CubicSDR for DSP like processing, along with most AI toolkits like Tensorflow.
The NVidia Jetson costs $99, so is nearly twice as expensive as a Pi 4. However if you want to experiment with AI, the 128-core CUDA GPU is an excellent entry level system.
Summary
I got used to the Raspberry Pi 4 fairly quickly and after a couple of weeks thought it was pretty similar to the Raspberry Pi 3. I then needed to do something on my Raspberry 3 and booted it up. After using the Pi 4, going back to the Pi 3, felt like I was working in molasses, everything was so slow. This is a real testament to how good the new Pi is, especially with 4GB of memory.
Yes, there are some teething problems with the new model, as there often is at the bleeding edge. But overall the Raspberry Pi 4 is a solid upgrade, and once you adopt it, you really can’t go back.
Erlang on the Raspberry Pi
Introduction
Erlang is a rather interesting programming language with a number of rather unique properties. If you are used to procedural languages like C or its many variants, then Erlang will appear very different. Erlang was originally developed by Ericsson to program telephone switches. The name is either based on Ericsson Language or is named after Agner Krarup Erlang depending on who you ask. In Erlang there are no loops, all loops are accomplished via recursion. Similarly once a variable is assigned, it can never be changed (is immutable). A lot of the function execution is controlled via pattern matching. Beyond the basic language the Erlang system is used to create highly available, scalable fault tolerant systems. You can even swap code on the fly without stopping the system. Erlang is still used in many telephony type applications, but it is also used to create highly reliable and scalable web sites. The best known being WhatsApp. Plus Facebook, Yahoo and many others have services implemented in Erlang serving massive numbers of users.
In this article we’ll begin to look at the basic language and consider an implementation of our flashing LED program for the Raspberry Pi implemented in pure Erlang. ERLang runs on most operating systems and is open source now. It is easy to add to the Raspberry Pi and is a very good system if you want to make use of a cluster of Raspberry Pis.
How to Run the Program
Erlang doesn’t come pre-installed with Raspbian, but it’s easy to add with the command:
sudo apt-get install erlang
After this completes you are good to go.
Erlang includes a terminal based command system where you can compile modules and call functions. To run the programs included here, you need to save the first file as lights.erl and the second as gpio.erl. Then you can compile and execute the program as indicated in the screenshot of my terminal window below:
Some things to note:
- Erlang is case sensitive and case has meaning. For instance variables start with a capital letter and functions with a lowercase letter.
- Punctuation is very important. The periods end a statement and in the erl shell will cause it to execute. If you miss the period, nothing will happen until you enter one (it assumes you have more text to enter). Similarly inside functions , and ; have a specific meaning that affects how things run.
- You do everything by calling functions both in the shell and in the Erlang language. So the c() function compiles a module (produces a .beam file). The q() function exits the shell after terminating all programs. lights:flashinglights() is our exported entry point function to run the program with. You can also call things like ls() to get a directory listing or cd() to change directories or pwd() to find out where you are. Remember to enter any lines with a period to terminate the line.
- To access the gpio /sys files, erl must be run from sudo. You could fix the file system permissions, but this seems easy enough.
Flashing LED Program
Below is my main module to flash the lights. Unlike the C or Fortran version of this program, there is no loop, since loops don’t exist in Erlang. Instead it uses recursion to accomplish the same things. (Actually in the Erlang tutorials there are design patterns on how to accomplish while or for loops with recursion). Notice that once a variable is assigned it can never be changed. But you accomplish the same thing with recursion by passing a modified version of the variable into a function. Similarly you can preserve variables using function closures, but I don’t do that here. I included edoc comments which are Erlang version of JavaDoc. Otherwise this is just intended to give a flavour for the language without going into too much detail.
%% @author Stephen Smith %% @copyright 2018 Stephen Smith %% @version 1.0.0 %% @doc %% A erlang implementation of my flashing lights program %% for the Raspberry Pi. %% @end -module(lights). -export([flashlights/0]). -author('Stephen Smith'). flashlights() -> Leds = init(), flash(Leds, 10). init() -> L0 = gpio:init(17, out), L1 = gpio:init(27, out), L2 = gpio:init(22, out), {L0, L1, L2}. flash(Leds, Times) when Times > 0 -> gpio:write(element(1, Leds), 1), timer:sleep(200), gpio:write(element(1, Leds), 0), gpio:write(element(2, Leds), 1), timer:sleep(200), gpio:write(element(2, Leds), 0), gpio:write(element(3, Leds), 1), timer:sleep(200), gpio:write(element(3, Leds), 0), flash(Leds, Times-1); flash(Leds, Times) when Times =< 0 -> true.
Erlang GPIO Library
Rather than write the file access library for the GPIO drivers myself, doing a quick Google search revealed several existing ones including this one from Paolo Oliveira.
%% @author Paolo Oliveira <paolo@fisica.ufc.br> %% @copyright 2015-2016 Paolo Oliveira (license MIT) %% @version 1.0.0 %% @doc %% A simple, pure erlang implementation of a module for %% <b>Raspberry Pi's General Purpose %% Input/Output</b> (GPIO), using the standard Linux kernel %% interface for user-space, sysfs, %% available at <b>/sys/class/gpio/</b>. %% @end -module(gpio). -export([init/1, init/2, handler/2, read/1, write/2, stop/1]). -author('Paolo Oliveira <paolo@fisica.ufc.br>'). %% API % @doc: Initialize a Pin as input or output. init(Pin, Direction) -> Ref = configure(Pin, Direction), Pid = spawn(?MODULE, handler, [Ref, Pin]), Pid. % @doc: A shortcut to initialize a Pin as output. init(Pin) -> init(Pin, out). % @doc: Stop using and release the Pin referenced as file descriptor Ref. stop(Ref) -> Ref ! stop, ok. % @doc: Read from an initialized Pin referenced as the file descriptor Ref. read(Ref) -> Ref ! {recv, self()}, receive Msg -> Msg end. % @doc: Write value Val to an initialized Pin referenced as the file descriptor Ref. write(Ref, Val) -> Ref ! {send, Val}, ok. %% Internals configure(Pin, Direction) -> DirectionFile = "/sys/class/gpio/gpio" ++ integer_to_list(Pin) ++ "/direction", % Export the GPIO pin {ok, RefExport} = file:open("/sys/class/gpio/export", [write]), file:write(RefExport, integer_to_list(Pin)), file:close(RefExport), % It can take a moment for the GPIO pin file to be created. case filelib:is_file(DirectionFile) of true -> ok; false -> receive after 1000 -> ok end end, {ok, RefDirection} = file:open(DirectionFile, [write]), case Direction of in -> file:write(RefDirection, "in"); out -> file:write(RefDirection, "out") end, file:close(RefDirection), {ok, RefVal} = file:open("/sys/class/gpio/gpio" ++ integer_to_list(Pin) ++ "/value", [read, write]), RefVal. release(Pin) -> {ok, RefUnexport} = file:open("/sys/class/gpio/unexport", [write]), file:write(RefUnexport, integer_to_list(Pin)), file:close(RefUnexport). % @doc: Message passing interface, should not be used directly, it is present for debugging purpose. handler(Ref, Pin) -> receive {send, Val} -> file:position(Ref, 0), file:write(Ref, integer_to_list(Val)), handler(Ref, Pin); {recv, From} -> file:position(Ref, 0), {ok, Data} = file:read(Ref, 1), From ! Data, handler(Ref, Pin); stop -> file:close(Ref), release(Pin), ok end. %% End of Module.
Summary
Erlang is a very interesting language. If you are interested in functional programming or how to create highly scalable reliable web servers, then Erlang is definitely worth checking out. We only looked at a quick introduction to the language, really by example. There are lots of good documentation, tutorials and samples just a Google search away. Perhaps in a future article we’ll look at processes and messages and how to build such a highly scalable server.
Kali Linux on the Raspberry Pi
Introduction
Raspbian is the main operating system for the Raspberry Pi, but there are quite a few alternatives. Raspbian is based on Debian Linux and there has been a good uptake on the Raspberry Pi which means that most Linux applications have ARM compiled packages available through the Debian APT package manager. As a consequence it’s quite easy to create a Raspberry Pi Linux distribution, so there are quite a few of them. Kali Linux is a specialist distribution that is oriented to hackers (both black and white hat). It comes with a large number of hacking tools for gaining access to networks, compromising computers, spying on communications and other fun things. One cool thing is that Kali Linux has a stripped down version for the Raspberry Pi that is oriented towards a number of specialized purposes. However with the apt-get package manager you can add pretty well anything that has been left out.
If you watch the TV show Mr. Robot (highly recommended) then you might notice that all the cool in-the-know people are running Kali Linux. If you want to get a taste of what it’s all about and you have a Raspberry Pi then all you need is a free micro-SD card to give it a try.
Are You a Black or White Hat?
If you are a black hat hacker looking to infiltrate or damage another computer system, then you probably aren’t reading this blog. Instead you are somewhere on the darknet reading much more malicious articles than this one. This article is oriented more to white hat hackers or to system administrators looking to secure their computing resources. The reason it’s important for system administrators to know about this stuff is that they need to know what they are really protecting against. Hackers are very clever and are always coming up with new hacks and techniques. So it’s important for the good guys to know a bit about how hackers think and to have defenses and protections against the imaginative attacks that might come their way. This now includes the things the bad guys might try to do with a Raspberry Pi.
Anyone that is responsible for securing a network or computer has to test their security and certainly one easy way to get started is to hit it with all the exploit tools included with Kali Linux.
Why Kali on the Pi?
A lot of hacking tasks like cracking WiFi passwords take a lot of processing power. Cracking WPA2 passwords is usually done on very powerful computers with multiple GPU cards and very large dictionary databases. Accomplishing this on a Raspberry Pi would pretty much take forever if it could actually do it. Many hacking tasks are of this nature, very compute intensive.
The Raspberry Pi is useful due to its low cost and small size. The low cost makes it disposable, if you lose it then it doesn’t matter so much and the small size means you can hide it easily. So for instance one use would be to hide a Raspberry Pi at the site you are trying to hack. Then the Raspberry Pi can monitor the Wifi traffic looking for useful data packets that can give away information. Or even leave the Pi somewhere hidden connected to a hardwired ethernet connection. Then Kali Linux has tools to get this information to external sources in a secretive way and allows you to remotely control it to direct various attacks.
Many companies build their security like eggs with a hard to penetrate shell and often locating a device on their premises can bypass their main security protections. You can then run repeated metasploit attacks looking for weaknesses from the inside. Remember your security should be more like an onion with multiple nested layers, so getting through one doesn’t give an attacker access to everything.
Installing Kali Linux
The Kali Linux web site includes a complete disk image of the Raspberry Pi version. You just need to burn this to a micro-SD card using a tool like ApplePi Baker. They you just put the micro-SD in your Raspberry Pi, turn it on and off you go. However there are a few necessary steps to take before you really start:
- The root password is toor, so change this first time you boot up.
- The Kali Linux instructions point out you need to refresh your SSH certificates since otherwise you get the ones included with the image. The download page has instructions on how to do this.
- The image is configured for 8Gig so if you have a larger SD card then you need to repartition it to get all the free space. I used the GParted program for this which I got via “apt-get install gparted”. Note that to use apt-get you need to connect to WiFi or the Internet. Another option is to get Raspbian’s configuration program and use that, it works with most variants of Debian Linux and allows you to do some other things like setup overclocking. You can Google for the wget command to get this.
- Update the various installed programs via “apt-get update” and “apt-get upgrade”. (If you aren’t still logged on as root you need to sudo these).
Now you are pretty much ready to play. Notice that you are in a nice graphical environment and that the application menu is full of hacking tools. These aren’t as many hacking tools as the full Kali distribution, but these are all ones that work well on the Raspberry Pi. They als limited the number so you can run off a really cheap 8 Gig micro-SD card.
I see people say you should stick to command line versions of the tools on the Pi due to its processing power and limited memory, but I found I could add the GUI versions and these worked fine. For instance the nmap tool is installed, but the zenmap graphical front end isn’t. Adding zenmap is easy via “apt-get install zenmap” and then this seems to work fine. I think the assumption is that most people will use the Raspberry version of Kali headless (meaning no screen or keyboard) so it needs to be accessed via remote control software like secure shell which means you want to avoid GUIs.
Summary
Installing Kali Linux on a micro-SD card for your Raspberry Pi si a great way to learn about the various tools that hackers can easily use to try and penetrate, spy on or interfere with people’s computers. There are quite a few books on this as well as many great resources on the Web. Kali’s website is a great starting point. Anyway I find it quite eye opening the variety of readily tools and how easy it is for anyone to use them.
Flashing LEDs in Assembler
Introduction
Previously I wrote an article on an introduction to Assembler programming on the Raspberry Pi. This was quite a long article without much of a coding example, so I wanted to produce an Assembler language version of the little program I did in Python, Scratch, Fortran and C to flash three LEDs attached to the Raspberry Pi’s GPIO port on a breadboard. So in this article I’ll introduce that program.
This program is fairly minimal. It doesn’t do any error checking, but it does work. I don’t use any external libraries, and only make calls to Linux (Raspbian) via software interrupts (SVC 0). I implemented a minimal GPIO library using Assembler Macros along with the necessary file I/O and sleep Linux system calls. There probably aren’t enough comments in the code, but at this point it is fairly small and the macros help to modularize and explain things.
Main Program
Here is the main program, that probably doesn’t look structurally that different than the C code, since the macro names roughly match up to those in the GPIO library the C function called. The main bit of Assembler code here is to do the loop through flashing the lights 10 times. This is pretty straight forward, just load 10 into register r6 and then decrement it until it hits zero.
@ @ Assembler program to flash three LEDs connected to the @ Raspberry Pi GPIO port. @ @ r6 - loop variable to flash lights 10 times @ .include "gpiomacros.s" .global _start @ Provide program starting address to linker _start: GPIOExport pin17 GPIOExport pin27 GPIOExport pin22 nanoSleep GPIODirectionOut pin17 GPIODirectionOut pin27 GPIODirectionOut pin22 @ setup a loop counter for 10 iterations mov r6, #10 loop: GPIOWrite pin17, high nanoSleep GPIOWrite pin17, low GPIOWrite pin27, high nanoSleep GPIOWrite pin27, low GPIOWrite pin22, high nanoSleep GPIOWrite pin22, low @decrement loop counter and see if we loop subs r6, #1 @ Subtract 1 from loop register setting status register bne loop @ If we haven't counted down to 0 then loop _end: mov R0, #0 @ Use 0 return code lsl R0, #2 @ Shift R0 left by 2 bits (ie multiply by 4) mov R7, #1 @ Service command code 1 terminates this program svc 0 @ Linus command to terminate program pin17: .asciz "17" pin27: .asciz "27" pin22: .asciz "22" low: .asciz "0" high: .asciz "1"
GPIO and Linux Macros
Now the real guts of the program are in the Assembler macros. Again it isn’t too bad. We use the Linux service calls to open, write, flush and close the GPIO device files in /sys/class/gpio. Similarly nanosleep is also a Linux service call for a high resolution timer. Note that ARM doesn’t have memory to memory or operations on memory type instructions, so to do anything we need to load it into a register, process it and write it back out. Hence to copy the pin number to the file name we load the two pin characters and store them to the file name memory area. Hard coding the offset for this as 20 isn’t great, we could have used a .equ directive, or better yet implemented a string scan, but for quick and dirty this is fine. Similarly we only implemented the parameters we really needed and ignored anything else. We’ll leave it as an exercise to the reader to flush these out more. Note that when we copy the first byte of the pin number, we include a #1 on the end of the ldrb and strb instructions, this will do a post increment by one on the index register that holds the memory location. This means the ARM is really very efficient in accessing arrays (even without using Neon) we combine the array read/write with the index increment all in one instruction.
If you are wondering how you find the Linux service calls, you look in /usr/include/arm-linux-gnueabihf/asm/unistd.h. This C include file has all the function numbers for the Linux system calls. Then you Google the call for its parameters and they go in order in registers r0, r1, …, r6, with the return code coming back in r0.
@ Various macros to access the GPIO pins @ on the Raspberry Pi. @ R5 is used for the file descriptor .macro openFile fileName ldr r0, =\fileName mov r1, #01 @ O_WRONLY mov r7, #5 @ 5 is system call number for open svc 0 .endm .macro writeFile buffer, length mov r0, r5 @ file descriptor ldr r1, =\buffer mov r2, #\length mov r7, #4 @ 4 is write svc 0 .endm .macro flushClose @fsync syscall mov r0, r5 mov r7, #118 @ 118 is flush svc 0 @close syscall mov r0, r5 mov r7, #6 @ 6 is close svc 0 .endm @ Macro nanoSleep to sleep .1 second @ Calls Linux nanosleep entry point which is function 162. @ Pass a reference to a timespec in both r0 and r1 @ First is input time to sleep in seconds and nanoseconds. @ Second is time left to sleep if interrupted (which we ignore) .macro nanoSleep ldr r0, =timespecsec ldr r1, =timespecsec mov r7, #162 @ 162 is nanosleep svc 0 .endm .macro GPIOExport pin openFile gpioexp mov r5, r0 @ save the file descriptor writeFile \pin, 2 flushClose .endm .macro GPIODirectionOut pin @ copy pin into filename pattern ldr r1, =\pin ldr r2, =gpiopinfile add r2, #20 ldrb r3, [r1], #1 @ load pin and post increment strb r3, [r2], #1 @ store to filename and post increment ldrb r3, [r1] strb r3, [r2] openFile gpiopinfile writeFile outstr, 3 flushClose .endm .macro GPIOWrite pin, value @ copy pin into filename pattern ldr r1, =\pin ldr r2, =gpiovaluefile add r2, #20 ldrb r3, [r1], #1 @ load pin and post increment strb r3, [r2], #1 @ store to filename and post increment ldrb r3, [r1] strb r3, [r2] openFile gpiovaluefile writeFile \value, 1 flushClose .endm .data timespecsec: .word 0 timespecnano: .word 100000000 gpioexp: .asciz "/sys/class/gpio/export" gpiopinfile: .asciz "/sys/class/gpio/gpioxx/direction" gpiovaluefile: .asciz "/sys/class/gpio/gpioxx/value" outstr: .asciz "out" .align 2 @ save users of this file having to do this. .text
Makefile
Here is a simple makefile for the project if you name the files as indicated. Again note that WordPress and Google Docs may mess up white space and quote characters so these might need to be fixed if you copy/paste.
model: model.o ld -o model model.o model.o: model.s gpiomacros.s as -ggdb3 -o model.o model.s clean: rm model model.o
IDE or Not to IDE
People often do Assembler language development in an IDE like Code::Blocks. Code::Blocks doesn’t support Assembler language projects, but you can add Assembler language files to C projects. This is a pretty common way to do development since you want to do more programming in a higher level language like C. This way you also get full use of the C runtime. I didn’t do this, I just used a text editor, make and gdb (command line). This way the above program has no extra overhead the executable is quite small since there is no C runtime or any other library linked to it. The debug version of the executable is only 2904 bytes long and non debug is 2376 bytes. Of course if I really wanted to reduce executable size, I could have used function calls rather than Assembler macros as the macros duplicate the code everywhere they are used.
Summary
Assembler language programming is kind of fun. But I don’t think I would want to do too large a project this way. Hats off to the early personal computer programmers who wrote spreadsheet programs, word processors and games entirely in Assembler. Certainly writing a few Assembler programs gives you a really good understanding of how the underlying computer hardware works and what sort of things your computer can do really efficiently. You could even consider adding compiler optimizations for your processor to GCC, after all compiler code generation has a huge effect on your computer’s performance.
Raspberry Pi Assembly Programming
Introduction
Most University Computer Science programs now do most of their instruction in quite high level languages like Java which don’t require you to need to know much about the underlying computer architecture. I think this tends to create quite large gaps in understanding which can lead to quite bad program design mistakes. Learning to program C puts you closer to the metal since you need to know more about memory and low level storage, but nothing really teaches you how the underlying computer works than doing some Assembler Language Programming, The Raspbian operating system comes with the GNU Assembler pre-installed, so you have everything you need to try Assembler programming right out of the gate. Learning a bit of Assembler teaches you how the Raspberry Pi’s ARM processor works, how a modern RISC architecture processes instructions and how work is divided by the ARM CPU and the various co-processors which are included on the same chip.
A Bit of History
The ARM processor was originally developed to be a low cost processor for the British educational computer, the Acorn. The developers of the ARM felt they had something and negotiated a split into a separate company selling CPU designs to hardware manufacturers. Their first big sale was to Apple to provide a CPU for Apple’s first PDA the Newton. Their first big success was the inclusion of their chip design in Apple’s iPods. Along the way many chip makers like TI which had given up competing on CPUs built single chip computers around the ARM. These ended up being included in about every cell phone including those from Nokia and Apple. Nowadays pretty up every Android phone is also built around ARM designs.
ARM Assembler Instructions
There have been a lot of ARM designs from early simple 16 bit processors up through 32 bit processors to the current 64bit designs. In this article I’m just going to consider the ARM processor in the Raspberry Pi, this processor is a 64 bit processor, but Raspbian is still a 32 bit operating system, so I’ll just be talking about the 32 bit processing used here (and I’ll ignore the 16 bit “thumb” instructions for now).
ARM is a RISC processor which means the idea is that it executes very simple instructions very quickly. The idea is to keep the main processor simple to reduce complexity and power usage. Each instruction is 32 bits long, so the processor doesn’t need to think about how much to increment the program counter for each instruction. Interestingly nearly all the instructions are in the same format, where you can control whether it sets the status bits, can test the status bits on whether to do anything and can have four register parameters (or 2 registers and an immediate constant). One of the registers can also have a shift operation applied. So how is all this packed into on 32 bit instruction? There are 16 registers in the ARM CPU (these include the program counter, link return register and the stack pointer. There is also the status register that can’t be used as a general purpose register. This means it takes 4 bits to specify a register, so specifying 4 registers takes 16 bits out of the instruction.
Below are the formats for the main types of ARM instructions.
Now we break out the data processing instructions in more detail since these comprise quite a large set of instructions and are the ones we use the most.
Although RISC means a small set of simple instructions, we see that by cleverly using every bit in those 32 bits for an instruction that we can pack quite a bit of information.
Since the ARM is a 32-Bit processor meaning among other things that it can address a 32-Bit address space this does lead to some interesting questions:
- The processor is 32-Bits but immediate constants are 16-Bits. How do we load arbitrary 32-Bit quantities? Actually the immediate instruction is 12-Bits of data and 4 Bits of a shift amount. So we can load 12 Bits and then shift it into position. If this works great. If not we have to be tricky by loading and adding two quantities.
- Since the total instruction is 32-Bits how do we load a 32-Bit memory address? The answer is that we can’t. However we can use the same tricks indicated in number 1. Plus you can use the program counter. You can add an immediate constant to the program counter. The assembler often does this when assembling load instructions. Note that since the ARM is pipelined the program counter tends to be a couple of instructions ahead of the current one, so this has to properly be taken into account.
- Why is there the ability to conditionally execute all the data processing instructions? Since the ARM processor is pipelines, doing a branch instruction is quite expensive since the pipeline needs to be discarded and then reloaded at the new execution point. Having individual instructions conditionally execute can save quite a few branch instructions leading to faster execution. (As long as your compiler is good at generating RISC type code or you are hand coding in Assembler).
- The ARM processor has a multiply instruction, but no divide instruction? This seems strange and unbalanced. Multiply (and multiply and add) are recent additions to the ARM processor. Divide is still considered too complicated and slow, plus is it really used that much? You can do divisions on either the floating point coprocessor or the Neon SIMD coprocessor.
A Very Small Example
Assembler listings tend to be quite long, so as a minimal set let’s start with a program that just exits when run returning 17 to the command line (you can see this if you type echo $@ after running it). Basically we have an assembler directive to define the entry point as a global. We move the immediate value 17 to register R0. Shift it left by 2 bits (Linux expects this). Move 1 to register R7 which is the Linux command code to exit the program and then call service zero which is the Linux operating system. All calls to Linux use this pattern. Set some parameters in registers and then call “svc 0” to do the call. You can open files, read user input, write to the console, all sorts of things this way.
.global _start @ Provide program starting address to linker _start: mov R0, #17 @ Use 17 for test example lsl R0, #2 @ Shift R0 left by 2 bits (ie multiply by 4) mov R7, #1 @ Service command code 1 terminates this program svc 0 @ Linux command to terminate program
Here is a simple makefile to compile and link the preceding program. If you save the above as model.s and then the makefile as makefile then you can compile the program by typing “make” and run it by typing “./model”. as is the GNU-Assembler and ld is the linker/loader.
model: model.o ld -o model model.o model.o: model.s as -g -o model.o model.s clean: rm model model.o
Note that make is very particular about whitespace. It must be a true “tab” character before the commands to execute. If Google Docs or WordPress changes this, then you will need to change it back. Unfortunately word processors have a bad habit of introducing syntax errors to program listings by changing whitespace, quotes and underlines to typographic characters that compilers (and assemblers) don’t recognize.
Summary
Although these days you can do most things with C or a higher level language, Assembler still has its place in certain applications that require performance or detailed hardware interaction. For instance perhaps tuning the numpy Python library to use the Neon coprocessor for faster operation of vector type operations. I do still think that every programmer should spend some time playing with Assembler language so they understand better how the underlying processor and architecture they use day to day really works. The Raspberry Pi offers and excellent environment to do this with the good GNU Macro Assembler, the modern ARM RISC architecture and the various on-chip co-processors.
Just the Assembler introductory material has gotten fairly long, so we won’t get to an assembler version of our flashing LED program. But perhaps in a future article.
Playing with my Raspberry Pi
Introduction
I do most of my work (like writing this blog posting) on my MacBook Air laptop. I used to have a good desktop computer for running various longer running processes or playing games. Last year the desktop packed it in (it was getting old anyway), so since then I’ve just been using my laptop. I wondered if I should get another desktop and run Ubuntu on it, since that is good for machine learning, but I wondered if it was worth price. Meanwhile I was intrigued with everything I see people doing with Raspberry Pi’s. So I figured why not just get a Raspberry Pi and see if I can do the same things with it as I did with my desktop. Plus I thought it would be fun to learn about the Pi and that it would be a good toy to play with.
Setup
Since I’m new to the Raspberry Pi, I figured the best way to get started was to order one of the starter kits. This way I’d be able to get up and running quicker and get everything I needed in one shot. I had a credit with Amazon, so I ordered one of the Canakits from there. It included the Raspberry Pi 3, a microSD card with Raspbian Linux, a case, a power supply, an electronics breadboard, some leds and resistors, heat sinks and an HDMI cable. Then I needed to supply a monitor, a USB keyboard and a USB mouse (which I had lying around).
Setting up was quite easy, though the quick setup instructions were missing a few steps like what to do with the heatsinks (which was obvious) or how to connect the breadboard. Setup was really just install the Raspberry Pi motherboard in the case, add the heat sinks, insert the microSD card and then connect the various cables.
As soon as I powered it on, it displayed an operating system selection and installation menu (with only one choice), so clicked install and 10 minutes later I was logged in and running Raspbian.
The quick setup guide then recommends you set your locale and change the default password, but they don’t tell you the existing password, which a quick Google reveals as “Raspberry”. Then I connected to our Wifi network and I was up and running. I could browse the Internet using Chromium, I could run Mathematica (a free Raspberry version comes pre-installed), run a Linux terminal session. All rather painless and fairly straight forward.
I was quite impressed how quickly it went and how powerful a computer I had up and running costing less than $100 (for everything) and how easy the installation and setup process was.
Software
I was extremely pleased with how much software the Raspberry Pi came with pre-installed. This was all on the provided 32Gig card, which with a few extra things installed, I still have 28Gig free. Amazingly compact. Some of the pre-installed software includes:
- Mathematica. Great for Math students and to promote Mathematica. Runs from the Wolfram Language which is interesting in itself.
- Python 2 and 3 (more on the pain of having Python 2 later).
- LibreOffice. A full MS Office like suite of programs.
- Lots of accessories like file manager, calculator, image viewer, etc.
- Chromium web browser.
- Two Java IDEs.
- Sonic Pi music synthesizer.
- Terminal command prompt.
- Minecraft and some Python games.
- Scratch programming environment.
Plus there is an add/remove software program where you can easily add many more open source Pi programs. You can also use the Linux apt-get command to get many other pre-compiled packages.
Generally I would say this is a very complete set of software for any student, hobbyist or even office worker.
Python
I use Python as my main goto programming language these days and generally I use a number of scientific and machine learning libraries. So I tried installing these. Usually I just use pip3 and away things go (at least on my Mac). However doing this caused pip3 to download the C++/Fortran source code and to try to compile it, which failed. I then Googled around on how to best install these packages.
Unfortunately most of the Google results were how to do this for Python 2, which I didn’t want. It will be so nice when Python 2 finally is discontinued and stops confusing everything. I wanted these for Python 3. Before you start you should update apt-get’s list of available software and upgrade all the packages on your machine. You can do this with:
sudo apt-get update # Fetches the list of available updates
sudo apt-get upgrade # Strictly upgrades the current packages
What I found is I could get most of what I wanted using apt-get. I got most of what I wanted with:
sudo apt-get install python3-numpy
sudo apt-get install python3-scipy
sudo apt-get install python3-matplotlib
sudo apt-get install python3-pandas
However I couldn’t find and apt-get module for SciKit Learn the machine learning library. So I tried pip3 and it did work even though it downloaded the source code and compiled it.
pip3 install sklearn –upgrade
Now I had all the scientific programming power of the standard Python libraries. Note that since the Raspberry Pi only has 1Gig RAM and the SD Card only has twenty something Gig free, you can’t really run large machine learning tasks. However if they do fit within the Pi then it is a very inexpensive way to do these computations. What a lot of people do is build clusters of Raspberry Pi’s that work together. I’ve seen articles on how University labs have built supercomputers out of hundreds or Pi’s all put together in a cluster. Further they run quite sophisticated software like Hadoop, Docker and Kubernetes to orchestrate the whole thing.
Summary
I now have the Raspberry Pi up and running and I’m enjoying playing with Mathematica and Sonic Pi. I’m doing a bit of Python programming and browsing the Internet. Quite an amazing little device. I’m also impressed with how much it can do for such a low cost. As other vendors like Apple, Microsoft, HP and Dell try to push people into more and more expensive desktops and laptops, it will be interesting to see how many people revolt and switch to the far more inexpensive DIY type solutions. Note that there are vendors that make things like Raspberry Pi complete desktop computers at quite a low cost as well.
