# Stephen Smith's Blog

Musings on Machine Learning…

with one comment

# Introduction

Last time we looked at a simple physical dynamical system namely Taylor Couette Fluid Flow, where a very simple experiment led to more and more complicated solutions as a parameter was varied, namely the speed of the inner cylinder. In this article we are going to look at an example from Mathematics and an example from Computer Science. What we are interested in is how very complicated behaviour results from very simply stated problems. We are looking into insights in how something as complicated as human intelligence can result from the simple behaviour of billions of our neural cells. Or for that matter can we have intelligence arise from the simple behaviour of billions of logic gates in a modern computer? The amazing thing is that as we study nature we see this phenomena more and more, whether it’s fractals appearing in nature or the chaotic behaviour of ecological systems. It turns out that much of the richness and complexity of our environment can result from a few very simple rules.

# The Mandelbrot Set

Everyone has seen fantastic images of the Mandelbrot Set. Such as:

Or a zoom in:

To see more detail along with more fabulous graphics, have a look at the WikiPedia article here.

The definition of the Mandelbrot Set is remarkably simple. It is the set of complex numbers c such that the quadratic map

Zn+1 = Zn^2 + c

converges. In the graphics black represents the convergent points, and then the colors are specified by how fast a point diverges. The Mandelbrot Set is a fractal, meaning that as you zoom in on it you see the same structures recurring at all magnifications. The key point is that we get this infinite complexity out of such a simple defining equation. We are used to simple formulas like quadratics leading to simple predictable behaviour like parabolas. However once you start iterating simple behaviours you start getting this amazingly rich complexity appearing.

# The Game of Life

Another well known example of complex behaviour from very simple rules is John Conway’s Game of Life. The definition of the game is quite simple, so I’ll just quote the WikiPedia entry:

“The universe of the Game of Life is an infinite two-dimensional orthogonal grid of square cells, each of which is in one of two possible states, alive or dead, or “populated” or “unpopulated”. Every cell interacts with its eight neighbours, which are the cells that are horizontally, vertically, or diagonally adjacent. At each step in time, the following transitions occur:

1. Any live cell with fewer than two live neighbours dies, as if caused by underpopulation.
2. Any live cell with two or three live neighbours lives on to the next generation.
3. Any live cell with more than three live neighbours dies, as if by overpopulation.
4. Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction.

The initial pattern constitutes the seed of the system. The first generation is created by applying the above rules simultaneously to every cell in the seed—births and deaths occur simultaneously, and the discrete moment at which this happens is sometimes called a tick (in other words, each generation is a pure function of the preceding one). The rules continue to be applied repeatedly to create further generations.”

For lots of examples and images have a look at the full WikiPedia article here. Below is a gif showing a set pattern spawning gliders that fly off diagonally:

Some games just die out, others are extremely chaotic. Some people tried to “solve” the Game of Life, ie given an initial condition have a formula that predicts what will happen without having to run the full simulation. But is was shown that you can actually create an Universal Turing Machine in the game and hence solving this is equivalent to the Halting Problem and hence is impossible.

# Summary

The Mandelbrot Set is one example of many where a very simple problem statement leads to infinitely complex solutions. The Game of Life shows how another simple statement leads to extremely complex and unpredictable behaviour. Further since you can create a Universal Turing Machine in the Game of Life, Turing’s Completeness theorem shows you could solve any problem computation using the Game of Life. This partly shows how Turing’s work is so influential and how so many things that we may not think of a computers, are in fact equivalent to computers.

Our brain consists of billions of neurons that have very simple rules that then get applied over and over in an iterative manner, so as we’ve seen this will lead to very complicated, rich and stable patterns emerging. Our thesis then is that this is the foundation of intelligence.

Written by smist08

June 2, 2017 at 6:27 pm

# Introduction

Back in 2013 I purchased a MacBook Air, installed XCode and wrote a small Objective-C program to draw a simple fractal on an iPad. Which I then blogged on here. Now we are a few years later and I thought I would give Apple’s new programming language Swift a try and see how iOS/OSX development has evolved as a result. For more details on Koch snowflakes and what I the program does, check out my original article.

# The Evolution from Objective-C to Swift

Objective-C was one of the first object oriented extensions to the C Programming language. Its first main usage was by Steve Jobs and NeXT Computer as the primary programming language of the NeXTStep operating system which later became OS/X and iOS. Objective-C had a lot of innovative ideas behind it like treating everything as sending messages between objects (rather than directly calling methods). But then C++ came along and became the main standard for an object oriented extension to C.

One of the complaints against C is that it puts a lot of burden on programmers since they are dealing with memory and the computer architecture at a very low level. You are manipulating memory pointers directly, allocating memory buffers, etc. This is all very powerful and produces very fast, compact and efficient programs. But there is a lot of room for error, since making a mistake here will lead to buffer overruns, program crashes and such. In the days of standalone computers this was annoying but not fatal. Now with the internet, these sorts of problems lead to security vulnerabilities and server crashes. All that being said, if you have skilled programmers, C, Objective C and C++ are very powerful and you can produce great reliable programs with them.

To address these problems, Sun Microsystems invented Java. Java was essentially an object oriented extension of C, but with all the pointers and low level memory access removed. Java then included a large standard class library to give an alternate way of doing all the low level things you did in C. Java compiled to P-Code which ran on a Java Virtual Machine. This could then be sandboxed to allow greater security. To some degree this was to try to reach a compromise between scripting languages like JavaScript or VBScript and true fully compiled languages like C++. I.e. to make it easier to program with less gotchas, but still maintain the compiler checks for correctness and modules features required to program large systems.

Microsoft saw the potential and growing success of Java and came up with their own competing system namely C#. C# was initially very similar to Java with a very similar class library. Microsoft actually originally had their own implementation of Java, but it really sucked and it was easier to move true Java programs to C# than it was to Microsoft Java. Similar to the Java VM, C# runs on Microsoft’s .Net framework which isolates you from the underlying operating system.

Java got off to a great start, but as Sun workstations went into decline, Sun couldn’t put the necessary R&D resources into supporting Java and forward progress slowed. Oracle bought Sun and took over Java, but Oracle doesn’t seem to be putting much effort into Java, besides suing the various users of it like Google.

Microsoft has been doing a lot of good work developing C# and has been putting a lot of work to evolving the language and evolving the .Net framework. Certainly modern C# has come a long way and contains a lot of powerful modern object oriented features that weren’t present initially and aren’t present in Java.

A couple of years ago Apple finally noticed this trend and produced their own modern object oriented language namely Swift. Swift isn’t a true object oriented extension to C, the core language has a lot of differences to C. Some things are quite similar like building expressions, but other things are quite different, like how you define variables. Swift has all the modern object oriented features like closures, extensions, generics, etc. which you would expect. Further since a lot of the language was re-imagined over C, it has a lot of nice built in features like ranges. If you look at just the core language, its quite clean, powerful and modern.

There are quite a few blog posts comparing these various languages such as these two articles on C# vs Swift: C# vs Swift and C# vs Swift. If you Google, there are a lot of discussions on the various points of these languages. Often the discussions also consider Go and Python.

# Frameworks

The ugliness in all these safe modern languages comes in with how they interact with the underlying operating system. Neither Apple nor Microsoft re-wrote their operating systems to be safe and natively support these. At some point you have the transition from the nice safe, clean object oriented world into the old pointer based C world. Microsoft with the underlying Windows DLLs and Apple with the Objective-C based application frameworks and then to the underlying Unix based operating system kernel.

Sun took the highest approach making its own frameworks for everything and then leaving it to the JVM implementation on each system to translate native to this, so to a Java programmer everything looks the same. This sounds great, but doesn’t work well in practice since it doesn’t give you access to all the operating system features and makes your program less competitive. This resulted in the development of JNI and Java programs natively calling through to the ugly world outside the JVM.

Microsoft built the .Net framework on top of Windows, which provides most things you need and has been filling in more and more. But you still often need to call native DLLs directly (which makes you application unsafe).

Apple decided to use the current iOS/OSX frameworks directly and allowed Swift to interact bi-directionally with Objective-C libraries. This then allowed Swift programmers to directly leverage their knowledge of UIKit for instance to write programs. The downside of this is that it puts a lot of ugly code directly in your nice clean Swift program to deal with these older frameworks.

# Koch Snowflakes Revisited

I ported my Objective-C Koch Snowflake program from 2013 over to Swift. This turned out to be pretty straight forward. I think the program source code is much cleaner once moved over to Swift and I definitely prefer Swift to Objective-C for programming. Since I’ve been doing mostly C# programming the past few years, it fells much more natural to me than Objective-C.

Although most of the code is cleaner, you can see a bit of ugliness around the interactions with the UIKit framework. I especially don’t like using the types their rather than the native Swift data types.

# Other Development Notes

For the UI, I used the standard storyboard screen designer which is shared by both Objective-C and Swift. Like most systems that edit your code, you just need to be careful not to edit the code inserted by the UI designer or they get out of sync and produce weird errors. I changed a variable name generated by the UI designer and it was a bit of a head scratcher tracing back from the error message to what was wrong.

I created the project as a standard single page application and set it to run on both an iPhone and iPad. There are now 12 standard devices of various iPad and iPhone models directly supported, I tried a couple of them, but certainly didn’t test with each one.

Generally, when you make a project you can create any new class in either Swift or Objective-C and have them interoperate. So you can bring in older code rather than porting it.

The debugger is quite nice, its easy to step through your code and see what is going on. Generally, XCode is a very powerful development platform and has a lot of great tools to support you in your programming. I haven’t added any unit tests yet, but I plan to have a look at the testing framework next and perhaps that will be the topic of a future blog.

# Summary

I think Swift is a huge improvement for programming iOS and OS/X over Objective-C.  Although Swift is open source and can be run on Linux, the Apple UI frameworks like UIKit are not open source. So I don’t think Swift will be any help in developing cross platform programs (unless they are very simple command line utilities). Swift is quite a modern language and its object oriented implementation is quite nice. Apple seems to be putting quite a bit of effort into Swift with version 3 of the language soon to be released. There is certainly a large community of iOS developers out there who should be putting it to good use.

This was a fun little project and I think I will be spending a bit more time dabbling is iOS development using Swift.

# Source Code Listings

```//
//  ViewController.swift
//  KochSnowFlake
//
//  Created by Stephen Smith on 2016-05-13.
//

import UIKit

class ViewController: UIViewController {

// MARK: Properties
@IBOutlet weak var fractalLevelTextField: UITextField!
@IBOutlet weak var fracView: FractalView!

//

fractalLevelTextField.text = "2";
fracView.level = 2;

selector: #selector(textChangeNot),
}

func textChangeNot( object: AnyObject )
{
if let enteredLevel = NSNumberFormatter().numberFromString(fractalLevelTextField.text!)
{
fracView.level = Int(enteredLevel);
fracView.setNeedsDisplay();
}
}

// Dispose of any resources that can be recreated.
}
}

//
//  FractalView.swift
//  KochSnowFlake
//
//  Created by Stephen Smith on 2016-05-13.
//

import UIKit
class FractalView: UIView {
var level = 1;

// Only override drawRect: if you perform custom drawing.
// An empty implementation adversely affects performance during animation.
override func drawRect(rect: CGRect) {
var frac: KochFlake;

// Drawing code

let currentColor = UIColor.blackColor();
let context = UIGraphicsGetCurrentContext()
frac = KochFlake(inContext: context!);

//Set the width of the "pen" that will be used for drawing
CGContextSetLineWidth(context,1);

//Set the color of the pen to be used
CGContextSetStrokeColorWithColor(context, currentColor.CGColor);

frac.KockSnowflake(level);

//Apply our stroke settings to the line.
CGContextStrokePath(context);

}
}

//
//  KochFlake.swift
//  KochSnowFlake
//
//  Created by Stephen Smith on 2016-05-13.
//

import UIKit

class KochFlake
{
var tg:TurtleGraphics;
var context:CGContextRef;

init(inContext: CGContextRef)
{
context = inContext;
tg = TurtleGraphics(inContext: context);
}

func KockSnowflake(level:Int)
{
tg.turn( 60 );
KockSnowflakeSide( level , size:400);
tg.turn( -120 );
KockSnowflakeSide( level, size: 400);
tg.turn( -120 );
KockSnowflakeSide( level, size: 400);
}

func KockSnowflakeSide(level:Int, size:Int)
{
if (level == 0)
{
tg.move( size );
}
else
{
KockSnowflakeSide( level - 1, size: size / 3 );
tg.turn( 60 );
KockSnowflakeSide( level-1, size: size / 3);
tg.turn( -120 );
KockSnowflakeSide( level-1, size: size / 3);
tg.turn(60);
KockSnowflakeSide( level-1, size: size / 3);
}
}
}

//
//  TurtleGraphics.swift
//  KochSnowFlake
//
//  Created by Stephen Smith on 2016-05-13.
//

import UIKit
let pi:CGFloat = 3.14159;

class TurtleGraphics
{
var x, y: CGFloat;
var angle: CGFloat;
var context: CGContextRef;

init(inContext: CGContextRef)
{
context = inContext;
x = 50.0;
y = 150.0;
CGContextMoveToPoint(context, x, y);
angle = 0.0;
}

func move( dist: Int )
{
x = x + CGFloat(dist) * cos( angle * pi / 180.0);
y = y + CGFloat(dist) * sin( angle * pi / 180.0);