Stephen Smith's Blog

Musings on Machine Learning…

Archive for the ‘Mobility’ Category

UI Testing in Swift

with one comment

Introduction

To round out my blog series on an introduction to Swift, this posting will be covering UI Testing. Previously we created a simple Swift program to draw a Koch Snowflake, adding some unit tests and then added some performance tests.

The source code for the project is on Google Drive here.

UI Testing actually runs the program like an end user would run the program and if you switch to the simulator while the test is running you can watch these actions take place. Unlike many other UI testing frameworks, this one just interacts with the screen controls, if done properly there is no code involving doing things at specific (x,y) co-ordinates. The magic that makes this work is the iOS accessibility layer that was created to help people with disabilities use Apple products. For instance, the VoiceOver feature that reads the screen needs to interact with the controls in the same way as our UI Tests.

This then means that UI Tests also provide a good means for testing some of the accessibility aspects of our iOS applications. Fully supporting accessibility is an often neglected area and really deserves more consideration. The great thing here is that by making your UI Tests thorough you are also validating that many accessibility technologies will also work.

UI Testing in XCode

When you create a new Swift project in XCode and select unit testing you also get a skeletal group for UI Tests with some setup and a dummy test. You create you test by selecting an empty (or not) test and then pressing record and then manually perform the tests. When you close the simulator a bunch of recorded code will be pasted into your project. This then is a great starting point for writing more thorough tests. You then use all the same XCTAssert type functions as in the unit testing framework to check for problems.

Screen Shot 2016-06-15 at 8.41.45 AM

Gotchas

Not Having Accessibility Setup Correctly

If you haven’t set an accessibility identifier for your control, you won’t get the correct code recorded. Recording will try its best, but it will give you something that probably won’t work. This happened to me. I kept the bad code from the first attempt in the file commented out so you can see it. Generally, if the accessibility is setup right, the code is simple complete and will work. If not, you will find things you did not recorded and other things having hardware or synchronicity problems (strange errors which if you google have workarounds but it all becomes quite complicated).

Screen Shot 2016-06-15 at 8.42.06 AM

Keyboards and other Hardware

I performed my tests on my MacBook which of course has a fixed keyboard. When recording tests, make sure you use the iOS keyboard (that is on the screen). Generally, you want the tests to use all the iOS stuff and not the macOS stuff which makes using the simulator easier. Another approach is to access text fields via the clipboard using cut/paste so as to avoid the keyboard entirely. I tend to think for a good UI test you should test all the cases, but perhaps not on every text field. Also beware text already in text boxes that may need to be cleared first. One way to do this (probably the best way) is to add a clear button in the text boxes properties and then press this. In the recorded sample I hit the delete key a couple of times. Note that tapping a field usually doesn’t select all the text.

Synchronization

Beware that if you cause something to popup or be created, chances are your test code will run faster than that and start using things before they are created. You will need to add wait loops to wait for controls to exist before using them. This case doesn’t happen in the Koch snowflake program. Generally, you don’t want to insert sleep type statements to wait a couple of seconds, this slows down your UI tests and can prove unreliable and lead to investigating a lot of false failed tests. Always better to look for specific events and to proceed quickly.

The Test

The code for the test is below. The setUp and teardown methods were generated by XCode and I didn’t change them. The code for the testExample routine was generated by recording, then I just cleaned up a bit of noise. The intent is that it sets fractal level to 3 and then to 4. If you click on the simulator while running, then you can see this happen. Unfortunately, there isn’t really a good way to validate that it works correctly, so this is really only a run without crashing sort of test, unless you manually observe it.

//
//  KochSnowFlakeUITests.swift
//  KochSnowFlakeUITests
//
//  Created by Stephen Smith on 2016-05-13.
//  Copyright © 2016 Stephen Smith. All rights reserved.
//

import XCTest

class KochSnowFlakeUITests: XCTestCase {

    override func setUp() {
        super.setUp()

        // Put setup code here. This method is called before the invocation of each
        // test method in the class.
        // In UI tests it is usually best to stop immediately when a failure occurs.

        continueAfterFailure = false

        // UI tests must launch the application that they test.
        //Doing this in setup will make sure it happens for each test method.
        XCUIApplication().launch()

        // In UI tests it’s important to set the initial state -
        // such as interface orientation - required for your tests before they run.
        // The setUp method is a good place to do this.
    }

    override func tearDown() {
        // Put teardown code here. This method is called after the invocation of
        // each test method in the class.
        super.tearDown()
    }

    func testExample() {

        let app = XCUIApplication()
        app.textFields["textField"].tap()

        let deleteKey = app.keys["delete"]
        deleteKey.tap()
        deleteKey.tap()
        app.textFields["textField"].typeText("3")

        let returnButton = app.buttons["Return"]
        returnButton.tap()

        deleteKey.tap()
        deleteKey.tap()
        app.textFields["textField"].typeText("4")
        returnButton.tap()
    }
}

Summary

The UI testing support built into XCode and Swift is quite nice. Certainly comparable to some quite expensive packages available in the Windows world. Since iOS and macOS are quite a controlled environment and the accessibility support is quite good, this makes this package quite nice. The main thing to watch out for is the proliferation of Apple hardware to check. It appears that going forwards Apple is spending quite a bit of time ensuring automated testing works quite well for their development platform.

Written by smist08

June 15, 2016 at 4:09 pm

Posted in Mobility, programming

Tagged with , , ,

My First Swift Application

with 4 comments

Introduction

Back in 2013 I purchased a MacBook Air, installed XCode and wrote a small Objective-C program to draw a simple fractal on an iPad. Which I then blogged on here. Now we are a few years later and I thought I would give Apple’s new programming language Swift a try and see how iOS/OSX development has evolved as a result. For more details on Koch snowflakes and what I the program does, check out my original article.

The Evolution from Objective-C to Swift

Objective-C was one of the first object oriented extensions to the C Programming language. Its first main usage was by Steve Jobs and NeXT Computer as the primary programming language of the NeXTStep operating system which later became OS/X and iOS. Objective-C had a lot of innovative ideas behind it like treating everything as sending messages between objects (rather than directly calling methods). But then C++ came along and became the main standard for an object oriented extension to C.

One of the complaints against C is that it puts a lot of burden on programmers since they are dealing with memory and the computer architecture at a very low level. You are manipulating memory pointers directly, allocating memory buffers, etc. This is all very powerful and produces very fast, compact and efficient programs. But there is a lot of room for error, since making a mistake here will lead to buffer overruns, program crashes and such. In the days of standalone computers this was annoying but not fatal. Now with the internet, these sorts of problems lead to security vulnerabilities and server crashes. All that being said, if you have skilled programmers, C, Objective C and C++ are very powerful and you can produce great reliable programs with them.

To address these problems, Sun Microsystems invented Java. Java was essentially an object oriented extension of C, but with all the pointers and low level memory access removed. Java then included a large standard class library to give an alternate way of doing all the low level things you did in C. Java compiled to P-Code which ran on a Java Virtual Machine. This could then be sandboxed to allow greater security. To some degree this was to try to reach a compromise between scripting languages like JavaScript or VBScript and true fully compiled languages like C++. I.e. to make it easier to program with less gotchas, but still maintain the compiler checks for correctness and modules features required to program large systems.

Microsoft saw the potential and growing success of Java and came up with their own competing system namely C#. C# was initially very similar to Java with a very similar class library. Microsoft actually originally had their own implementation of Java, but it really sucked and it was easier to move true Java programs to C# than it was to Microsoft Java. Similar to the Java VM, C# runs on Microsoft’s .Net framework which isolates you from the underlying operating system.

Java got off to a great start, but as Sun workstations went into decline, Sun couldn’t put the necessary R&D resources into supporting Java and forward progress slowed. Oracle bought Sun and took over Java, but Oracle doesn’t seem to be putting much effort into Java, besides suing the various users of it like Google.

Microsoft has been doing a lot of good work developing C# and has been putting a lot of work to evolving the language and evolving the .Net framework. Certainly modern C# has come a long way and contains a lot of powerful modern object oriented features that weren’t present initially and aren’t present in Java.

A couple of years ago Apple finally noticed this trend and produced their own modern object oriented language namely Swift. Swift isn’t a true object oriented extension to C, the core language has a lot of differences to C. Some things are quite similar like building expressions, but other things are quite different, like how you define variables. Swift has all the modern object oriented features like closures, extensions, generics, etc. which you would expect. Further since a lot of the language was re-imagined over C, it has a lot of nice built in features like ranges. If you look at just the core language, its quite clean, powerful and modern.

There are quite a few blog posts comparing these various languages such as these two articles on C# vs Swift: C# vs Swift and C# vs Swift. If you Google, there are a lot of discussions on the various points of these languages. Often the discussions also consider Go and Python.

Frameworks

The ugliness in all these safe modern languages comes in with how they interact with the underlying operating system. Neither Apple nor Microsoft re-wrote their operating systems to be safe and natively support these. At some point you have the transition from the nice safe, clean object oriented world into the old pointer based C world. Microsoft with the underlying Windows DLLs and Apple with the Objective-C based application frameworks and then to the underlying Unix based operating system kernel.

Sun took the highest approach making its own frameworks for everything and then leaving it to the JVM implementation on each system to translate native to this, so to a Java programmer everything looks the same. This sounds great, but doesn’t work well in practice since it doesn’t give you access to all the operating system features and makes your program less competitive. This resulted in the development of JNI and Java programs natively calling through to the ugly world outside the JVM.

Microsoft built the .Net framework on top of Windows, which provides most things you need and has been filling in more and more. But you still often need to call native DLLs directly (which makes you application unsafe).

Apple decided to use the current iOS/OSX frameworks directly and allowed Swift to interact bi-directionally with Objective-C libraries. This then allowed Swift programmers to directly leverage their knowledge of UIKit for instance to write programs. The downside of this is that it puts a lot of ugly code directly in your nice clean Swift program to deal with these older frameworks.

Koch Snowflakes Revisited

I ported my Objective-C Koch Snowflake program from 2013 over to Swift. This turned out to be pretty straight forward. I think the program source code is much cleaner once moved over to Swift and I definitely prefer Swift to Objective-C for programming. Since I’ve been doing mostly C# programming the past few years, it fells much more natural to me than Objective-C.

Although most of the code is cleaner, you can see a bit of ugliness around the interactions with the UIKit framework. I especially don’t like using the types their rather than the native Swift data types.

Screen Shot 2016-05-16 at 9.46.04 AM

Other Development Notes

For the UI, I used the standard storyboard screen designer which is shared by both Objective-C and Swift. Like most systems that edit your code, you just need to be careful not to edit the code inserted by the UI designer or they get out of sync and produce weird errors. I changed a variable name generated by the UI designer and it was a bit of a head scratcher tracing back from the error message to what was wrong.

I created the project as a standard single page application and set it to run on both an iPhone and iPad. There are now 12 standard devices of various iPad and iPhone models directly supported, I tried a couple of them, but certainly didn’t test with each one.

Generally, when you make a project you can create any new class in either Swift or Objective-C and have them interoperate. So you can bring in older code rather than porting it.

The debugger is quite nice, its easy to step through your code and see what is going on. Generally, XCode is a very powerful development platform and has a lot of great tools to support you in your programming. I haven’t added any unit tests yet, but I plan to have a look at the testing framework next and perhaps that will be the topic of a future blog.

Screen Shot 2016-05-16 at 9.44.39 AM

Summary

I think Swift is a huge improvement for programming iOS and OS/X over Objective-C.  Although Swift is open source and can be run on Linux, the Apple UI frameworks like UIKit are not open source. So I don’t think Swift will be any help in developing cross platform programs (unless they are very simple command line utilities). Swift is quite a modern language and its object oriented implementation is quite nice. Apple seems to be putting quite a bit of effort into Swift with version 3 of the language soon to be released. There is certainly a large community of iOS developers out there who should be putting it to good use.

This was a fun little project and I think I will be spending a bit more time dabbling is iOS development using Swift.

Source Code Listings

//
//  ViewController.swift
//  KochSnowFlake
//
//  Created by Stephen Smith on 2016-05-13.
//  Copyright © 2016 Stephen Smith. All rights reserved.
//

import UIKit

class ViewController: UIViewController {

    // MARK: Properties
    @IBOutlet weak var fractalLevelTextField: UITextField!
    @IBOutlet weak var fracView: FractalView!

    override func viewDidLoad() {
        super.viewDidLoad()
        // Do any additional setup after loading the view, typically from a nib.
        //

        fractalLevelTextField.text = "2";
        fracView.level = 2;

        NSNotificationCenter.defaultCenter().addObserver(self,
               selector: #selector(textChangeNot),
               name: UITextFieldTextDidChangeNotification, object: fractalLevelTextField);
    }

    func textChangeNot( object: AnyObject )
    {
        if let enteredLevel = NSNumberFormatter().numberFromString(fractalLevelTextField.text!)
        {
            fracView.level = Int(enteredLevel);
            fracView.setNeedsDisplay();
        }
    }

    override func didReceiveMemoryWarning() {
        super.didReceiveMemoryWarning()
        // Dispose of any resources that can be recreated.
    }
}

//
//  FractalView.swift
//  KochSnowFlake
//
//  Created by Stephen Smith on 2016-05-13.
//  Copyright © 2016 Stephen Smith. All rights reserved.
//

import UIKit
class FractalView: UIView {
    var level = 1;

    // Only override drawRect: if you perform custom drawing.
    // An empty implementation adversely affects performance during animation.
    override func drawRect(rect: CGRect) {
        var frac: KochFlake;

        // Drawing code

        let currentColor = UIColor.blackColor();
        let context = UIGraphicsGetCurrentContext()
        frac = KochFlake(inContext: context!);

        //Set the width of the "pen" that will be used for drawing
        CGContextSetLineWidth(context,1);

        //Set the color of the pen to be used
        CGContextSetStrokeColorWithColor(context, currentColor.CGColor);

        frac.KockSnowflake(level);

        //Apply our stroke settings to the line.
        CGContextStrokePath(context);

    }
}

//
//  KochFlake.swift
//  KochSnowFlake
//
//  Created by Stephen Smith on 2016-05-13.
//  Copyright © 2016 Stephen Smith. All rights reserved.
//

import UIKit

class KochFlake
{
    var tg:TurtleGraphics;
    var context:CGContextRef;

    init(inContext: CGContextRef)
    {
        context = inContext;
        tg = TurtleGraphics(inContext: context);
    }

    func KockSnowflake(level:Int)
    {
        tg.turn( 60 );
        KockSnowflakeSide( level , size:400);
        tg.turn( -120 );
        KockSnowflakeSide( level, size: 400);
        tg.turn( -120 );
        KockSnowflakeSide( level, size: 400);
    }

    func KockSnowflakeSide(level:Int, size:Int)
    {
        if (level == 0)
        {
            tg.move( size );
        }
        else
        {
            KockSnowflakeSide( level - 1, size: size / 3 );
            tg.turn( 60 );
            KockSnowflakeSide( level-1, size: size / 3);
            tg.turn( -120 );
            KockSnowflakeSide( level-1, size: size / 3);
            tg.turn(60);
            KockSnowflakeSide( level-1, size: size / 3);
        }
    }
}

//
//  TurtleGraphics.swift
//  KochSnowFlake
//
//  Created by Stephen Smith on 2016-05-13.
//  Copyright © 2016 Stephen Smith. All rights reserved.
//

import UIKit
let pi:CGFloat = 3.14159;

class TurtleGraphics
{
    var x, y: CGFloat;
    var angle: CGFloat;
    var context: CGContextRef;

    init(inContext: CGContextRef)
    {
        context = inContext;
        x = 50.0;
        y = 150.0;
        CGContextMoveToPoint(context, x, y);
        angle = 0.0;
    }

    func move( dist: Int )
    {
        x = x + CGFloat(dist) * cos( angle * pi / 180.0);
        y = y + CGFloat(dist) * sin( angle * pi / 180.0);

        CGContextAddLineToPoint(context, x, y);
    }

    func turn( angleIncrement: Int)
    {
        angle = angle + CGFloat(angleIncrement);
    }
}

 

Written by smist08

May 16, 2016 at 8:00 pm

Wearable Devices and Sports

with 2 comments

Introduction

I happen to be on holiday this week on the Sunshine Coast (in BC, not Aus). I’ve doing a lot of running and cycling so I thought I would blog a bit on how new devices like GPS watches, step counters and Phone Apps are helping track sports. I have a Garmin GPS watch and an iPhone 4s. So what can I do with these and what is the potential as these devices improve?

The GCC

This year Sage is again participating in the GCC (the Get the World Moving Coporate Challenge). Basically you form teams of 7 co-workers and each of you wears a pedometer for the duration of the event. You then enter your steps, meters swam and km cycled into the website each day.

You then are tracked as you walk around the world and compete with other teams, either generally, within your company or within your area. The website is quite good, provides lots of useful information and tips on how to improve your health and fitness levels.

gcc

To do this tracking just requires a pedometer and their website. No other high tech gadgetry required. It will be interesting to see if more low tech solutions like this one (though the web site and pedometer are both fairly sophisticated) or solutions requiring more hardware like smart watches and extra devices will become the norm.

Garmin GPS Watches

There has been a lot of talk about Apple, Google and Microsoft coming out with smart watches this fall. Further several manufacturers like Samsung already have devices on the market. Then there have been a number of failures like Nike’s entry in this field. I think a lot of these companies have been looking at the success Garmin has had here. Garmin has transformed itself from manufacturing standalone GPS’s (which have now largely been replaced by functionality built into every phone) to making quite useful GPS sports watches.

The watches tend to be a bit bigger than a normal watch but still not uncomfortable to wear when running. Perhaps not the greatest fashion accessory, they are really quite useful. Besides recording your speed, location, distance and elevation in great detail, they also have heart rate monitors to give you quite a bit of information. Then they have a web site that is no extra charge to store and share all your routes and runs. For instance here.

garminconnn

The info is collected by the watch and then you upload it to your PC when you get back and then from there to their web site.

Generally this then gives you all sorts of metrics where you can see how you did, your pace every kilometer, how you did on uphill’s and downhill’s, etc. You can then track your progress and have a good idea of how you are doing.

Runtastic

There are quite a few fitness tracking apps for the iPhone. I just chose Runtastic because I liked the dashboard display in their app store ad. But otherwise it’s been fine, except for a bit too much promotion for the pro version. There are ads in the app and web site, but at a reasonable level, I think for the service you are getting.

There are a lot of attachments available to mount your phone on your bike; however, I’ve found that doing this really drains your phone’s battery quickly (i.e. in about an hour) and so isn’t really all that practical.

Runtastic Road Bike

More typically it’s better to leave the display off since then it doesn’t seem to use that much battery. Also if you stop to take pictures or something, make sure you switch back to the app first. The iPhone doesn’t have good multi-tasking, so unless the app is the active one, it probably won’t be doing anything.

Once you are finished your ride, the app uploads the data to the website and allows you to share what you are doing via social media (as any of my Facebook friends know). For instance this one here.

runtastic2

Like the Garmin website, this one gives you lots of information and makes it easy to track your progress as you try to improve your sport.

The Future

I think that companies like Apple are looking at this market and hope it is a bit like the early MP3 music player market was. Then a company like Apple could come along, redefine the market, make it dead simple and create a much larger market than what the early technology startups could achieve.

Whether Apple can repeat the iPod success in this market is yet to be seen. And they are certainly going to face a lot of competition as Microsoft and Google are hoping they can do the same thing.

Garmin type devices have better battery life and better durability than phones. However phones have better apps and greatly benefit from continuous Internet connectivity. So what are successful future devices going to need? From my perspective they will need:

  • Better battery life. Operating with only a couple of hour’s batter life is insufficient. This should really be a week.
  • Better durability. They can’t just fry when they get a little wet in the rain. Cycling and running are outdoor sports performed in any weather. Athletes don’t want extra clothing or gear to keep their watch dry. Further it would be great if these work for swimming. After all there are already a great many regularly triathlon watches that work great while swimming.
  • Intelligent support for more sports. Useful metrics gathered while golfing for instance. What about soccer, football or hockey?
  • Do not require a separate data plan. If you have to pay $50 per month to a cell phone provider then they are dead in the water.

Another area where there is great research going on is developing more sensors that measure things like blood glucose levels, blood pressure, etc. It will be interesting to see how tracking these additional metrics can help athletes.

There are also appearing apps that intelligently use the Phone’s camera to do things like analyze golf swings and tennis strokes. As these improve we may reach the stage where casual players can get real professional coaching and feedback right from their phone.

On the flip side, there is a lot of concern about the possible privacy implications of these devices. For instance if I record heart rate monitor information and it starts detecting abnormal behavior, could an insurance company find out and cancel my insurance? Could it be used in other adverse ways? Generally this sort of medical information is very protected. Will these devices, services and web sites offer the necessary levels of personal privacy protection? Will I find out I have a heart condition because suddenly I start receiving ads for defibulators and pace-makers? There is certainly a lot of concern about this out there and there have been many Science Fiction stories about the possible abuses. Hopefully these won’t all turn out to be prophetic.

Summary

By Christmas shopping season we are going to be inundated by new intelligent watches and other form factors that can help us track and improve our fitness levels. They will track all sorts of metrics for us, provide feedback and even professional levels of coaching. It will be interesting to see if this sparks a greater level of interest in fitness and sports. Maybe these will even help with the current epidemic obesity levels in our society.

Written by smist08

July 5, 2014 at 4:17 pm

Google Mobile Trends

with 2 comments

Introduction

In a previous blog I talked about Apple’s mobile directions following their annual developer’s conference. This past week Google help their annual I/O developer’s conference in San Francisco. So it seems like a good time to comment on Google’s mobile trends. There is a lot of similarity between Apple and Google’s directions. But there are also differences since each company has a slightly different view on what direction to take. Both companies are very large and have their hands in a lot of pies. As a consequence their announcements tend to be quite diverse and it’s interesting to see how they try to unify them all.

New Android

Like Apple announced a new version of iOS, so too Google announced a new version of Android namely “L”. I don’t know what happened to the cute code names like Kit Kat or Ice Cream Sandwich, but “L” it is. One big feature they’ve added is 64 Bit support. Apple recently introduced 64 Bit processors in their latest phones and now Google devices can catch up. This means that most new higher end phones are now all going to sport 64 Bit quad core processors. This is an amazing amount of computing power in such tiny devices.

As with most new operating systems these days, it includes a new UI look. With the new Android this new update is called Material Design and follows the fashion of a flatter more austere look to things.

There are many other new features in the new version of Android which will hopefully make people more productive.

Wearable’s Everywhere

A big trend in rumored and announced, but generally not shipping products are wearable’s. Google leads the way in these with Google Glasses. Then there are the endless smart watch announcements, rumors and even the odd product.

smartwatch

I have a Garmin GPS Watch with a heart rate monitor. Combined with the website where I upload the data, this is a great device to track my cycling and running. It would be nice if its battery lasted longer and if it was more waterproof so I could swim with it. In the meantime there are many great apps to do the same sort of things with your phone. These all do more than the Garmin watch, but the phone is bulkier and less durable in damp environments.

Having a waterproof watch that can do more than my Garmin and has a longer battery life would be fantastic.

Although in the early stages, both Google and Apple see the fitness market for metrics and tracking as a huge potential market. Both companies are both partnering and developing new sensors to measure new things, like small waterproof sensors for swimming or unique ways to measure other sports like for golf swings. Similarly measuring all sorts of biometric data beyond heart rate to include blood pressure, blood glucose levels and all sorts of other things. Eventually these will morph into a Star Trek like medical tricoder.

Home Automation

Google has now purchased a number of home automation companies including Nest. And are now integrating these with Android to provide full control of everything in your home from your phone. Including remotely setting the thermostat, receiving smoke detector alerts and monitoring security cameras. Most of these things are available now as separate discrete components but Google is working especially hard to make this whole area much more unified.

nest_thermostat_insteon-800x420

Online Cars

Another big area of interest is integrating into cars. Already most cars can interface to iPhones and Android phones to make calls hands free and play music. Now the goal is to sell Android (and iOS) into the auto industry. To have better more connected GPS (with real time traffic updates) and access to all your music library. Further many car companies are enabling using your car as a Wi-Fi hotspot.

I’m not sure how far this should go, since it all gets very distracting. Already we have so many potential distractions in cars. And just things like texting are causing many accidents.

Everything in the Cloud

With all Google’s products, the emphasis is storing all data in the cloud. They will only store things on local devices if there is a huge outcry of people that need to work offline (like on airplanes). Chromebooks really showed that this was possible and Google has led the way in offering lots of free cloud storage and making sure everything they do will interact seamlessly with these cloud documents.

They tout the convenience of this that things are always backed up, so if your laptop is stolen or destroyed you don’t lose anything. However critics worry about privacy concerns with storing sensitive data under someone else’s control, especially a search provider. It’s rather scary to corporate compliance officers that sensitive corporate documents might start showing up in people’s search results. Often this wouldn’t be due to Google doing something maliciously as someone just misclicking the visibility of the document to allow it to be viewed by anyone, and by anyone this means anyone on the Internet (and not just anyone that finds it on the corporate network).

All that being said, Google, Apple and Microsoft are all pushing this model like mad, and a lot of innovations that are in the pipeline completely rely on the adoption of this philosophy. It certainly is convenient to have all your photos and videos automatically uploaded and not to have to worry about sync’ing things by hand anymore.

Big Data

Google really started the big data revolution when they published the details of their Map Reduce algorithm that the original Google search was built upon. This then spawned a huge industry of open source tools and databases all built around this algorithm. Map Reduce was revolutionary since it let people get instant search results on searching for everything. Basically it worked by having a database of everything people might search for, like a giant cache that could return results instantly.

The limitation of Map Reduce is that constructing queries is quite difficult and often requires the database to be rebuilt in a different way. If you don’t do that, although the main query the database solves returns instantly, any other query takes a week to process.

Google is now claiming Map Reduce and all the industry like Hadoop based on it are completely obsolete. They were heavily promoting their new Cloud Dataflow service where the claim this service can also do efficient real time analytics as well as preserve the performance of the main functionality.

It will be interesting to see what this new service can really do and will it really threaten all the various NoSQL databases like MongoDB.

Summary

There are a lot of interesting things going on in the mobile world. It will be interesting to see if all our phones are replaced by watches or glasses in a couple of years. It will be interesting to see what great things come of all these new cloud big data services.

Written by smist08

June 28, 2014 at 5:28 pm

Apple Mobile Trends

with 2 comments

Introduction

With Apple’s WWDC conference just wrapping up, I thought it might be a good time to meditate on a few of the current trends in the mobile world. I think the patent wars are sorting themselves out as Google and Apple settle and we are seeing a lot more competitive copying. Apple added a lot of features that competitors have had for a while as well as adding a few innovations unique to Apple.

The competitive fervor being shown in both the Google and Apple mobile camps is impressive and making it very hard for any other system to keep up.

ios8_large

Cloud Storage

Apple has had the iCloud for a while now, but with this version we are really seeing Apple leverage this. When Google introduced the Chromebook they used this video to show the power of keeping things in the Web. This idea has been copied somewhat by Microsoft. But now Apple has taken this to the next level by allowing you to continue from device to device seamlessly, so you can easily start an e-mail on your phone and then continue working on it on your MacBook. No having to e-mail things to yourself, it all just seamlessly works.

Apple also copied some ideas from Google Drive and DropBox to allow copying files across non-Apple devices like Windows as well as sharing documents between applications. So now this is all a bit more seamless. It’s amazing how much free cloud storage you can get by having Google, Microsoft, Apple and Dropbox accounts.

Generally this is just the beginning as companies figure out neat things they can do when your data is in the cloud. If you are worried about privacy or the NSA reading your documents, you might try a different solution, but for many things the convenience of this outweighs the worries. Perhaps a bigger worry than the FBI or NSA is how advertisers will be allowed to use all this data to target you. Apple has added some features to really enable mobile advertising, whether these become too intrusive and annoying has yet to be seen.

Copying is the Best Compliment

Apple has also copied quite a few ideas from Google, Blackberry and Microsoft into the new iOS. There is a lot more use of transparency (like introduced in Windows Vista). There is now a customizable and predictive keyboard adding ideas from Blackberry and Microsoft. Keyboard entry has been one of Apple’s weaknesses that it is trying to address. Similarly the drive option in the iCloud is rather late to the game.

Apps versus the Web

There is a continuing battle between native applications and web applications for accessing web sites. People often complain that the native mobile application only gives them a subset of what is available on the full web site, but then on the other hand the consensus is that the native mobile apps give a much better experience.

True web applications give a unified experience across all devices and give the same functionality and the same interaction models. This is also easier for developers since you only need to develop once.

However Apple is having a lot of success with apps. Generally people seem to find things easier in the Apple App store than in browsing and bookmarking the web. Apple claims that over half of mobile Internet traffic is through iOS apps now (but I’m not sure if this is skewed by streaming video apps like Netflix that use a disproportionate amount of bandwidth).

Yet another Programming Language

Apple also unveiled a new programming language Swift for mobile development. One of the problems with C, C++ and Objective C programming is the use of pointers which tends to make programming harder than it needs to be. Java was an alternative object oriented extension to C with no pointers, then C# came along copying much of Java. Generally most scripting languages like Basic, JavaScript, Python, etc. have never had pointers.

Rather than go down the road of Java and C#, Swift has tried to incorporate the ease of use of scripting languages, but still give you full control over the iOS API. How this all works out is yet to be seen, but it will be interesting if it makes iPhones and iPads really easy to program similar to the early PCs back in the Basic days.

swift-screenshot

The Internet of Things

Apple introduced two new initiatives, their Health Kit and Home Kit. Health kit is mostly to encourage adding medical sensing devices to your iPhone, whereas Home Kit is to extend iOS into devices around the home and to control them all from your iPhone.

The Health Kit is designed to centralize all your health related information in one central place. There is getting to be quite a catalog of sensors and apps to continuously track your location, speed, heart rate, pulse, blood pressure, etc. If you are an athlete, this is great information on your fitness level and how you are doing. Garmin really pioneered this with their GPS watches with attached heart rate monitors. I have a Garmin watch and it provides a tremendous amount of information when I run or cycle. I don’t think this is much use for the iPhone, which I always leave behind since I don’t want to risk it getting wet, but this might really take off if Apple really releases a smart watch this fall like all the rumors say.

Home Kit is a bit of reaction to Google buying Nest, the intelligent thermostat. Basically you can control all your household items from your phone, so you can warm up the house as you are driving home, or turn all the lights on and off remotely. We have a cottage with in-floor heating, it would be nice if we could remotely tell the house to start heating up in the winter a few hours before we arrive, right now it’s a bit cold when we first get there and turn on the heat. However with zoned heating we would need four thermostats and at $250 each, this is rather excessively expensive. I think the price of these devices has to come down quite a bit to create some real adoption.

There is a lot of concern about having all of these hacked and interfered with, but if they get the security and privacy correct, then these are really handy things to have.

Summary

Apple has introduced some quite intriguing new directions. Can Swift become the Basic programming languages for mobile devices? Will Health Kit and Home Kit usher in a wave of new wonderful intelligent devices? Will all the new refinements in iOS really help users have an even better mobile experience? Will native apps continue to displace web sites, or will web sites re-emerge as the dominant on-line experience? Lots of questions to be answered over the next few months, but it should be fun playing with tall these new toys.

Written by smist08

June 7, 2014 at 4:27 pm

Google Forks WebKit

with one comment

Introduction

WebKit is the underlying HTML rendering library used primarily by the Apple Safari and Google Chrome browsers. It is used in a lot of other projects like the Blackberry Browser, Opera, Tizen, Kindle and even some Microsoft e-mail clients. Even Nokia was a big WebKit user before switching to Windows Phone. Generally it’s been considered a great success, rallying the web around standards and making life easier for web developers.

webkit

WebKit is a solid open source project with lots of support. This is one reason it’s so successful. Currently in Internet browsers there are three main HTML rendering engines: the Internet Explorer Trident engine, the Mozilla Firefox Gecko engine and then WebKit.

The big news around the Internet on this front recently is that Google is forking WebKit (meaning starting a new open source project based on WebKit) and then taking it in its own direction with a project called Blink. This raises all sorts of questions: like what it means to web developers? What is Google’s real agenda? Will this damage web standards? Will this slow WebKit development? In this blog posting I want to give my perspective on a few of these questions.

History

Actually WebKit was started out of a similar controversy. Back in 2001, Apple forked the KHTML/KJS HTML rendering engine used by the browser that is part of the KDE Linux User Interface system. Basically Apple wanted something better and more tuned for its OS X project. The result was the Safari browser built on the first WebKit HTML engine. At the time no one in the Linux community was happy about this, but in the end looking back, success makes everything all right.

So now that Google is forking WebKit, claiming that it’s for the same reasons that Apple forked KHTML, will history repeat itself and a much better HTML rendering engine will emerge? Or will this just fragment the market into more slightly different HTML rendering engines making life more difficult for web developers?

WebKit’s Mobile Success

In recent years, now that Android and the iPhone have completely taken over the mobile phone market, developing web sites with HTML5 and JavaScript has become much easier. This is because WebKit is used in both of these families of devices. This means to cover 95% of the mobile market you just need to target WebKit. This greatly simplifies development and testing. Further WebKit follows web standards diligently, it keeps up with evolving standards, has great performance and great quality.

I think that WebKit has been a major contributor to the combined success of both Android and the iPhone. You can easily browse most websites from these devices. Plus when Apps incorporate browser controls they are using WebKit.

Further both Apple and Google are contributing actively to WebKit. It’s been an interesting combination of co-operation and competition. When new hardware devices come out, initially it tends only be accessible for Apps. But Google tends to very quickly add JavaScript APIs for the device to WebKit. Then Apple tends to follow suite quite quickly. Further each is driven to keep incorporating the latest version into their devices since they don’t want to let the other get ahead of them.

One of the worries of Google forking WebKit and going its own way is that we will lose the competitive nature of Apple versus Google that has been driving WebKit forwards.

Why Fork?

So why is Google forking WebKit? A lot of opinion on the Internet is that this is a strategy to sabotage Apple. I guess Google could be egomaniacal enough to think that WebKit will fail without them participating. But Apple is such a big company with so much money and talent, I think they can do just fine with WebKit, after all they did start it without Google’s help. Further I suspect the army of independent open source programmers that contribute to WebKit will continue to do so and won’t switch to Blink.

Google’s official reason is that the code in WebKit is getting too burdened with supporting code for Safari, Chrome and all the other various things it does. That if they take WebKit and remove anything that Chrome doesn’t use then they will have a smaller, faster and easier to develop code base. Basically they claim they want to move the HTML engine forwards more tightly coupled with their multi-processor architecture to improve security and performance. That doing this while supporting competing architectures within the same code base is getting harder and harder.

When Apple started WebKit and later when Google joined it, both Google and Apple were primarily worried about Microsoft and wanted to have Browser technology clearly superior to Internet Explorer. Now with their success, Microsoft is now pretty well non-existent in the mobile world. I think as a result Google isn’t feeling threatened by Microsoft anymore and is turning its attention to Apple. Generally relations between the two companies have been getting colder and colder in recent years.

Actually Google currently only uses the HTML and CSS rendering part of WebKit called WebKitCore. They stopped using the JavaScript component JavaScriptCore in favor of their own V8 JavaScript engine. The V8 JavaScript engine has been blowing away the competition in JavaScript benchmarks for some time now. In fact the V8 JavaScript engine is also the heart of Node.js the highly successful JavaScript server side processing framework. I think Google is looking to get the same sort of success out of Blink that they got from V8.

What’s the Problem?

The problem is for developers. Right now developing good web pages that run nicely anywhere means targeting IE, FireFox and WebKit which then covers the main HTML/CSS rendering engines. Unfortunately HTML and CSS are very complicated and quite subtle. Although all adhere to the published web standards, there are differences in interpretation. Also there are emergent properties that get exploited as features, things that aren’t really in the standard but have appeared in an implementation.

In the mobile world right now, developers have it easier since they can target Android, iOS, Blackberry, Tizen and Symbian by just targeting WebKit. This makes life much easier since you really can develop once and deploy pretty much anywhere. It will be a pity to lose this, and potentially quite expensive for smaller development organizations.

I imagine that many source code files will continue to be shared by WebKit and Blink. But for how long? When will we have to pay attention between differences between Blink based browsers and WebKit based browsers?

Summary

Although I find it appealing that Google is hoping to do for HTML/CSS rendering speed what it did for JavaScript execution speed with V8, I’m really worried that this is going to fragment HTML5 development for mobile devices. I tend to think this will cause more web developers to decide that if I need to develop Android and iOS separately then I may as well do both natively in Apps. To me this will be a sad further fragmentation and polarization of mobile developer communities.

Written by smist08

April 13, 2013 at 3:40 pm

My First Experience Writing an iPad App

with 2 comments

Introduction

To get a feel for iOS programming I thought I would create a simple iOS application to display a Koch snowflake. There would be a simple edit box where you enter the fractal level and then it will draw the snowflake. I thought this would be a good example to get a feel for the XCode development environment, simple user interaction with controls and a flavor for some graphics programming.

Two jobs before I started with Computer Associates to work on CA-Accpac/2000, I worked for a company that created stock market software. They were working on a new workstation version for the NeXT computer. So with that company (over 20 years ago), I gathered good experience in this environment. We programmed in Objective-C and used the innovative NeXTStep development environment which had many neat features like using DisplayPostScript for rending on-screen graphics.

Now so many years later, here I am programming iOS and relearning Objective C. Although Apple has had many battles with Adobe over the years resulting in display postscript never being mentioned, I still see it there in how you do things. Since I’m newly relearning all this stuff and there have been many changes in the past 25 years, don’t take everything I say as the best way to do things, they are the way I first figured out how to do things for this simple project. Especially if you can do something visually in XCode versus writing code, I probably wrote code because that’s what I’m most comfortable with.

XCode

XCode is Apple’s IDE for doing both native Mac OS and IOs development. XCode is a very rich development environment with many built in tools for things like unit testing, debugging, screen designing and such. It has built in support for the Git source code control system, which is used by default for all projects. There is much built in documentation and help, as well as emulators to test and debug your iPad and iPhone applications. The main requirement of XCode is that it runs on MacOS. Hence I have to run it on my trusty MacBook Air.

XCode has a great many productivity helpers like auto-complete and many way to graphically create your programs rather than coding them. The thing that gave me the most trouble was the graphical part, I’m fine with writing code, but graphically connecting things wasn’t as intuitive to me as I would have expected.

xcode

Koch Snowflakes

Koch snowflakes are simple fractals that are a good way to give an idea of how fractals can build complexity out of simplicity. Basically you start with a base shape, in this case a triangle, then you replace each line segment with a new shape, in this case the two lines with a “v” in the middle. Then you do this recursively to get more and more complicated shapes. Below is the progression from level 0, the base shape to level 1, with the base shape lines replaced by the fractal generator and then so on as the level increases.

koch-snowflake-progression

This is a fractal because in the limit as the level goes to infinity, the shape has a fractal dimension, in that it is somewhere between 1 dimensional and 2 dimensional in a defined mathematical sense.

Turtle Graphics

To me the easiest way to draw fractals is with a turtle graphics library. This is a simple drawing library where you tell a turtle to either turn or move forwards. As he moves he leaves a trail. Hence the base shape for the Koch snowflake is forward 1, turn 60, forward 1, turn -120, forward 1, turn 60 forward 1. This is then really easy to apply recursively to draw fractals.

Objective C

Objective-C was one of the first object oriented extensions to C. It was implemented as a pre-processor that generated C code which was then compiled using regular C development tools. This greatly simplified implementation, but the syntax reflects that it was designed to be easily processed by a preprocessor. It’s certainly evolved since its early days, but is perhaps considered a bit clunky as a result. Objective-C has all the object oriented features like inheritance and classes, but lacks a lot of the complexity of C++. The object oriented features are more similar to Java, but unlike Java, Objective-C still has all of C under it, meaning all the pointers and pointer related features that Java removed; hence you can easily crash your iOS app if you make pointer mistakes like in a C program.

Both Java and C++ overloaded the pointer syntax to call methods and such making them behave like function pointers in a structure. Objective C tried for a syntax to reflect message passing using their square bracket syntax of [object method] to send a message to an object. See the source code listings below for some examples. Recently Objective C has adopted the pointer type syntax as an option but considers it good coding practice to only use this for accessing class properties.

My iPad App

When you create a new project in XCode you get a complete working program and then only have to fill in your own code for your functionality. I edited the iPad storyboard to have a single page consisting of a label and text field for the fractal level and then a view to draw the image on. I created a couple of classes, one for turtle graphics, another to draw the fractal and then one to act as the interface to the UI form. All in all, not a lot of code and it seems to run fairly well. I placed the source code listings at the end so as not to clutter up this article. Beware that WordPress often changes characters for typographic reason, things like regular double quotes to 66 or 99 type quotes, this tends to introduce syntax errors if you cut and paste the code, so beware.

Storyboards are a relatively new feature to XCode, they allow you to define many screens within a single file and to connect them all together, so a button on one screen can trigger a transition to another screen, all setup graphically with no code. This is a great tool for quickly prototyping applications, but since for this app, I only have one screen it isn’t really used.

To connect the controls in the storyboard to the code, you create matching variables in the generated interface file and then drag an arrow from a small o in the margin to the matching control in the storyboard file (a process I find a bit cumbersome). Then you can ask for notifications and set properties for the various controls. You can see a couple of examples in the interface file below.

With any object oriented framework like Cocoa Touch there is quite a steep learning curve. Not only do you need to find the properties and methods to call to do things for you, but you also need to learn when you have to extend one of the system base classes. In the case of drawing the fractal, it’s a matter of putting a view control on the page, but then you need to extend the default class to override its drawRect method that is called whenever the view need redrawing. This is similar to handling a WM_PAINT message in Windows. Again you can have a look at the code down below.

Below are a couple of screen grabs of running this iPad app in the iPad emulator on my MacBook Air:

level2

 

level4

Summary

Sadly you can’t just post and distribute iPad apps, but have to go through the Apple iTunes store. I’m not going to bother posting this app, so you can’t play with it. I do like being able to post source and app to let people just run it, but Apple doesn’t allow this.

Creating this app was fun. XCode is quite a good development environment and they make it easy to write code quickly. iOS is a very full featured operating system with many built in services and a great deal of power. This are tons of books and internet articles on iOS development along with all the Apple documentations. Now to dig in a bit deeper to what you can do.

Source Code Listings

//
//  csFractal.m

//  Fractal1
//
//  Created by Stephen Smith on 2013-03-05.
//  Copyright (c) 2013 Stephen Smith. All rights reserved.
//

#import "csFractal.h"
#import "csTurtleGraphics.h"

@implementation csFractal
{
    csTurtleGraphics *tg;
    CGContextRef context;
}

- (id)initWithContext: (CGContextRef) inContext
{
    self = [super init];
    if (self)
    {
        context = inContext;
        tg = [[csTurtleGraphics alloc] initWithContext:context];
    }
    return self;
}

- (void) KockSnowflake:(int)level
{
    [tg turn: 60];
    [self KockSnowflakeSide: level size: 500];
    [tg turn: -120];
    [self KockSnowflakeSide: level size: 500];
    [tg turn: -120];
    [self KockSnowflakeSide: level size: 500];
}

- (void) KockSnowflakeSide: (int)level size:(double) size
{
    if (level == 0)
    {
        [tg move: size];
    }
    else
    {
        [self KockSnowflakeSide: level-1 size: size/3];
        [tg turn: 60];
        [self KockSnowflakeSide: level-1 size: size/3];
        [tg turn: -120];
        [self KockSnowflakeSide: level-1 size: size/3];
        [tg turn:60];
        [self KockSnowflakeSide: level-1 size: size/3];
    }
}

@end

//
//  csTurtleGraphics.m
//  Fractal1
//
//  Created by Stephen Smith on 2013-02-23.
//  Copyright (c) 2013 Stephen Smith. All rights reserved.
//

#import "csTurtleGraphics.h"

const double pi = 3.14159;

@implementation csTurtleGraphics
{
    double x, y;
    double angle;
    CGContextRef    context;
}

- (id)initWithContext: (CGContextRef) inContext
{
    self = [super init];
    if (self)
    {
        context = inContext;
        x = 50.0;
        y = 150.0;
        CGContextMoveToPoint(context, x, y);

        angle = 0.0;
    }
    return self;
}

- (void)move:(int) dist
{
    x = x + dist * cos( angle * pi/ 180.0);
    y = y + dist * sin( angle * pi/ 180.0);
    CGContextAddLineToPoint(context, x, y);

}

- (void) turn: (int) angleIncrement
{
    angle = angle +angleIncrement;
}
@end

//
//  csViewController.m
//  Fractal1
//
//  Created by Stephen Smith on 2013-02-10.
//  Copyright (c) 2013 Stephen Smith. All rights reserved.
//

#import "csViewController.h"

@implementation csViewController

@synthesize textField;
@synthesize fracView;

- (void)viewDidLoad
{
    [super viewDidLoad];
                // Do any additional setup after loading the view, typically from a nib.
    textField.text = @"2";
    [fracView setLevel: 2];

    [[NSNotificationCenter defaultCenter]
        addObserver:self
        selector:@selector(textChangeNot:)
        name:UITextFieldTextDidChangeNotification
        object:textField];

}

- (void) textChangeNot: (id) object
{
    [fracView setLevel: textField.text.intValue];
    [fracView setNeedsDisplay];
}

- (void)didReceiveMemoryWarning
{
    [super didReceiveMemoryWarning];
    // Dispose of any resources that can be recreated.
}

@end

//
//  csFractalView.m
//  Fractal1
//
//  Created by Stephen Smith on 2013-02-17.
//  Copyright (c) 2013 Stephen Smith. All rights reserved.
//

#import "csFractalView.h"
#import "csFractal.h"

@implementation csFractalView
{
    csFractal *frac;
    int level;
}
- (id)initWithFrame:(CGRect)frame
{
    self = [super initWithFrame:frame];
    if (self) {
        // Initialization code
        level = 1;
    }
    return self;
}

- (void) setLevel: (int)lev
{
    level = lev;
}

// Only override drawRect: if you perform custom drawing.
// An empty implementation adversely affects performance during animation.
- (void)drawRect:(CGRect)rect
{
    // Drawing code

    UIColor* currentColor = [UIColor blackColor];
    CGContextRef    context = UIGraphicsGetCurrentContext();

    //Set the width of the "pen" that will be used for drawing
    CGContextSetLineWidth(context,1);
    //Set the color of the pen to be used
    CGContextSetStrokeColorWithColor(context, currentColor.CGColor);

    frac = [[csFractal alloc] initWithContext: context];
    [frac KockSnowflake: level];

    //Apply our stroke settings to the line.
    CGContextStrokePath(context);
}

@end

Written by smist08

March 16, 2013 at 4:59 pm

Frustrations in Developing Mobile Applications

with 14 comments

Introduction

Recently I’ve been talking to many people about various techniques to develop portable mobile applications. In the good old days of the 90s with the Wintel monopoly usually you could just develop for Windows and you would reach 99% of the market. The main challenge was just adapting to new versions of Windows where you would get things like UAC thrown at you.

Now suddenly we are developing for various Windows devices, various Apple devices and various Android/Linux devices. Plus we have some other contenders like Blackberry clamoring for our attention. The market is now highly fragmented and all of these have considerable market share.

I develop business applications and the functionality I’m most interested in has to do with ERP and CRM workflows. This means I’m not writing games, although it would be fun to produce a game like “Angry Accountants” or “ERPville”.

I know I’ve blogged about mobile development a few times like here and here; but my thinking on this keeps changing and I’m still not happy with the whole situation. There are many mobile frameworks and I’m only touching on a couple of representative ones here. I’ve got to think there will be a better solution, but until then I feel like ranting.

mobile device frustration

Going Native

There is an appeal to going native. The native development environments are really excellent. I’ve been playing with Apple’s XCode development tools for OS/X and iOS development and they are really amazing. They’ve progressed a lot since I last saw them over 20 years ago when I worked for a company that did NeXTStep development for the NeXT cube. Similarly Visual Studio 2012 for Windows 8 development is really quite good and so are all the Android tools.

If I only needed to development for one of these, I would be happy with any one of them. But keeping several in my brain at once really hurts.

You get the best results for the given platform with any one of these, but you don’t really get anything reusable except the basic design. All the platforms use a different object oriented extension of C (namely Objective C, Java and C#). All the platforms have different operating system functions and different separations between what you do in the application versus have as a service.

C Reborn

One surprising thing I found from talking to people was that the idea of writing as much as you could in C. All the main platforms use extensions of C and all support compiling and running C code. This reminds me of the old days where you tried to write a portable application for Mac, Windows and Linux by isolating the operating system dependent parts and then writing as much code as possible in good old portable C. Funny how what was old can be new again. But then it was a good idea back then, why wouldn’t it be a good idea now?

Air

Much maligned Adobe always seems to have a proprietary solution in the game. With Flash being booted from most platforms, Air seems to have followers in some areas. Some people really like Adobe development tools, I’ve always found them strange. Like with Flash, uses ActionScript which is a nice object oriented extension to JavaScript, but then that makes it all non-standard. Then strangely you have to structure Flash projects as a movie which I’ve never liked. Air seems to claim to follow standards but then keeps dragging in Flash technologies. My own bias is that if you go down this route, you may as well stick with using JavaScript which is then more standard and more cross platform.

The other problem with Adobe is that they are the leading vendor in producing software with giant security flaws. This means they are more likely to be blocked or dropped from platforms. It is also a big risk for app development since your app could be tarred by Adobe’s problems.

Xamarin

Xamarin takes the Mono project and ports it to mobile devices like iOS and Android. The goal then is that you can develop a C# Windows application that will also run on iOS and Android. We tried Mono as a way to move some .Net projects to Linux, but just ran into too many problems and had to give up. As a result Mono has left a bad taste in my mouth so I’m inclined to avoid this. I also wonder how much code you will have putting the .Net runtime on top of the native iOS or Android operating systems. Is this just going to have too many layers and is it just going to be too fat and bloated?

If they can pull it off with high quality and compatibility there is potential here, but I suspect, like Air, you will just get a big non-standard mess.

JavaScript/HTML5/CSS

Ever since the first Netscape browser we’ve been promised that the web will standardize all programming. Then came a proliferation of web standards and incompatible browsers. Now things are coming back together in the web world. We have good standardization on HTML5, JavaScript and CSS. We have a number of browsers with good support for these and they run on pretty much all PCs, laptops, tablets and phones. So you would think you can just develop once as a web application and run happily everywhere.

Unfortunately all the vendors have a vested interest in their app stores (like iTunes). Vendors like Apple, Google and Microsoft make 30% off all software sold through their stores. They make nothing on people running web applications from browsers. As a consequence quite a few native platform functionalities are held back deliberately from the web. Then they market hard that for the best experience you must use a native app form their store or you are getting a second rate experience. Strangely the reverse is often the case where the app is just providing a subset of some web site and you lose abilities like being able to zoom.

In the current market/environment it’s very hard to compete against native apps with web apps which is really too bad. I think at some point the app store monopoly will fall apart, but that is today’s reality.

Phonegap

Phonegap is an open source library to try to bridge the gap between HTML/JavaScript apps and native apps. It adds a hardware API for JavaScript apps and allows them to be packaged for distribution via app stores. Phonegap was recently purchased by Adobe which really worried me. So far Adobe hasn’t done anything bad (that I’ve seen) and hopefully it will survive as a good open source solutions.

The main risks with Phonegap is that it usually lags the native apps in adoption of new operating system features, Apple may at some point start rejecting apps made with Phonegap and Adobe may start adding proprietary Flash like technology.

Beside these drawbacks the other problem is that your app is still made out of Browser controls and not the UI widgets that are part of the underlying operating system. You can style away a lot of differences but discerning users will be able to tell the difference.

Phonegap is a great technology which does really help JavaScript/HTML apps be more native and is really worth considering if you go down this road.

Summary

I’m still frustrated. I’m not really happy with the quality of apps produced by the cross platform technologies and I don’t like developing the same thing multiple times using the native SDKs.

I also find it a bit monotonous to develop the same program over and over again for iOS, Android, Blackberry and Windows.

Written by smist08

February 23, 2013 at 11:34 pm

Voice Input and Concierge Services

with 2 comments

Introduction

Some of the most exciting new technologies appearing on mobile phones are around voice recognition and concierge or personal assistant type of applications. These include ambitious applications like Apple’s Siri, along with a number of initiatives from Google including Google Now and Google Voice Search.

The voice recognition by itself is a truly amazing technology, but this is only a fraction of the story. After the voice input is recognized the query is combined with other input, like your location, to determine a lot of context for what you are asking about, identifies the problem domain and gives a truly meaningful answer along with relevant data to correctly answer or respond to your query.

Of all the technologies on Star Trek, we don’t see any sign of a working warp drive or transporter, but being able to ask a computer anything on any topic and get a good answer, we seem to have that now. So perhaps if Star Trek IV was set another ten years ahead, then Scotty wouldn’t have had any trouble interacting with our primitive computers.

star_trek_4_apple_mac_plus1

Device or Service?

An incorrect assumption is that you can integrate apps running on your phone to these services. This is the wrong way to think about how they work. They aren’t a voice recognition/query engine running on your device. In fact they send all the (nearly) raw input to a major data center to process them. Even though there isn’t a device API for accessing Siri, developers have found clever ways around this, by putting clever things in the contact list and constructing special text messages, but again this is really just using Siri as voice recognition software. The real intent of Siri is much deeper; it’s really a task completion engine.

These engines are really taking your voice input and then mapping them to various problem domains which then talk to many APIs on the backend. The goal isn’t to run an app and then just provide a voice recognition engine that translates voice commands into regular app commands as if the user had typed them. The goal is really that you don’t need device apps. When you ask Siri a question, you don’t need a matching app running, if you ask about airline info, it gets it, if you ask about weather, it gets it. You don’t need to run the right app.

In a way a limitation of current mobile phones is the need to download and install so many apps. Do you really need all of these? Most of the apps on my phone are specialized query information gathering apps like weather, news and such. The real beauty of these new personal assistant type applications is that they eliminate the need for all these other apps. Wouldn’t a phone or tablet be much easier if you didn’t need to find and install all these apps? Isn’t this the original appeal of the Internet to PC users? You don’t need to install dozens of applications (which got more and more painful); all you needed was a Browser and nothing else. To some degree these personal assistant applications become a workable Browser for mobile devices, where you no longer need all these apps anymore. Sure there are some special purpose apps for playing games and performing specialized functions, but generally you can just use Siri, Google Voice Search or Google Now for most things that you probably use Apps for now. Sure these aren’t perfect yet, just like the original Netscape Browser wasn’t perfect, but they are getting there very quickly.

Integrating to ERP and CRM

OK, so we don’t integrate to these new services via Apps talking to APIs on devices, so if we want to integrate our CRM or ERP into say Siri, how do we do it? Suppose we want to ask Siri what is the status of an Order from a vendor, or we want to ask Siri what is the credit limit of a customer I’m about to visit?

The key is to have this information available on the Internet via RESTful Web Services like SData. The reason for RESTful Web Services is that they allow discovery by search engine spiders. Generally shortened URLs give the list of how to build the rest of the URL, this allows a general engine to discover all the data. RESTful Web Services are the new Internet standard and all these services are built to interact with them.

The key is for vendors (like Sage) to make the right agreements with these services, so that the data can be accessed in a secure way, and you aren’t doing something like exposing all your ERP data to the Internet in general. Security and the rules for who can access what are crucial. Standard sign-on mechanisms like OAuth are going to have to be used.

The other thing is that all this data must be in a central location. This means that any ERP or CRM data that is going to be available to these services must be sync’ed to a central cloud location. This then fits in with Sage’s connected services strategy of sync’ing key on-premise data to the cloud (of course if you are already running your CRM or ERP in the cloud then you can skip this step). I blogged about Sage’s Hybrid Cloud here. From Sage’s Hybrid Cloud we can expose the correct data via SData Web Services for anyone that wants to participate in these services. Then Sage can make the correct deals with the services and is responsible that all the security concerns are setup correctly.

This can then lead to a company’s employees and customers being able to make general inquiries into these services and for the right questions have them mapped to a problem domain in the ERP or CRM space, have the backend systems provide answers with relevant data added from the Hybrid Cloud.

None of these services would look into the Hybrid Cloud in real time, they all operate like Search Engines which are continuously polling sites and updating their master databases, then for performance reasons all the real queries are handled as highly optimized Big Data queries against a master search database, so that all questions are magically answered instantly.

Overtime the questions answered can become more and more sophisticated, incorporating more and more sources of business data. Perhaps you can ask Siri: What’s the best way to increase my company’s revenue? And then get back a useful answer.

Summary

I think these personal assistant type applications are going to become more and more prevalent in the mobile world (or even on regular computers). To me it’s exciting to consider participating in this and to think about all the questions that we can help answer.

Star-Trek-The-Original-Series-TV-1966-movie-props

Developing Windows 8 Style UIs

with 5 comments

Introduction

Microsoft has release Windows 8 to manufacturing with a whole new User Interface technology. Up until a few days ago, this was called Metro, but now Microsoft just dropped that name in favor of “Windows 8 Style UIs”. A bit of a strange name, but full product names rarely just roll off the tongue.

I’ve spent a little time playing with developing “Windows 8 Style UIs” and thought I’d spend this blog post covering some of my experiences. Let’s just call them W8SUs for the rest of this post.

Closed Development System

One of the main goals for this new UI development system is to copy Apple’s success with iOS development and the iTunes store. In the Apple world, you can only develop native iPad and iPhone apps using the Apple SDK on a Mac computer. Further you can only distribute your applications by posting them on the Apple iTunes App store, passing a certification process and in the process allowing Apple to take 30% of the revenue. This has been making Apple billions of dollars and Microsoft would like to emulate that.

You can only develop W8SUs in Visual Studio. VS2012 generates a bunch of proprietary cryptographic code signing info must be there to run. Further you must be signed on with a Windows Live developer account. Another gotcha is that you can only develop for these on Windows 8 (or Windows Server 2012). If you install Visual Studio 2012 on a Windows 7 computer, it won’t install any Windows 8 development components there.

Once you do all this, you can’t just compile your application, zip it up and give it to a friend to run. Like Apple, W8SUs can only be installed via the Microsoft Store. There is an enterprise distribution system to install apps developed for an enterprise across an enterprise, but this again is tightly controlled. Even if you install on another computer via your developer license, it will be time bombed to only work for 1 month.

This is all very new to Windows developers. I’m not entirely sure how it will be received. Apple is successful because of all the revenue their store generates. However most of these are low cost consumer applications. Not sure how this will play out in the enterprise market.

Visual Studio 2012

You can develop these UIs in either JavaScript/HTML or C#/XAML. I chose JavaScript/HTML since that is what I already know. You can use either VS 2010 or 2012, I figured, I may as well go with the newest even though it’s a release preview. Actually VS 2012 worked pretty well. Debugging these applications is fairly easy and the tools are pretty good. Since JavaScript is object oriented more by convention than an enforced part of the language, intellisense has to guess what is valid, and although not always correct, it still does a pretty good job. The only place I found it difficult was when you get an exception as part of an event, and then it can be pretty tricky to find the true culprit, since it usually isn’t part of the call stack.

VS 2012 comes with a set of template to give you a start for your W8SUs. These templates give you a working program with some faked in data. When developing for W8SU in JavaScript/HTML, you need to interact with a number of core operating system components which are either built into the environment by some automatically included references or via some proprietary UI controls. For instance the scrolling ListView that is the hallmark of the opening Start Page is a proprietary control that includes all the standard Win8 interactions. When you are programming in JS, the core of the program consists of handling some proprietary events for the program execution state and call the API to invoke the data binding functions. Once you get away from this you can program individual pages of your application pretty much as standard web apps using standard Web libraries like JQuery or HighChart. Then you string together the page navigation using some proprietary APIs.

So you are doing full real web development with JavaScript/JQuery/HTML/CSS, but you are producing an application that will only run on Windows 8 installed from the Microsoft store. A bit of a strange concept for Web Developers, where the promise was to write once and run anywhere. I think you can structure your program to keep most of it re-usable to generate a Web app version using a separate page navigation system and some sort of alternative to the ListView control.

JavaScript Restrictions

When running under W8SU, you are essentially running under a modified version of IE 10. However there are a number of annoying restrictions compared to running IE 10 regularly. In previous versions of IE, many standard web functions, like parsing XML, were handled with ActiveX controls. Now IE can do many of these things in the standard web way, so it’s better to not use the ActiveX way. So if you try to use an older library that detects you are running under IE and tries to use one of these, then you get a very severe security exception. In general you can’t use any Add-ons or ActiveX controls, included those that used to be built into IE and Windows. I found a work around is to fool libraries to think they are running under Firefox rather than IE and that often gets around the problem.

Plus W8SU removes some features of standard JavaScript that it thinks are “dangerous” for some reason. For instance you can’t use the JavaScript alert and prompt statements. These are banned. This is annoying because again, many libraries will use these for unexpected errors and instead of seeing the error; you get a horrible security exception.

Another annoying thing is that the screen isn’t entirely standard like a standard web page. The page will not scroll, so if your content goes off the side, then it is just truncated, scroll bars are never added to the whole page. If you want scrolling then you need to put your content in a ListView or some other control which then causes other complexities and problems. I’m hoping this is really a bug that gets corrected by the real release.

Some of the controls also seem a bit buggy, which hopefully will be corrected by release. For instance if you put a ListView inside a ListView control, it gets quite confused. Also if you put a proprietary date picker in a ListView control then it ends up read-only.

Since these are based on IE, they use IE’s caching mechanisms. Currently there is no way to clear these caches from the system. The only way is to know the secret folders it uses and to go in and manually delete these. If you clear the cache in IE 10, it has no effect on W8SU programs. This is mostly annoying when doing application development, since re-running the program won’t re-download new static content from your web site. Again hopefully this is fixed by release.

SData

Using SData from a W8SU is really quite easy. There is an API called “WinJS.xhr” which makes asynchronous RESTful web service calls.

    Promise = WinJS.xhr({
       type: "POST",
       url: sdataurl,
       user: "USERID",
       password: "PASSWORD",
       data: atomdata
    });

It has the exact parameters you need for making SData calls. It returns a promise which is W8SU’s way of notifying you when an asynchronous request returns, basically you can set functions to be called if the call succeeds, fails or to get progress. You can also join promises together, so you can make a bunch of asynchronous calls and then wait for them all to finish.

Summary

I think Window’s 8 Style UIs have a lot of potential. I worry they are being rushed to market too quickly and aren’t quite ready for prime time. I also worry that the touch focus is going to turn everyone with a standard laptop or desktop off Windows 8 entirely. Hopefully the technology gets a chance to evolve and that new devices beyond the Surface tablet hit the scene to give it a really good user experience.