Stephen Smith's Blog

Musings on Machine Learning…

Archive for June 2014

Google Mobile Trends

with 2 comments


In a previous blog I talked about Apple’s mobile directions following their annual developer’s conference. This past week Google help their annual I/O developer’s conference in San Francisco. So it seems like a good time to comment on Google’s mobile trends. There is a lot of similarity between Apple and Google’s directions. But there are also differences since each company has a slightly different view on what direction to take. Both companies are very large and have their hands in a lot of pies. As a consequence their announcements tend to be quite diverse and it’s interesting to see how they try to unify them all.

New Android

Like Apple announced a new version of iOS, so too Google announced a new version of Android namely “L”. I don’t know what happened to the cute code names like Kit Kat or Ice Cream Sandwich, but “L” it is. One big feature they’ve added is 64 Bit support. Apple recently introduced 64 Bit processors in their latest phones and now Google devices can catch up. This means that most new higher end phones are now all going to sport 64 Bit quad core processors. This is an amazing amount of computing power in such tiny devices.

As with most new operating systems these days, it includes a new UI look. With the new Android this new update is called Material Design and follows the fashion of a flatter more austere look to things.

There are many other new features in the new version of Android which will hopefully make people more productive.

Wearable’s Everywhere

A big trend in rumored and announced, but generally not shipping products are wearable’s. Google leads the way in these with Google Glasses. Then there are the endless smart watch announcements, rumors and even the odd product.


I have a Garmin GPS Watch with a heart rate monitor. Combined with the website where I upload the data, this is a great device to track my cycling and running. It would be nice if its battery lasted longer and if it was more waterproof so I could swim with it. In the meantime there are many great apps to do the same sort of things with your phone. These all do more than the Garmin watch, but the phone is bulkier and less durable in damp environments.

Having a waterproof watch that can do more than my Garmin and has a longer battery life would be fantastic.

Although in the early stages, both Google and Apple see the fitness market for metrics and tracking as a huge potential market. Both companies are both partnering and developing new sensors to measure new things, like small waterproof sensors for swimming or unique ways to measure other sports like for golf swings. Similarly measuring all sorts of biometric data beyond heart rate to include blood pressure, blood glucose levels and all sorts of other things. Eventually these will morph into a Star Trek like medical tricoder.

Home Automation

Google has now purchased a number of home automation companies including Nest. And are now integrating these with Android to provide full control of everything in your home from your phone. Including remotely setting the thermostat, receiving smoke detector alerts and monitoring security cameras. Most of these things are available now as separate discrete components but Google is working especially hard to make this whole area much more unified.


Online Cars

Another big area of interest is integrating into cars. Already most cars can interface to iPhones and Android phones to make calls hands free and play music. Now the goal is to sell Android (and iOS) into the auto industry. To have better more connected GPS (with real time traffic updates) and access to all your music library. Further many car companies are enabling using your car as a Wi-Fi hotspot.

I’m not sure how far this should go, since it all gets very distracting. Already we have so many potential distractions in cars. And just things like texting are causing many accidents.

Everything in the Cloud

With all Google’s products, the emphasis is storing all data in the cloud. They will only store things on local devices if there is a huge outcry of people that need to work offline (like on airplanes). Chromebooks really showed that this was possible and Google has led the way in offering lots of free cloud storage and making sure everything they do will interact seamlessly with these cloud documents.

They tout the convenience of this that things are always backed up, so if your laptop is stolen or destroyed you don’t lose anything. However critics worry about privacy concerns with storing sensitive data under someone else’s control, especially a search provider. It’s rather scary to corporate compliance officers that sensitive corporate documents might start showing up in people’s search results. Often this wouldn’t be due to Google doing something maliciously as someone just misclicking the visibility of the document to allow it to be viewed by anyone, and by anyone this means anyone on the Internet (and not just anyone that finds it on the corporate network).

All that being said, Google, Apple and Microsoft are all pushing this model like mad, and a lot of innovations that are in the pipeline completely rely on the adoption of this philosophy. It certainly is convenient to have all your photos and videos automatically uploaded and not to have to worry about sync’ing things by hand anymore.

Big Data

Google really started the big data revolution when they published the details of their Map Reduce algorithm that the original Google search was built upon. This then spawned a huge industry of open source tools and databases all built around this algorithm. Map Reduce was revolutionary since it let people get instant search results on searching for everything. Basically it worked by having a database of everything people might search for, like a giant cache that could return results instantly.

The limitation of Map Reduce is that constructing queries is quite difficult and often requires the database to be rebuilt in a different way. If you don’t do that, although the main query the database solves returns instantly, any other query takes a week to process.

Google is now claiming Map Reduce and all the industry like Hadoop based on it are completely obsolete. They were heavily promoting their new Cloud Dataflow service where the claim this service can also do efficient real time analytics as well as preserve the performance of the main functionality.

It will be interesting to see what this new service can really do and will it really threaten all the various NoSQL databases like MongoDB.


There are a lot of interesting things going on in the mobile world. It will be interesting to see if all our phones are replaced by watches or glasses in a couple of years. It will be interesting to see what great things come of all these new cloud big data services.

Written by smist08

June 28, 2014 at 5:28 pm

Troubleshooting Problems in the Cloud

with one comment


Imagine that you are managing a large cloud based solution with lots of moving parts, thousands of users and fairly complicated business logic going on all the time. Now an irate customer phones into support complaining that something bad happened. Sometimes from the explanation you can figure out what happened quite easily. Sometimes what the user described sounds completely inexplicable and impossible. Now what do you do to troubleshoot the problem?

Easy Problems

First let’s review how easy problems are solved and can be dealt with quickly:

  • From the description of the problem, the programmer goes, oh of course we should have thought of that and on reviewing the code finds and easy fix which can be deployed easily. (Too bad this doesn’t happen more often).
  • The programmer scratches his head, but carefully goes through the customer’s steps and reproduces the bug on his local system, so he can easily investigate in the debugger and solve the problem.
  • The problem is a bit of a mystery so a programmer and support analyst using GoToMeeting to watch the customer work. On watching them work, they realize what this customer is doing differently to cause the problem and then the programmer can reproduce and debug the problem.
  • On examining the standard application logging, an unhandled exception is found in the log around the time the customer reported the problem and from this the code can be examined and the cause of the exception can be determined and fixed.

These are the standard and generally easy problems to fix. But what happens when these methods don’t yield any results?

Harder Problems

Here are some examples of harder problems that can drive people crazy.

  • The problem happens infrequently and can’t be consistently reproduced. But it happens frequently enough that customers are getting rather annoyed.
  • The system works fine most of the time, but every now and again, the system goes berserk for some reason. Then it just magically goes back to normal, or you need to restart the servers to get things back on track.
  • The problem only happens to certain users, perhaps at certain times. You can watch it happening to them via GoToMeeting, but can’t for the life of you get it to happen to you or to happen in your test environment and many other users never experience this.
  • Often problems aren’t outright bugs, but other strange behaviors, like the whole system suddenly slowing down for no apparent reason and then some time later it goes back to normal, again for no apparent reason.

A lot of time these sorts of things are due to interactions between what people are doing in different parts of the system. Predicting and testing all these situations is very difficult and often a result of emergent phenomena which we talk about a bit later.

Preventative Measures

Generally the only way to solve the hard problems is to instrument your web application so you know exactly what is going on. Not only does this help with solving hard problems, but it also helps with making using your web site a better experience for users, since you can log what they do, how long it takes and where they are getting stuck. Often usability problems are more serious to users than program failures or other bugs.

Instrumenting you application generally means having good logging and making sure you log a lot of metrics that you can monitor. This can also means having extra APIs to the running application where you can inquire on the state of various components, find out how many of one type of object is currently in use for instance. Often your infrastructure like your web server will do a lot of logging for you, so be aware of this as there is no need to log things twice.

Having a dashboard to track these metrics is very helpful.  The dashboard can both make reports and graphs from the application logs as well as use the application’s diagnostic API to provide useful information about what is happening. Often you can integrate with third party vendors like New Relic or Microsoft Application Insights, but one way or another you need this.


One objection to logging a lot of information is that the process of doing this can slow things down so much it becomes unacceptable. This is certainly true if you are logging synchronously to a file (i.e. not continuing on until the data is written). But modern systems get around this problems by logging asynchronously. The system doing the logging just fires the log message at a listener and goes on without waiting for or needing a reply. This causes logging to become much less a burden on the application. It then moves the problem to the listener application to log the messages, and if it gets too far behind it usually starts spilling messages. Most common logging infrastructures like Microsoft’s ETW already have this ability to take advantage of.

Some things you must track via APIs or logging:

  • Any resource used in the system. Java and C# programmers often think they can’t have leaks because of garbage collection, but garbage collection isn’t very smart and often important resources will leak due to unexpected references or a circular dependency. In a 24×7 web application that needs to essentially run forever in a busy state, this is incredibly important.
  • Generally what users are doing, so you know what the possible interactions might be and what are the concurrent things you may need to do to replicate the problem.
  • Any exceptions or errors thrown by the program this tends to be a given, but make sure you include programmatically handled errors since these can be useful as well.
  • Make sure you log performance metrics, if something takes longer than expected, log it.
  • Any calls to external systems and the results (with performance stats).
  • Assert type conditions where the program finds itself in an unexpected state (is make these get logged and not compiled out for production).
  • Anything the programmer considers a sensitive area of the program that they are worried about.

There are also some things that must not be logged, these include any sort of passwords, decryption keys or sensitive customer data (ie nearly all customer data). Generally a lot of people need to work with the logs and there can’t be any sensitive information there as this will be considered a major security problem.

Make sure you have some good software (either home grown, open source or commercial) to search and analyze your logs. These logs will get incredibly big and will need to be carefully managed (usually archiving them each day). There are many good packages to make this chore far easier.

Emergent Behaviors

Emergent behavior refers to complex unexpected behaviors arising from large sets of much simpler processes. Modern web applications are getting bigger and bigger with many moving parts and many interactions between different systems and subsystems. We are already starting to see emergent behavior arising. Nothing on the scale of the system suddenly becoming intelligent, but on the scale where predicting what will happen in all cases has become impossible. It doesn’t matter how much QA you apply, chaos theory and mathematics quite clearly state that the system is beyond simple prediction. That doesn’t mean it is unstable. If done properly your system should still be quite stable, meaning small changes won’t cause radically different things to happen.

The key point is to keep this mind when building your system, and to make sure you have plenty of instrumentation in place so that even if you can’t predict what will happen, you can still see what is happening and act on it.


Diagnosing and solving hard problems in a large web application with thousands of concurrent users can be a lot of fun and very challenging. Having good preventative measures in place can make life a lot easier. You don’t want to be continually pushing out newer versions of your application just to add more logging because you can’t figure out what is going on.

Apple Mobile Trends

with 2 comments


With Apple’s WWDC conference just wrapping up, I thought it might be a good time to meditate on a few of the current trends in the mobile world. I think the patent wars are sorting themselves out as Google and Apple settle and we are seeing a lot more competitive copying. Apple added a lot of features that competitors have had for a while as well as adding a few innovations unique to Apple.

The competitive fervor being shown in both the Google and Apple mobile camps is impressive and making it very hard for any other system to keep up.


Cloud Storage

Apple has had the iCloud for a while now, but with this version we are really seeing Apple leverage this. When Google introduced the Chromebook they used this video to show the power of keeping things in the Web. This idea has been copied somewhat by Microsoft. But now Apple has taken this to the next level by allowing you to continue from device to device seamlessly, so you can easily start an e-mail on your phone and then continue working on it on your MacBook. No having to e-mail things to yourself, it all just seamlessly works.

Apple also copied some ideas from Google Drive and DropBox to allow copying files across non-Apple devices like Windows as well as sharing documents between applications. So now this is all a bit more seamless. It’s amazing how much free cloud storage you can get by having Google, Microsoft, Apple and Dropbox accounts.

Generally this is just the beginning as companies figure out neat things they can do when your data is in the cloud. If you are worried about privacy or the NSA reading your documents, you might try a different solution, but for many things the convenience of this outweighs the worries. Perhaps a bigger worry than the FBI or NSA is how advertisers will be allowed to use all this data to target you. Apple has added some features to really enable mobile advertising, whether these become too intrusive and annoying has yet to be seen.

Copying is the Best Compliment

Apple has also copied quite a few ideas from Google, Blackberry and Microsoft into the new iOS. There is a lot more use of transparency (like introduced in Windows Vista). There is now a customizable and predictive keyboard adding ideas from Blackberry and Microsoft. Keyboard entry has been one of Apple’s weaknesses that it is trying to address. Similarly the drive option in the iCloud is rather late to the game.

Apps versus the Web

There is a continuing battle between native applications and web applications for accessing web sites. People often complain that the native mobile application only gives them a subset of what is available on the full web site, but then on the other hand the consensus is that the native mobile apps give a much better experience.

True web applications give a unified experience across all devices and give the same functionality and the same interaction models. This is also easier for developers since you only need to develop once.

However Apple is having a lot of success with apps. Generally people seem to find things easier in the Apple App store than in browsing and bookmarking the web. Apple claims that over half of mobile Internet traffic is through iOS apps now (but I’m not sure if this is skewed by streaming video apps like Netflix that use a disproportionate amount of bandwidth).

Yet another Programming Language

Apple also unveiled a new programming language Swift for mobile development. One of the problems with C, C++ and Objective C programming is the use of pointers which tends to make programming harder than it needs to be. Java was an alternative object oriented extension to C with no pointers, then C# came along copying much of Java. Generally most scripting languages like Basic, JavaScript, Python, etc. have never had pointers.

Rather than go down the road of Java and C#, Swift has tried to incorporate the ease of use of scripting languages, but still give you full control over the iOS API. How this all works out is yet to be seen, but it will be interesting if it makes iPhones and iPads really easy to program similar to the early PCs back in the Basic days.


The Internet of Things

Apple introduced two new initiatives, their Health Kit and Home Kit. Health kit is mostly to encourage adding medical sensing devices to your iPhone, whereas Home Kit is to extend iOS into devices around the home and to control them all from your iPhone.

The Health Kit is designed to centralize all your health related information in one central place. There is getting to be quite a catalog of sensors and apps to continuously track your location, speed, heart rate, pulse, blood pressure, etc. If you are an athlete, this is great information on your fitness level and how you are doing. Garmin really pioneered this with their GPS watches with attached heart rate monitors. I have a Garmin watch and it provides a tremendous amount of information when I run or cycle. I don’t think this is much use for the iPhone, which I always leave behind since I don’t want to risk it getting wet, but this might really take off if Apple really releases a smart watch this fall like all the rumors say.

Home Kit is a bit of reaction to Google buying Nest, the intelligent thermostat. Basically you can control all your household items from your phone, so you can warm up the house as you are driving home, or turn all the lights on and off remotely. We have a cottage with in-floor heating, it would be nice if we could remotely tell the house to start heating up in the winter a few hours before we arrive, right now it’s a bit cold when we first get there and turn on the heat. However with zoned heating we would need four thermostats and at $250 each, this is rather excessively expensive. I think the price of these devices has to come down quite a bit to create some real adoption.

There is a lot of concern about having all of these hacked and interfered with, but if they get the security and privacy correct, then these are really handy things to have.


Apple has introduced some quite intriguing new directions. Can Swift become the Basic programming languages for mobile devices? Will Health Kit and Home Kit usher in a wave of new wonderful intelligent devices? Will all the new refinements in iOS really help users have an even better mobile experience? Will native apps continue to displace web sites, or will web sites re-emerge as the dominant on-line experience? Lots of questions to be answered over the next few months, but it should be fun playing with tall these new toys.

Written by smist08

June 7, 2014 at 4:27 pm