Stephen Smith's Blog

Musings on Machine Learning…

Posts Tagged ‘amazon

Devices and the Cloud

leave a comment »

Introduction

We are currently seeing a proliferation of devices being release from a new line of Tablets from Amazon, new tablets and phones from Samsung, the iPhone 5 from Apple and a certain amount of anticipation for all the Windows 8 based devices that should be coming next month.

Amazon is quickly becoming a major player in the tablet market; they have branched out from just delivering e-book readers to having a complete line of high performance tablets at very competitive prices. Amazon then makes most of its money selling books, music and movies to the owners of these tablets. These are Android based tablets that have been setup and configured with a fair bit of extra Amazon software to integrate seamlessly with the Amazon store.

In the Android world, Samsung continues to deliver exceptional phones and tables of all shapes and sizes.

Apple has just shipped the iPhone 5 and we would expect new iPads sometime early next year.

Meanwhile Microsoft has released to manufacturing Windows 8 and devices based on this should be appearing on or after October 26.

With each generation of these devices we are getting faster processors, more memory, higher resolution displays, better sound quality, graphics co-processors, faster communications speeds and a plethora of sensors.

With so many players and with the stakes so high (probably in the trillions of dollars), competition is incredibly intense. Companies are competing in good ways, producing incredible products at very low prices and driving an incredible pace of innovation. People are also competing in rather negative ways with very negative attacks, monopolistic practices, government lobbying and high levels of patent litigation. The good news is that the negative practices don’t seem to be blunting the extreme innovation we are seeing. We are getting used to being truly amazed with each new product announcement from all these vendors. Expectations are getting set very high at each product announcement and launch event. There is collateral damage with once powerful companies like RIM or Nokia making a misstep and being left in the dust. But overall the industry is growing at an incredible pace.

Amazing new applications are being developed and released at a frenetic pace for all these devices. They know where you are, what you are doing and offer advice, tips or provide other information. There are now thousands of ways to communicate and share information. They all have deep integrations with all the main social media services. They communicate with natural language and high levels of intelligence.

The Data Cloud

Many of these devices are extremely powerful computers in their own right. The levels of miniaturization we’ve achieved is truly astounding. These devices are all very different, but what they share is that they are all connected to the Internet.

We now access all our data and the Internet from many devices, depending on what we are doing. We many have a powerful server that we do processing on, we may have a powerful desktop computer with large screen monitors, we may have a laptop that we use at work or at home, we may have a tablet computer that we use when travelling or at the coffee shop, we all have smart phones that we rarely use to phone people with, we may have a smart TV, we may have a MP3 player and so on. The upshot is that we no longer use exclusively use one computing device. We now typically own and regularly use several computing devices; it seems most people have a work computer, a home computer, a tablet and at least one smart phone. People want to be able to do various work related tasks from any of these. So how do we do this? How do we work on a document at work, then later we get a thought and want to quickly update it from our phone? Most of these devices aren’t on our corporate LAN. Typically we are connecting to the Internet via Wifi or via a cell phone network. The answer is that we are no longer storing these documents on a single given computer. We are now storing these to a centralized secure cloud storage (like iCloud, GDrive, or DropBox). These clouds are a great way to access what we are working on from any network and any device. Then each device has an App that is optimized for that device to offer the best way possible to work on the document in the given context. Many of these services like Google Apps even let multiple people work on these documents at once completely seamlessly.

Further many of these devices can keep working on our documents when we are off-line. For instance working on something on an iPad while on an airplane. Then when the device is back on-line it can synchronize any local data with the master copy in the data cloud. So the master copy is in the cloud, but there are copies off on many devices that are going on-line and off-line. Then modern synchronization software keeps then all in sync and up to date. Google had an interesting add for its Chrome notebooks where they keep getting destroyed, but the person in the add just keeps getting handed a new one, logs in, and continues working from where he left off.

The Grid

What we are ending up with is a powerful grid of computing devices accessing and manipulating our data. We have powerful servers in data centers (located anywhere) doing complex analytics and business processes, we have powerful laptop and tablet computers that can do quite powerful data input, editing and manipulation. Then we have small connected devices like phones that are great for quick inquiries or for making smaller changes. Our complete system as a whole consists of dozens of powerful computing devices all acting on a central data cloud to run our businesses.

In olden days we had mainframe computing where people connected to a mainframe computer through dumb terminals and all processing was done by the mainframe computer that was guarded and maintained by the IS department. Then the PC came along and disrupted that model. We then had separate PCs doing their own thing independent from the central mainframe and independent from the IS department. Eventually this anarchy got corralled and brought under control with networks and things like domain policies. Then we took a turn back to the mainframe days with Web based SaaS applications. Rather than run on the corporate data center, these run in the software vendors datacenter and the dumb terminal is replaced by the Web Browser. This then re-centralized computing again. Now this model is being disrupted again with mobile devices. Where the computing power is back in the hands of the device owners who now controls what they are doing once again.

The difference now from the PC revolution is that everything is connected and out of this highly connected vast grid of very powerful computing devices we are going to see all new applications and all new ways of doing things. It’s interesting how quickly these disruptive waves are happening.

Summary

It’s really amazing what we are now taking for granted, things like voice recognition, you can just ask your phone a question and actually get back the correct answer. The ability to get a street view look at any location in the world. It’s an exciting time in the technology segment with no end in sight.

In this articles I was referring to documents, which most people would associate with spreadsheets or word processing documents. But everything talked about here can equally apply to all the documents in an ERP or CRM system such as Orders, Invoices, Receipts, Shipments, etc. We will start to see the same sort of distributed collaborative systems making it over to this space as well.

Advertisements

Written by smist08

September 22, 2012 at 8:07 pm

Choosing Between Cloud Providers

with one comment

Introduction

It seems that every day there are more cloud providers offering huge cloud based computing resources at low prices. The sort of Cloud providers that I’m talking about in this blog posting are the ones where you can host your application in multiple virtual machines and then the cloud service offers various extension APIs and services like BigData or SQL databases. The extension APIs are there to help you manage load and automatically provision and manage your application. The following are just a few of the major players:

  1. Amazon Web Services. This is the most popular and flexible service. There are many articles on how much web traffic is handles by AWS these days.
  2. Microsoft Azure. Originally a platform for .Net applications, it now supports general virtualization and non-Microsoft operating systems and programs.
  3. Rackspace. Originally a hardware provider, now offers full services with the OpenStack platform.
  4. VMWare. Originally just a virtualization provider, has now branched out to full cloud services.

There are many smaller specialty players as well like Hiroku for Ruby on Rails or the Google App Engine for Java applications. There are also a number of other large players like IBM, Dell and HP going after the general market.

All of these services are looking to easily host, provision and scale your application. They all cater to a large class of applications, whether hosting in the cloud a standard Windows desktop application, or providing the hardware support for a large distributed SaaS web application. Many of these services started out for specific market niches like Ruby or .Net, but have since expanded to be much more general. Generally people are following the work of Amazon to be able to deploy seamlessly anything running in a virtual machine over any number of servers that can scale according to demand.

Generally these services are very appealing for software companies. It is quite expensive and quite a lot of trouble maintaining your own data center. You have to man it 24×7, you are continually buying and maintaining hardware. You have to have these duplicated in different geographies with full failover. Generally quite a lot of activities that distract you from your main focus of developing software. Fewer and fewer web sites are maintaining their own data centers. Even large high volume sites like NetFlix or FourSquare run on Amazon Web Services.

Which to Choose?

So from these services which one do you choose, how do you go about choosing. This is a bit of game where the customer and the service provider have very different goals.

For a customer (software developer), you want the cheapest service that is the most reliable, high performance and easiest to use. Actually you would always like the cheapest, so if something else comes along, you would like to be easily able to move over. You might even want to choose two providers, so if one goes down then you are still running.

For the service provider, they would like to have you exclusively and to lock you in to their service. They would like to have you reliant on them and to attract you with an initial low price, which then they can easily raise, since switching providers is difficult. They would also like to have additional services that they can offer you down the road to increase your value to them as a customer.

OpenStack

Both Amazon and Azure look to lock you in by offering many proprietary services, which once you are using, makes switching to another service very difficult. These are valuable services, but as always you have to be careful as to whether they are a trap.

Amazon pretty much owns this market right now. New players have been having trouble entering the market. Rackspace suddenly realized that just providing outsourced hardware wasn’t sufficient anymore and that too much new business was going to Amazon. They realized that creating their own proprietary services in competition with Amazon probably wouldn’t work. Rackspace came up with the disruptive innovation of creating an open source cloud platform called OpenStack that it developed in conjunction with Nasa. They also realized that so many people were already invested in Amazon that they made it API compatible with several Amazon services.

OpenStack has been adopted by many other Cloud providers and there are 150 companies that are officially part of the OpenStack project.

This new approach has opened up a lot of opportunities for software companies. Previously to reduce lock-in to a given vendor, you had to keep you application in its own virtual image and then do a lot of the provisioning yourself. With this you can start to automate many processes and use cloud storage without suddenly locking yourself into a vendor or to have to maintain several different ways of doing things.

Advantages for Customers

With OpenStack, suddenly customers can start to really utilize the cloud as a utility like electricity. You can:

  1. Get better geographic coverage by using several providers.
  2. Get better fault tolerance. If one provider has an outage, your service is still available via another.
  3. Better utilize spot prices to host via the lowest cost provider and to dynamically switch providers as prices fluctuate.
  4. Have more power and flexibility when negotiating deals with providers.
  5. Go with the provider with the best service and switch as service levels fluctuate.

One thing that scares software companies is that as soon as they commit to one platform, then do a lot of work to support it, then suddenly have a new service appears that leapfrogs the previous services. Keeping up and switching become a major challenge. OpenStack starts to offer some hope in getting off this treadmill, or at least making running on this treadmill a bit easier.

Is OpenStack Ready?

At this point OpenStack doesn’t offer as many services as Azure or AWS. Its main appeal is flexibility. The key will be how well or the major companies backing OpenStack can work together to evolve the platform quickly and how strong their commitment is to keeping this platform open. For instance will we start to see proprietary extensions in various implementations, rather than committing back to the home open source project?

Amazon and Azure have one other advantage, and that is that they are subsidized by other businesses. For instance Amazon has to have all this server infrastructure anyway in order to handle the Christmas shopping rush on its web store. So it doesn’t really have to charge the full cost, any money it makes off AWS is really a bonus. By the same token Microsoft is madly trying to buy market share in this space. It is taking profits from its Windows and Office businesses and subsidizing Azure to offer very attractive pricing which is very hard to resist.

Apple uses this strategy for iCloud. iCloud runs on both Amazon and Azure. This way it isn’t locked into a single vendor. Has better performance in more regions. Won’t go down if one of these services goes down (like Azure did on Feb. 29). Generally we are seeing this strategy more and more as people don’t want to put their valuable eggs all in one basket.

Summary

With the sudden explosion of Cloud platform providers, suddenly there are huge opportunities for software developers to reduce costs and expand capabilities and reach. But how do you remain nimble and quick in this new world? OpenStack provides a great way to provide a basis for service and then allows people to easily move to new services and respond to the quickly changing cloud environment. It will be interested to see how the OpenStack players can effectively compete with the proprietary and currently subsidized offerings from Microsoft and Amazon. Within Sage we currently have products on all these platforms. SalesLogix cloud is on Amazon, SageCRM.com is on Rackspace and Sage 200 (UK) is on Azure. It’s interesting to see how these are all evolving.

Accpac on the Amazon Cloud

with 10 comments

Introduction

The Amazon Elastic Compute Cloud (EC2) (http://en.wikipedia.org/wiki/Amazon_Elastic_Compute_Cloud) is a service offered by Amazon.com that allows people to rent virtual computers to run applications on. Some of the innovations offered by this solution include:

  • Very easy to get started, you just need an Amazon account, attach it to EC2, and off you go.
  • Very inexpensive, with a good (nearly) free trial (http://aws.amazon.com/ec2/pricing/).
  • Scalable and expandable depending on your needs.

Often the simplicity of getting started with this solution gets lost, since people are usually confronted with the advanced features at the beginning, which you don’t need to worry about until later. Just be re-assured that this is a solution that can grow with you. Below is a diagram of some of the services offered:

In this blog posting we will look at how to deploy Accpac on the Amazon EC2 cloud and discuss some of the trade-offs and choices that can be made along the way.

Terminology

One thing that makes using Amazon EC2 intimidating is the terminology. But here is a quick guide to the key points.

  • Amazon Machine Image (AMI) – These are virtual machine snapshots that you take as a starting point to doing work. Amazon provides a number of these as starting points, there are a number of public ones offered by other people plus you can create your own. Basically when you want a new virtual machine you take one of these as your starting point.
  • Instances – You create an instance from an AMI and the instance is the virtual machine that you actually run. When you specify the instance you specify the resources it has including memory, disk space and computing power. For more on the instance types see: http://aws.amazon.com/ec2/instance-types/.

You manage all these things from the Amazon Management Console:

Deploying Accpac

Deploying Accpac to Amazon EC2 is fairly straight forward. You just need to select a starting virtual image (AMI) of something that Accpac supports, create an instance of that, run the instance, install and configure Accpac into that image and off you go. There are a couple of “gotchas” to watch out for that we will highlight along the way.

  1. Go to http://aws.amazon.com/ec2/ and sign up for an account.
  2. Run the AWS Management Console (https://console.aws.amazon.com/ec2) and create a PKI security key pair. You will need to do this before doing anything else. This will be the security token you use to connect to your virtual image running on EC2.
  3. On the upper left of the management console, make sure it is set to the region that is closest to you like perhaps “US West”.
  4. Click the “Launch Instance” button on the AWS Management Console. You will now be prompted to choose a starting AMI. A good one to choose is: “Getting Started on Microsoft Windows Server 2008” from the Quick Start tab. This one has IIS and SQL Server Express Installed.
  5. Select “Small” for the instance type, unless you know you will need more resources quickly. Then accept the defaults for the advanced instance options. Same for the tags screen (i.e. you probably don’t need any).
  6. On the “create key pair” screen, select the key you created in step 2 (or if you skipped that then you need to create a pair now).
  7. On the configure firewall screen, remove the opening for SQL Server, you don’t need this. The only two holes in the firewall should be RDP and HTTP. If you are hosting client data, then you should add HTTPS and setup Accpac to use that (see https://smist08.wordpress.com/2010/11/20/setting-up-sage-erp-accpac-6-0a-securely/).
  8. Now you can review your settings and Launch your instance. It can take 15 minutes or so for the instance to launch, mostly due to the time it takes Windows Server 2008 to boot. So this is a good time to go get a coffee.

At this point we have created a virtual image and have it running. From the AWS Management Console EC2 dashboard, we should see one running instance. We should also see 1 EBS volume. The EBS volume is the disk image of your virtual image. If you want to you can create snapshots of your EBS volume (you have to pay to store these) so you can go back to them if you mess up your image. So now we have our own Windows 2008 server running in the Amazon cloud, great, but now what do we do? How do we connect to it? How do we transfer files to it? How do we browse to it? What are the Administrator and SQL Server passwords? Now we’ll go through the steps of getting the Administrator Password, connecting via RDP and installing Accpac.

  1. Select the instance that you have running in the management console. From the instance actions menu, choose “Get Windows Admin Password”.  If this doesn’t work, you may need to give the instance a bit more time to start. You will get a dialog that wants you to take the file you downloaded back at step 2, load it into notepad and copy/paste its contents into this dialog. Then this dialog will go off and do a long cryptographic calculation and tell you the Windows Password.
  2. Now you can run Remote Desktop and connect to your instance (if you choose Connect from the instance menu it will download a file that will start RDP with the right parameters). Use the public DNS as the computer name (from the pane with the instance details below the instance list). Administrator is the login. Be careful because copy/pasting the password can be difficult because Windows tends to add a space when you copy the password. If copy/paste doesn’t work, try just typing the password. Now you are logged in and running. Perhaps the first thing you want to do is change the Administrator password to something easier to type and remember. Now you can treat this virtual Windows Server 2008 just like any other remote server.
  3. Copy the installation image for Accpac onto the virtual machine. You can use an FTP site or any other file copy mechanism to do this. On convenient method that Windows 7 has is that RDP can make local drives accessible to the remote computer. If you choose Options – Local Resources you can expose some drives to the remote computer and then they will show up in Windows Explorer there.
  4. Now we need to enable SQL Server, by default the service is disabled and authentication is set to Windows Authentication only. Go to Admin Services – Services and set the SQL Server services to Automatic and start them. In the SQL Server configuration manager enable TCP/IP and set the port to 1433. In the management console set the authentication to SQL Server and authentication, then go to the sa user and enable it. Now restart the SQL Server service. Create your Accpac databases such as PORTAL, SAMSYS, SAMINC, SAMLTD, …
  5. Run the Accpac installation you copied into the image and perform the usual steps to get Accpac up and running. When running database setup, make sure you use localhost as the server name and not the current Windows instance name, because this will change each time you run the image.

We now have Accpac up and running and can access Accpac via RDP. To access the Portal use the public DNS as the server name in the usual URL for running the portal:

http://<public_DNS_goes_here>/SageERPAccpac/portal60a/portal.html

Viola you are running in the cloud.

If you shutdown this instance and restart it, you will get a new computer name and a new public DNS. This can be rather annoying if you like to setup Browser shortcuts and such. If you want to avoid this you need to allocate a static Elastic IP address from the AWS (doing this costs a small amount of money). Then you can associate this IP address with the instance and now it will stick. Further you could purchase a meaningful URL and associate it with this IP address. If you don’t want to purchase a URL, another trick is to use TinyURL.com to generate a URL for your IP address. This isn’t a very meaningful URL but it’s better than the raw IP address.

How Well Does It Run?

Once running, how does it compare to a local server? With the small configuration you are limited a bit in memory. It seems that running the Sage ERP Accpac 6.0A portal on the virtual image in a RDP image is a bit slow.  However running the browser locally and hitting the server remotely is quite quick. This implies that the small image is sufficient for the server processes for a few users. However you will need to increase the memory and/or the processing power for more. The nice thing with Amazon is that you can change this fairly easily and only pay for what you are using. It also shows that the Amazon datacenters have quite good network latency, probably better than you can get hosting yourself for remote users.

Going Production

So can you go production with this? Certainly the platform can support it. The current sticking point is terminal server or Citrix licenses. These are available through various programs such as: http://community.citrix.com/pages/viewpage.action?pageId=141100352. However you need to be part of one of these Microsoft or Citrix programs where they give you specific permission to migrate your licenses to EC2. While we still have Windows Desktop components this is a potential sticking point. However once Sage ERP Accpac 6.1A comes out and we can run all the main accounting applications through the web, then this problem goes away.

Amazon is also addressing other compliance type concerns, for instance achieving PCI DSS Level 1 Compliance (http://aws.amazon.com/security/pci-dss-level-1-compliance-faqs/?ref_=pe_8050_17986660) and ISO 27001 Certification (http://aws.amazon.com/security/iso-27001-certification-faqs/?ref_=pe_8050_17986660). Receiving these sort of certifications remove a lot of obstacles to using Amazon for a production environment.

Also if you want to back up your data locally then you will need to copy a backup of your SQL Server database over the Internet which could be quite time consuming, but you can let it run in the background.

Summary

Amazon’s EC2 Service offers an excellent way to access extra computing resources at a very low cost. You can deploy services to regions around the world and dynamically adjust the computing resources you are using. For developers this is a very cheap way to obtain access to test servers when in development. For partner this is an excellent way to establish demo servers. For education this is an excellent method to learn how to work with different operating systems and to practice installations.

Written by smist08

December 17, 2010 at 9:41 pm