Stephen Smith's Blog

Musings on Machine Learning…

Archive for March 2009

On Paranoia and Security

leave a comment »

I bought Bruce Schneier’s boxed set of three books: “Applied Cryptography”, “Practical Cryptography” and “Secrets and Lies”. Hopefully reading all this will make me sufficiently paranoid to deal with the security threats we’ll be facing as we move into the SaaS world. Bruce says that originally when he wrote “Applied Cryptography” he thought all the security problems on the Internet could be solved by Mathematics. That a few powerful cryptographic algorithms would solve all the security problems out there. He now realizes that this isn’t the case. That there are so many other weak links to be exploited, like bad implementations, lack of vigilance, human error, etc. In fact many people feel that if they are connected to a web site via SSL or TLS that they are fully secure. However this just isn’t the case.

SSL and TLS only protect the connection between the client computer and the server. They don’t protect the client computer. They don’t secure or encrypt data stored in the Browser’s memory. They don’t force you to use secure passwords. They don’t force you to check the validity of the root certificate authority used by a server. They don’t force you to use the maximum encryption settings possible. They don’t force you to run anti virus and spybot software.

Generally the security business is described as a “Red Queen’s Race”. This means the people trying to protect systems are running harder and harder just to stay in the same place. It seems the advances made by hackers are very impressive. Even now that most crytographic algorithm’s patents have expired and governments aren’t trying to supress them anymore as military secrets, that it will take much more than mathematics to provide a secure Internet.

Another point he makes is that it is possible to create a secure operating system. But since there is no liability to software vendors when break-ins occur, that there is no real motivation for anyone to make a more secure operating system. For instance it would cost Microsoft billions to really address the problems in Windows, but since there isn’t any liability, besides a bit of bad press, why would they? As it is they are content just to spend a bit of time releasing Windows Updates they discover holes found by hackers. But what about all the holes that hacker’s have found and not told them about?

But in spite of all the negativity, it is possible to create a reasonably secure system (ie secure enough that hackers will look elsewhere for easier targets). With a reasonable amount of vigilance, following of best practices and intelligence, you can run a secure system. But you have to stay alert and not believe that SSL and a firewall are all protecting.

Written by smist08

March 28, 2009 at 4:08 am

Posted in Security

Tagged with

On Adding Features to a Large Existing Application

leave a comment »

Designing a new application from scratch is often a much easier job than adding features to a large existing application that has been around for quite a few years. For a given new feature you have to consider how it interacts with all the existing features. How easy is it for an existing user with a huge database to use this feature, does he have to go through thousands of records entering new data? Or can this be done automatically? If an existing feature is close, can it be modified to the new desired functionality? Or will doing so disrupt a number of users, who use this existing feature for some unexpected purpose.

The trade-offs can be complex and will affect different user communities differently. For instance it might be easier to add a field onto the side so you don’t disrupt any functionality existing users may or may not be  using. But then you could impose a heavy data conversion cost to users with large databases when they adopt the new users. Plus it can be confusing for new users to have to scratch their heads about the existing feature and whatever it does plus the new feature. When do they use which? Why do they need to fill in two fields for each new record? When new fields get added this year, over time this feature creep can make applications very complex. Keeping control of this with fewer more powerful features can be a major challenge.

There is also the question of making a new feature flexible versus making it easy (or automatic) to just work with no user configuration. If there is configuration, the feature can be quite flexible and provide quite a lot of functionality that is adaptable to different business needs. But the cost now is learning how to do this configuration, and having to actually do it. Ie the feature doesn’t just work out of the box. But when you do this, you have to make a lot of assumptions on what business’s need. Have you made the right decisions? Or have you made the new feature useless, since it doesn’t adapt to the businesses exact needs?

Just some of the fun things that need to be considered as we add new features to our existing applications.

Written by smist08

March 21, 2009 at 11:02 pm

Sage Spark Live

with one comment

I went to the launch party for Sage Spark last Friday. Lots of fun, music, tatooing, food, etc. Sage Spark builds on Sage Billing Boss which is a free online invoicing program. This website is online now in beta testing at It builds a whole set of business services around Billing Boss to provide a much more complete portal of services for small businesses. It is oriented to businesses who are either too small for Simply Accounting, or aren’t interested in doing full accounting themselves at this point. Basically it lets them send invoices and process credit card orders. These businesses probably then just throw all their invoices over the wall to a book keeper or accountant to do their books ever year.

Sage Spark then provides a market place and connections to vendors like bookkeepers so these businesses can get the work they need done , done without wasting their precious time. There are links to get IT services, sales and marketing services. Basically so a business can just concentrate on their core business whatever that might be (perhaps rebuilding engines or knitting sweaters), and then basically outsource all their other business needs. So they can concentrate on what they do well and other professionals (similar small businesses) handle all the other parts.

With social networking and the connectivity of people, this is really becoming the way things are getting done these days. Definitely a valuable and innovative service.

Written by smist08

March 19, 2009 at 3:51 am

Posted in Business

Improving Product Quality

leave a comment »

Improving product quality is always a hugely controversial topic. Not the desire to improve quality, which everyone agrees should always be a continual process. The controversy comes in on how to improve quality. For instance does fixing lots of minor issues (bugs) improve the quality of the product? Does adding features customers are missing and demanding add to the quality of the product? Do you need to really go deep and address underlying problems of reliability and performance? All of these need some attention. But how do you decide where to spend your time? For instance is a 20% increase in performance equal to 30 minor bugs being fixed? Is there a formula to decide these things?

I think to truly affect quality, it must be from the customer’s perspective. Sorry QA (Quality Assurance Department), but this usually isn’t strongly related to bug counts in the bug tracking system. You really need to talk to customers to determine which parts of the product are adversely affecting them. Is it due to a procedure that has too many steps? Is due to something they perform over and over being slow? Is it lots of minor annoyances in fit and finish?

The real measure of quality is customer satisfaction. And to improve customer satisfaction, you really need to go to the customer and find out what is bugging them. Deciding to improve quality by deciding to do an extra month of general QA just doesn’t provide the desired results. Often the customer’s main quality perception problems have nothing to do with the product, but are due to say receiving a bad invoice or not being able to get through on a customer support line. Sometimes just the frequency of hotfixes and service packs can affect perceived quality. These areas need to be addressed, and often can have an effect equal to quite a large number of real bugs in the product.

Quality is a continuous process. It isn’t just the testing phase of a new product release. It has to permeate all aspects of the company and its culture. You need to be actively working to improve a products quality at all times. You have to continuously be listening to your customers, so you can respond to their needs and pain points. You need to make sure your finite resources are focused on solving the real customer issues and not just applied randomly. This is always a challenge, but it is one of the things that makes software development fun and rewarding.

Written by smist08

March 14, 2009 at 4:30 am

Difficulties with SQL Statements

leave a comment »

In the good old days, database APIs separated the commands from the data, so usually you passed in a command like update or read to one parameter and the data to update or read in a separate parameter as a data buffer. In this case the data wouldn’t interfere with the commands or cause any sort of code execution.

Now in the brave new world (well not that new really), data is mixed with commands in the form of SQL statements that are passed to the database as a single string containing both the data and the command, like “SELECT * FROM ARCUS WHERE IDCUS = 1200 AND SWACT = 1”. Now if the data is manipulated by hackers they can insert additional commands to be executed.  For instance in the string before what if the use typed “1200 OR 1=1”, then it would return data for all customers. To avoid this you put the data in quotes like “IDCUS = ‘1200’. But then what if quotes are in the string? The solution is to “escape” them. For SQL this means doubling them up, ie changing ‘ to ”.

Theoretically this should make the SQL statement secure. However there are some subtle points, such as if the string is all single quotes then you need a buffer twice the size of the original to handle the data. What about leading and trailing spaces, how should these be handled? Is escaping single quotes sufficient? Are there other “special” characters that need to be dealt with?

Generally if you know your database you can handle this easily. However a gotcha can happen if you change databases. For instance the scheme above works fine for SQL Server and TSQL, but if you switch to MySQL, it becomes inadequate because \ is a valid escape character. So if the user enters \’, it gets changed to \” which now leaves one unquoted quote for hackers to maliciously use.

Besides user data causing invalid SQL statements, hackers can exploit these problems to find ways to execute extra code. If you can close the quote then the can add any SQL command they like, resulting in a SQL Injection attack.

Anyway it adds to the consequences of getting it wrong. Some databases provide routines to safely quote strings in their APIs. Its best to use these, since the database vendor should know all the ways to escape characters (documented or not) and hopefully these routines have been well tested (including by white hat hackers).

Written by smist08

March 7, 2009 at 4:49 am