Sunday, 14 December 2014

User Interface Usability Checklist Part 1

User interfaces are changing quickly, and it can be default to keep up with new designs and trends. The smaller the team the more corners are cut to try and get the product to market. Lots of people advocate the creating and releasing the minimum viable product. This is fine, but companies try to do so without the minimum viable staff level.

Creating a great user interface takes good design, user testing and lots of iteration. Unfortunately user testing is often the first "cost" that is abandoned. The next is the available design time. So when you become incredibly constrained then making good design choices becomes your only course to prevent a horrible mess at the end of the project. I have created a usability check list of common design mistakes. I have seen all of these mistakes in projects from the most expensive to the most basic free apps, and if you can avoid all of them you are well on the way to producing a high quality application.

User Interface Usability Checklist

 1: Using checkboxes where you should use radio buttons

 Checkboxes imply that you can select more than one item, radio buttons indicate that only a single option from the list is available. Sadly too many interfaces have single selecting checkboxes. Radio buttons are also often not the best option, sometimes icons or hyperlinks are better interaction options. I have generally found that radio buttons are one of the least understood generic UI elements, only multi-select lists cause more confusion/

2: Using a checkbox out of enable/disable context

It is important to take care in making sure your checkbox is an obvious boolean state switch, depending on the wording it can be unclear what occurs when you unselect the checkbox. e.g. a checkbox with the label Always print receipt slip. When it is on you know what will occur, but with it off will it never print a receipt or sometimes print a receipt? Remove the ambiguity by renaming the checkbox or provide a radio button with all of the options available.

3: Using command buttons as toggles

Toggling states tends to cause confusion and mistakes, more so when you are using a magically re-naming button.

4: Using tabs as radio buttons

This can cause confusion because the standard use of tabs is for them to just break up areas of the same object into different visual regions of information. If you use them as a data manipulation method to achieve the same type of functionality as a radio button this will confuse users as they will be unsure of what will happen with the "hidden" data, is it saved to the system or not?

5: Too many tabs

Determining if an interface has too many tabs is generally down to individual use cases, however, I have a basic rules of thumb. If you require 5 tabs to display your interface it is important to think is this the best way of presenting the data. More than 5 tabs starts implying that you should reconsider how the interface is displayed. Of course this can be broken, but only with careful consideration.

6: Using input controls for display only

Sometimes people use disabled input fields to display data that cannot be edited. Disabled fields imply there is a way of unlocking them for access, if this is not the case then disabled fields should not be used. However, sometimes people take this one step further and use input fields for display text. input fields should only be used if data entry is a possibility.

7: Always using text fields when data entry needs to be constrained

HTML5 introduced a number of field types, such as email. It is important to use these rather than a generic text field with client and/or server validation (or worst still no validation), especially in the mobile context. Try typing an email address in a standard textbox on a mobile device and watch the spelling correction mess up your entry. Highly frustrating for any end user I am sure you will agree.

8: Disappearing menu items

While contextual presentation of information can be useful in aiding the understanding and interaction flows of an application, it can also hide the available actions and cause confusion in long term users who memorise the application's menu structure. Microsoft XP's start menu is a prime example of this with its short and long menus.

9: Intolerant data fields

I am sure many people have experienced the following problems.

  • Your password must be between 8-10 characters and must not contain $*!. It can be very frustrating trying to create a memorable password that meets those constraints.
  • Enter your phone number, but I'm not going to tell you if I will accept with or without international code until you fail validation as the end of the form.
  • That post code you just used which works on ever other website is not a valid post code. A common mistake is needing to type in post codes in a specific format. XXXYYYY is invalid but XXX YYYY is fine... 

Try to be intelligent in validation so that the user does not get told they are wrong when the computer can easily check to see if a minor reformatting to match the database requirements is needed.

10: No defaults for input fields

Providing defaults can really speed up the interaction with your systems design. Consideration should be placed into supplying defaults.

11: Poor defaults

Defaults are important in allow appropriate customisation of your application, while allowing users to quickly access and use your product. Poor defaults can make your application look rubbish and make people think that the desired behaviour does not exist in your application. Excel's default behaviour regularly annoys me and has caused countless support issues in many companies. If you type in a number such as a phone number it will happy cut off leading zeros and stop the number from having its original meaning. I do not understand this default behaviour, people do not just enter leading zeros for fun, they have a purpose. This is just one of many default behaviours which are undesirably in Excel.

12: Negative checkboxes

Yes there are plenty of websites that have negative checkboxes, check this box to not receive a torrent of useless span from us, they do this to try and trick people into checking the box when they do not want to. Use positive checkboxes to help make sure your users are making the correct decisions rather than checking the box by mistake.

13: Check the titles are right

We have tabbed browsers, multi-window applications, try to identify the tab or window by putting something a little more useful than the domain or application name in the Window / Brower title information.

14: Don't use self-links

It can cause people to consider that they might get directed to a different version of the page you are on. If the page changes regularly then a self link might seem a simple solution to get the page refresh, however, if possible aim for a more dynamically updating page to assist the user in understanding that there is new/changed content in relation to the context of the page.

15: Competing search boxes

Multiple search boxes on a page can often cause problems, using the wrong one can at a minimum make you have to go back, or at worse think there is no information available for a search when there is and so make incorrect decisions from this.

16: Poor search results

Searching is not a simple matter. Most people have no experienced Google and realise that it is possible to have relevance ranked, spelling corrected, sentence contextual returned results. As a developer we realise it is not that simple to achieve a search as good as Google, but often you will have the benefit of a less varied set of data to search over than every web page in the world and so can create a pretty good search rather than just a basic exact text match.

17: Inconsistent terminology

This is incredibly common, and can cause failed interaction very easily. Many terms get switched around interchangeably; user, administrator, client, customer. Try to use just one term depending on what you are referring to, and importantly make sure the whole company agrees, without this agreement the multiplicity of terms will slip into the product.

18: TL;DR (too long didn't read)

Try to make every word count. Too many words can introduce ambiguity, so always try to be short and concise. This requires careful attention and revisiting of old designs to make sure you adhere to this principle.

19: Speaking the language of your audience

Your language should be directed to your specific audience. As a user interface designer you probably know the term "radio button" but if your audience is not part of the programming community they probably do not. One of my favourite examples of this is when a friend of mine was dealing with a support call and he said over the phone "press the windows key". The response was that this did not do anything. As he was not progressing on the phone he went to the person's desk to see this for himself. He found out that "press the windows key" had been interpreted as "press the key that is tapped to the window". Not many people know what the windows key is despite every Microsoft based computer keyboard having one for ages.

20: Vague or misleading error messages

First on the programming cost saving list is error handling. It is sad but true, but clear error messages directing the end user in what they can do to fix the problem is essential. It is very frustrating to get an error message and have no real idea what to do next. What can be worse is a poorly programmed error which suggests the cause is one element, when in fact it is another. These are much harder to deal with and tend to only be picked up if you have a good test team with a good test plan.

Tuesday, 4 November 2014

Why should you use PDO?

I was recently asked if I could think of any good questions to ask a PHP programmer in an interview. Obviously this got me curious as to what Google might return for this query.

I noticed that a few people recommended asking questions in relation to PDO. All people suggesting questions on PDO were forming questions where the answer essentially was that PDO should be implemented so that switching database is easy/easier.

I have worked on numerous commercial products for a number of years and none of these have ever switched database.

How common a change it is to switch database on a project? I can't imagine the % is very high, although I have not been able to find any figures on this. Additionally of the systems that require switching I suspect they are probably moving from an archaic language / database which does not support the system they are moving to, and so you are largely doing a complete rewrite anyway. Will SuperSQL++ 2040 support your PDO queries from PHP 5.6? Perhaps a database abstraction layer is of minimal real benefit?

There are only 3 real reasons for switching database that I can think of:

Proprietary features
If you want to use features that only exist in a specific database then generics are not the answer, use of proprietary features implies PDO is not useful in this case.

Cost
A reasonable cause for switching DB vendor, but how often will payment terms be altered over the lifetime of a product? Is switching worthwhile vs the cost of testing, development and deployment, even if you did have PDO implemented?

Support
If the DB is no longer supported then sticking to it becomes a significant business risk, however, if you are using a system that is no longer supported and PDO would have been beneficial then you really did back the wrong horse. Most databases have been around for many years and I doubt any of the PDO driver ones are going away soon.

So does PDO bring any other benefits?

Well it does have named prepared statements. These are superior to MySQLi's "?" sequencial parameters.

Then there is client side prepared statements, this can actually lead to improved performance in PDO statements over using MySQLi.

What are the Costs?

Implementing PDO in terms of coding does not really add any noticeable cost, however, there are resource costs which should be considered. There is a small overhead on every PDO command versus using proprietary extensions. During my testing I have found this overhead to be 2-10% depending on the nature of the query. This essentially translates to a 2-10% increased database server resource cost. Some examples can show PDO is faster, but I suspect that most people would agree that overall in an application there is a small percentage loss in performance.

Perhaps more significantly you can't use proprietary features. There are minor inconviences like not having num_rows() for select statements. 

Then there are major issues like not being able to perform asynchronous queries. Depending on your use case you can see up to 50% performance increases performing asynchronous MySQLi queries.

So who wins?

While I think there are always more important things to do than change database easily having the option is always a benefit, even if it is a very small one. I would think that a 2% performance overhead would be enough to displace this benefit, but when you consider that you might be turning your nose up at 50% performance increases then PDO quickly becomes the loser.

I am curious if there is any evidence to the contrary to my believe that switching DB is uncommon, and perhaps more importantly if I have missed any other benefits that using PDO may have over MySQLi.

Tuesday, 5 August 2014

More weird adventures in Cache SQL - SOLVED

Well made a new interesting discovery today. Combining TOP with ORDER BY can severely slow down a query.

I was looking into a slow query which was taking around 10 seconds to return 69 results. It was getting the "TOP 100" from a specific date range and then ordering by ID descending. It was fascinating to me to discover that removing the TOP 100 returned the results in 0.005 seconds. Removing the ORDER BY instead of the TOP 100 also resulted in again producing the results in 0.005 seconds.

So the moral of the story is don't use TOP and ORDER BY if you want your results back quickly.

I have no idea why this kills performance right now, but it is perhaps even more interesting that even though the query TOP 100 ORDER BY ID ASC results in the same set of results as TOP 100 without an ORDER BY command, it takes 8.8 seconds rather than 0.005 seconds. There are definitely some huge optimisations available in respect to these queries.

Update

Well after reporting this to Intersystems I was directed to %NOTOPOPT query optimisation method. It states that without this TOP is optimised for first row retrieval rather than the full result set. This implies to me that a TOP 1 should be faster without %NOTOPOPT and that returning more than 1 has the chance to be faster with it. However, when I was testing across various queries whether it was TOP 1 or TOP 100 with an ORDER BY I always retrieved the results much faster with %NOTOPOPT. If you have a TOP query running slowly I recommend testing with this optimisation option, as you may see much better performance as I did.

Saturday, 2 August 2014

So which is the best mobile keyboard 2014

PhotographerCindy J Grady
I have been lucky enough to experienced using keyboards on iOS, Android and Windows Phone (8 and 8.1) which has really shown me how important a good input method is in the mobile experience.

Each keyboard has unique features which makes choosing the best keyboard a bit of a struggle. Until recently I had completely dismissed iOS keyboard for the following reasons

1. 4" display means that the keys are harder to press accurately
2. Selecting & deleting words is trickier than both Android and Windows Phone not just due to the smaller target area, but also the implementation requires a "long press" to activate the magnifying glass to achieve accurate cursor location.
3. Lack of swype / shape writing style input.

While options 1 and 2 were of just minor inconvenience option 3 is a major issue.

Swyping / Shape writing

There are three great features of shape writing

1. It is a much faster input method on the whole than key pressing as demonstrated by the numerous keyboards of this style achieving Guiness World Records.

2. It allows a more generalised area of keypress which is of great advantage on a smaller screen, or even allows you to cope with a smaller keyboard on a larger screen giving the benefit of seeing more of what you have typed so far.

3. The input can be done at full speed with a single digit. Rather than the common double thumb entry for iOS to generate speed shape style writing can be performed one handed.

What Does iOS have up its sleeve

Historical Autocorrect

Obviously we have all had a laugh at damn you autocorrect and probably have received a couple of weird messages due to autocorrection errors. However, autocorrection is pretty clever, but on iOS it actually tracks back up to 4 words based on what you are entering and can correct the 4th word. Contextually autocorrecting 4 words back will tend to help weed out a huge number of errors, and is certainly a clever feature to have,

Cross Platform Support

Apple has always integrated its devices well, anyone with an iPad, Apple TV and Mac will know that so many things transfer relatively seamlessly across without the need to be tech-savvy. iMessages synchronising with your devices is a good example of this. Well the iOS keyboard has another one of these clever features, simply create keyboard macros on your Mac and they are straight away available on your iPhone and iPad.

While the stock Android keyboard offers keyboard macros (as well as some less popular alternatives) these macros are limited in length. If for example you had a long HTML sign off signature you were particularly proud of this would probably be too big for the dictionary. However, Kii keyboard will store macros that are large enough, but the dictionary does not sync across devices, so you need to re-enter any macros on all devices you want to use.

Is this enough?

I can understand that these features can be sufficient to turn the tide for some, but shape writing is such a massive time saver for 99% of the text I input that synced macros is simply not important enough. So sadly iOS keyboard is ruled out of the running, leaving Google Keyboard, Windows 8.1, Swype, Swiftkey, SlideIT & Kii.

Google Keyboard

Google have built a very good basic keyboard. It is fast and responsive, even on lower end devices, which is not true of some of the other options. I have found it to be quite accurate and fast, and is my recommendation if you have a slower device as there is nothing quite as frustrating as a sluggish keyboard response.

SwiftKey

SwiftKey recently became free, which is quite a bonus in its favour. The next work predictions are quite scary, it makes you realise just how much you repeat yourself and various phrases. I find this aspect a bit creepy almost as if it knows what you are about to say. It does however, inspire me to try and vary my language a bit more almost like a challenge to say, "No you are wrong! I was actually going to use a 16th century old English word, take that dictionary!"

This is probably the second best keyboard for non-shape writing input, with iOS beating it and Windows 8.1 coming in a very close third. I know that some people cannot get on with shape writing and if you are one of those then SwiftKey is probably your best bet on Android.#

SlideIT

I will be brief about SlideIT to save you sometime, the predictions are not a good as the other options, it does seem to be relatively low on resources, but overall the other options including the stock keyboard are superior.

Kii

Kii tries to be a combination of SwiftKey and Swype, the options available are staggering it definitely has the best options for personalisation. I have found that its predictions are good, but not the best and so I am not placing it at the top, however, there are some relatively unique features such as dual language input to allow seemless corrections for bi-lingual users (no dictionary switching required) and the ability to customise the size and placement of the keyboard. Kii is certainly worth a try and you may find that the personalisation options make it the best option for you.

Swype

Swype continues to offer the best predictions and best corrections. By comparison I rarely find the need to go back and make corrections. It is nearly perfect, but there are a couple of let downs. I always turn off Dragon Speak as I find it is very bad at managing to identify anything I say, this means there is no quite way of activating Google voice for dictation. This is a minor inconvience for me, but I could see others finding it a deal breaker. I would also love to be able to customise symbol key placement and perhaps create gestures like Kii offers. It does over some very useful customisation in the form of keyboard size and placement, which can allow you to perform one handed typing on a much larger device. I have found it possible to shape write one handed on a Nexus 7 without issue and with many devices pushing 6" it could be of great benefit to many.

Fundamentally the accuracy and the best autocorrection leads me to choose Swype as the best Android Keyboard, but is it the best keyboard of all?

Windows 8.1

Windows 8 has very good correction algorithms for corrections making it one of the best keyboards for typing, but 8.1 introduced shape writing. The accuracy of the shape writing is similar to that of Swype, leading to rarely requiring post writing corrections. It also has a couple of unique features, you can for example add or remove capitalisation from an already complete word just by placing the cursor in it and pressing shift. Additionally the voice recognition is of a similar quality to that of Google's and with this minor addition I would rate the keyboard as a fraction above Swype's offering, making the Windows 8.1 keyboard the best mobile keyboard. Sadly it is only available on Windows Phone, and the Windows Phone platform while great in many ways is not the equal of Android right now.

iOS 8

One of the features that I am really interested in is iOS 8's opening up of the keyboards. I am really curious how many people will switch away to Swype or Swift key.

Sunday, 27 July 2014

How to improve the speed of your web page, the basics

Is saving a few seconds really important?

Image by FracFx
While practically any change, no matter how sure you are that it is a positive improvement, can be met with anger from the user base performance improvements are not one of them.

In an ideal world all webpages would appear the instant we demanded them, however achieving good performance has costs, and those costs need to be offset against the benefits. Features and complexity sells and performance is just one of those features which can be easily ignored and tough to sell, especially if it is say the difference between 10 seconds and 5 seconds.

The benefits of speed are often underrated and can be difficult to appreciate. For example a fast application is easier to use, not just because of the reduced frustration of waiting, but because it is easier to explore. If you click in the wrong place as the next piece of information appears instantly you can easily explore the options on the page. If every option takes ages not only are you more careful in your decisions taking long to complete any interaction, you also will not explore and perhaps find new better pathways to complete your task.

Another aspect of speed is not only does it allow you to explore, because it does not take long to get back to a known pathway, but it gives you further confidence that the application is well made. Confidence in the application again has unseen benefits such as the user is more likely to assume that the problem they are encountering is of their own making and not the application, reducing support calls as they try to resolve their issues themselves.

Also people can start using more optimised work practices to save themselves time, often what the developer sees as the "correct" pathway can be ignored because the "incorrect" pathway is faster.

Saving time, even seconds can add up to a lot more than the some of its parts.

What do I do next?

Well done you have managed to secure resource to improve performance, but where do you start? There is only one place to start measuring. Always start by monitoring and measuring how long the page takes to display. There is an issue that by logging time taken does actually slow down the page which might be considered an issue in a live environment. However, it is important to remember that the performance bottlenecks spotted in a test environment may not full reflect your live platform.

For the remainder of this article I am going to focus purely on the HTML, CSS and JavaScript and the page rendering performance. To truly improve performance you need to consider all factors and in fact some of the best gains can be achieved through a more holistic approach. For example a great performance improvement pattern is to anticipate the next request and cache this pathway. This works extremely well with a string find UI, before the user clicks "Find Next" the system has already calculated where the next string match is and jumps to it immediately. Simple approaches like this can get around the requirements for massive amounts of power or super efficient algorithms. Of course most of my best gains have been working on the "back end" processes, but there is plenty that can be done at the "front end" too.

Minimise

It is not exactly rocket surgery to realise that reducing the amount of data sent to the browser will improve performance. This is often the simplest change to make and most of the time gives you the biggest boost in performance.

TCP/IP connections initially send a packet of 14kb followed by the rest of the data. If you are accessing the page via cellular data or perhaps through a satellite connection then latency of the connection becomes an issue. Latency can be in the hundreds of milliseconds so if you are aiming for great responsiveness then the initial 14kb becomes important. If at all possible it would be optimal to provide all of the data necessary to display the visible portion of the page in the first 14kb. To achieve this it is vital to minimise everything that is being sent

Lets take a look at a corporate website example. City Link is a large UK logistics company. On a decent broadband connection their website pops up pretty quickly, but, it could be faster.

Images

The home page comes in at a total of 567 kb. 256 kb of the total are the images. Most of these are compressed appropriately, it is possible to compress these without any loss to image quality and achieve a reduction of around 153 kb. Now of course you could argue that some of the images could be replaced with HTML and CSS for a much larger reduction in size, if there is opportunity for this then it would obviously be a good place to investigate next.

HTML

The HTML for the page is 29 kb, by just removing carriage returns and unnecessary space this page drops to 22kb. Now the results are not quite so impressive if you consider that this will be sent over the line compressed, but we are still able to see a drop of 7kb to 6kb.

CSS

Again none of the CSS files are compressed, while the built in server compression again helps hide the benefits of compression it is still possible to save 3kb off the 30.6 kb total without the need to try and combine CSS rules, or cull any unnecessary rules.

JavaScript

The page loads 17 JavaScript resources totalling 245 kb of data. Again a lot of this is not compressed and it is easy to save 8kb of data just by "minifying" the JS.

Requests

The page makes a total of 52 request to get all of the required files. Each request has quite an overhead and has quite an impact on how quickly the data gets to the user as well as the amount of traffic required to pass this data in. It is possible to combine a number of the images and use image slicing to produce the images. As the images are almost entirely green, yellow and white as the pallet is not diverse combining the images will compress well and can substantially reduce the overall image requirements on the page.

Combining the CSS will have a similar effect. There are 4 css files, although one is served from Google the other 3 can be combined quite easily to reduce the number of requests.

Load the JavaScript as late as possible

JavaScript blocks the rendering of the page, moving the JavaScript from the header to the end of the body can often get the visuals up and running before the event control in the JavaScript is required.

Do not load what you do not need

I might be missing something in the code, but I cannot see where the Jquery.mousewheel.min.js is used. If this is indeed not in use then removing this code gets rid of an unnecessary http request, 1 kb of data and also stops the browser delaying render while it parses this file. Try to load JavaScript or any other resource as conditionally as possible, yes your site will Cache resources so the overall effects might not be too bad, but getting rid of an unnecessary parsing will improve the responsiveness even just a little.

Fix any broken HTML

When the HTML is in error the browser triggers its error handling, this is only a fraction slower, but may trigger various quirks modes, this probably leads to extra CSS and HTML to "fix" the problem. Running a validator such as HTML Tidy is invaluable in preventing this path. City Link only triggers 7 warnings, this is far fewer than the majority of pages I have looked at, but these 7 could quickly be addressed for a perfect score.

Sadly I have used technologies which prevent you from getting a perfect score for example they force you to use lang or type='javascript' when you are working in HTML5 because the source will not compile otherwise. fixing broken HTML is not always possible, but when it is try to use HTML and rid your page of those warnings.

Do not use XHTML

Some websites use XHTML such as City Link. Besides the fact that support for this version of HTML within browser is poor and they tend to just parse as if it were HTML, it is not compatible the with XHTML 2 draft and perhaps more importantly it requires more verbose HTML than HTML 5. Save precious bandwidth and choose HTML 5.

Friday, 25 July 2014

The big constraint in Development and how it can bring down a team

I always consider the primary constraint in software development to be time. While technically this constraint could be considered a function of money, it is difficult for a developer to assert direct influence over budget. Most organisations require developers to go quite high up the chain of influence in order to try affect the proportion of budget development have access to. Supply and demand factors for the product that development are supplying is difficult for them to directly influence, in most corporate structures sales and marketing are the real influencers of this factor. Yes they rely on the quality of product that development provide, but it is unlikely that and improvement in the product quality will have an equivalent impact on sales. The ability of sales and marketing to reach new customers and alter demand is a much more influential factor.

It is one of the great features of software development that you can essentially do anything the customer asks for, but only if you have enough time. Developers realise how precious this commodity is. There will always be bugs in the product that can be fixed, there will always be ways you can enhance the existing features and there will always be brand new features which can benefit the customer. As your team is not infinite in size and your system resources are finite you are always unable to write super efficient bug free code covering every desired feature.

All the time a developer is working on a given section of work they will be performing cost benefit decisions, how much time do they spend optimising their changes for performance, how much time do they spend reviewing/testing it for bugs or coding defensively against them. They weigh these factors up against what they perceive as the risks of the code and against the relative importance of the change.

Software developers have normally entered the field due to a natural aptitude for logic. Logical reasoning will commonly be a very important to how they perceive the world. This is critical in understanding how best to manage a team of developers. A manager must be able to provide logical reasoning for the decisions he makes and more importantly communicate these reasons effectively to their team.

The development team will often know what processes appear to be impeding their time, they will be aware of many of the issues the product they are working on, and have a good idea of how long each will take to fix. They will probably even be able to provide you with an ordered list based on their cost benefit analysis of what should be done. Now if that list tallies with the managers scheduling plans then the team will be functioning smoothly, the developers will naturally respect the managers decisions. However, if that list does not tally then there is potential for a break down within the team dynamic.

The manager will need to explain their priority order clearly and provide rational explanations for the differences in opinion. I have not yet been in a team where the developer has not altered their opinion in the face of rational argument. It is vital that the manager should be open to change in the same way. If the manager cannot provide a consistent rational argument to influence a change in opinion of the developer they should strongly consider if their ideas are the best way forward. If the decision making is obviously particularly contentious then it is important for the manager to review their decisions at the end of the process and if they were wrong admit to this, if they were right they should provide this evidence to the team. Everyone has some level of ego so obviously this should not be presented in a gloating manner, however, as I have stated developers tend to work from a logical stand point, if the manager adequately proves their decision making process was effective then the developer will be able to maintain their respect and the team should continue to function well.

The more the developers see the decisions of their manager as irrational the more respect they will lose, the lose of respect will lead to reduced communication and a drop in the effectiveness of the team.

Developers normally want to release the best products possible and realise that the product quality and breadth is influenced by the time available to code any seemingly irrational scheduling or processes eat into this time and are seen by the developer as barriers thrown up to stop them producing a great product. I have not spoken to a developer yet who does not agree with the agile manifesto and I believe it is clear that this document was borne out of experiencing frustratingly irrational management processes that the developers saw as restricting the precious resource of time.

Sunday, 20 July 2014

Intersystems Cache - Write Performance

In previous blog posts I have mentioned that Cache has good write performance, in fact in certain circumstances it is faster than many other databases. However, it is rarely the case that optimising performance is not desirable. Every one loves a performance boost, it is the most universally loved product change. The closest I have ever experienced to a complaint about a performance boost is "its so fast I didn't believe it had worked".

Any way I was working on improving query performance on a server when I noticed some intermittent slow down. I traced the performance issue to a task which performs a huge burst of write operations. The write operations were so heavy that the rest of the server was experiencing slow down.

Obviously the code is under review to see if the amount of data being written is necessary, but I was curious about optimising the write operation itself. With read operations I have noticed that dynamic SQL is slower than compiled SQL which is in turn slower than direct global access, in certain circumstances this performance difference can be 10-100 times difference. I wanted to determine if this is the case with write operations as well.

The Test

I thought a simple test should be sufficient, just write to the database 10,000 times 3 small strings. 

Test 1 Dynamic SQL

For better code reuse and readability it is sometimes superior to use dynamic sql, however, there is often quite a performance penalty. I assumed this would be the slowest method and it did not disappoint. My laptop managed to complete the operation in 2.173 seconds.

Test 2 Compiled SQL

If possible use compiled SQL over the dynamic variety, not only is the syntax highlighting of benefit, the performance gain is substantial. The test ran through in 0.238 seconds, nearly a 10 times performance improvement and the code is just as readable. The only downside of compiled SQL is that occasionally in a production environment I have found that queries can stop performing correctly and you need to clear the cached queries, this is relatively uncommon though.

Test 3 Direct Global Write

Unsurprisingly writing globals was dramatically faster at 0.042 seconds. Writing directly to globals has many issues in terms of code re-use and clarity, additionally it requires a lot of extra code around it to make sure the data matches the requirements for the fields, and its not exactly ACID compliant without substantial work. That being said if the performance is insufficient for the task and hardware upgrade is out of the question it can become necessary to use global access. 

Conclusion

Avoid writing dynamic SQL if you can, compiled SQL will really boost the performance and without any real cost. If you have to move lots of data about really quickly and are happy to deal with the limitations then globals can really help get that application performing well.