blog

Mobile Development: Native vs. Web

Tue, 31 Jan 2012 10:26:09 +1300

Having recently completed some research for a whitepaper on mobile development, in particular looking at options for a code once, deploy to many framework, I thought it might be timely to put down some on the insights that this research has lead to, in particular the growing debate around "native" mobile apps vs. HTML5 based web apps.

First lets set the scene...

With the smartphone market now firmly established as a genuine application platform, and the lack of single dominant platform, mobile app developers are being faced with the challenge of developing and maintaining apps across multiple platforms. Rather than up skilling in each individual platform for each mobile environment, many developers are turning to cross platform development frameworks, hoping these will assist with the utopian goal of code once, deploy to many.

As soon as you start looking at the various cross platform development frameworks currently available you begin to see these are generally based on one of two approaches:

A framework that allows you to code in a single language (e.g. Java, JavaScript, C++, C# etc) and then provides a blackbox that produces a native app based on the specific SDK for each supported mobile platform (e.g. iOS, Android etc).

A framework based on web technologies (primarily HTML5, CSS, and JavaScript) allowing you to build an app using standard web technologies with the framework providing access into the phone hardware (camera, location services, multimedia etc) as well as providing a wrapper so the web app can be accessed and used in the same manner as a normal "native" app (rather than having to open a browser and navigate to a website).

The key point here is that both of these approaches provide a developer with a means of creating an application without the need to re-code the application separately for every platform they wish to support.

When you start looking at the various options (and there are a multitude of genuine framework offering now available - the scene is ripe for a period of consolidation and buyouts, a topic for a whole separate post maybe) the first decision you need to make is which of these two approaches you wish to use.

This decision will inevitably draw you into the debate as to which is best, "native" apps, or web apps.

The debate...

Web based apps have traditionally been the poor cousin on both mainstream PC platforms as well as in the mobile space. There have been many valid reasons for this, some of the key ones being:

Web apps are generally accessed by first opening you web browser, and then entering the necessary URL to get to the website where the web app is hosted.

The user interface capabilities available using web technologies has traditionally lagged well behind the capabilities of native desktop or mobile applications.

Web apps have traditionally not been available in an offline version, rather requiring the user to be online in order to access and use the web application.

Because web apps are rendered on the device through the browser, they are generally slower on the performance and responsiveness front.

With the rich and polished user experience Apple introduced with the iPhone, the second point above is most often pointed to as the key reason for developing a native code application, whether you are coding directly for that platform (e.g. Objective C on iOS), or whether you're looking at cross platforms development options enabling you to code once and deploy a "native" app to multiple mobile platforms.

However with the growing adoption of HTML5, and the increasing sophistication available in Javascript libraries, the gap between native apps and web apps has been significantly closed. This includes utilising HTML5 for offline storage in web apps, and the introduction of wrappers allowing the web app to be presented to the as a standalone app without the need to access via a web browser.

For the user experience, the use of HTML5, advanced JavaScript libraries such as Sencha Touch, and frameworks such as PhoneGap, all combine to provide a web app that looks and behaves very close to what can be achieved by native apps (i.e. developing in the native code for a given platform). Developers may be able to pick the differences, but they are subtle enough that an average user would not notice.

From a performance perspective, web apps still need to be rendered on the device, but the ever increasing processing power now available on smart phones has largely mitigated any performance concerns for all but the most complex mobile apps.

When taken from the context of wanting to make an app available on multiple platforms (e.g. iOS and Android), a web app has many advantages from an architectural perspective.

Web technologies are inherently platform independent, they were designed from the very start as a stack of technologies able to be used in a consistent fashion across multiple platforms.

Most cross platform development frameworks are designed to operate within a specific domain (e.g. mobile platforms, desktop OS's etc), however web technologies are common across all platforms allowing you to present your web app on smartphones, tablets, PC's etc.

The growing acceptance of web technologies is permeating right through to the likes of Microsoft, who are incorporating web technologies such as HTML5 and JavaScript as core technologies for developing apps for the upcoming Windows 8 platform. Adobe has also taken a strong position on web technologies with their recent purchase of PhoneGap, which is likely to be used to fill the gap left by their discontinuation of Flash for mobile.

When all this is combined, we find that most of the commonly thrown up reasons for developing "native" rather than web apps are either non issues, or fast heading in that direction. Native apps will always provide some advantages due to the fact they are inherently "closer to the metal", but web technologies such as HTML5 and new JavaScript libraries will start to swing the pendulum in favour of looking at web apps as the first choice option.

First lets set the scene...

As soon as you start looking at the various cross platform development frameworks currently available you begin to see these are generally based on one of two approaches:

A framework that allows you to code in a single language (e.g. Java, JavaScript, C++, C# etc) and then provides a blackbox that produces a native app based on the specific SDK for each supported mobile platform (e.g. iOS, Android etc).
A framework based on web technologies (primarily HTML5, CSS, and JavaScript) allowing you to build an app using standard web technologies with the framework providing access into the phone hardware (camera, location services, multimedia etc) as well as providing a wrapper so the web app can be accessed and used in the same manner as a normal "native" app (rather than having to open a browser and navigate to a website).

When you start looking at the various options (and there are a multitude of genuine framework offerings now available - the scene is ripe for a period of consolidation and buyouts, a topic for a whole separate post maybe) the first decision you need to make is which of these two approaches you wish to use.

This decision will inevitably draw you into the debate as to which is best, "native" apps, or web apps.

The debate...

Web based apps have traditionally been the poor cousin on both mainstream PC platforms as well as in the mobile space. There have been many valid reasons for this, some of the key ones being:

Web apps are generally accessed by first opening you web browser, and then entering the necessary URL to get to the website where the web app is hosted.
The user interface capabilities available using web technologies has traditionally lagged well behind the capabilities of native desktop or mobile applications.
Web apps have traditionally not been available in an offline version, requiring the user to be online in order to access and use the web application.
Because web apps are rendered on the device through the browser, they are generally slower on the performance and responsiveness front.

When taken from the context of wanting to make an app available on multiple platforms (e.g. iOS and Android), a web app has many advantages from an architectural perspective:

Web technologies are inherently platform independent, they were designed from the very start as a stack of technologies able to be used in a consistent fashion across multiple platforms.
Most cross platform development frameworks are designed to operate within a specific domain (e.g. mobile platforms, desktop OS's etc), however web technologies are common across all platforms allowing you to present your web app on smartphones, tablets, PC's etc.
The growing acceptance of web technologies is permeating right through to the likes of Microsoft, who are incorporating web technologies such as HTML5 and JavaScript as core technologies for developing apps for the upcoming Windows 8 platform. Adobe has also taken a strong position on web technologies with their recent purchase of PhoneGap, which is likely to be used to fill the gap left by their discontinuation of Flash for mobile.

User Stories - How to get an earlier and better handle of your customers business problem

Tue, 17 Jan 2012 16:52:48 +1300

Documenting the real business problem that a project team is tasked to solve is easy when you have one local Subject Matter Expert (SME) or business owner to talk to. A discussion in front of a whiteboard will generally enable you to get a good idea of the core problem that needs to be solved. You can then use whatever style of documentation to record it as you like.

This process gets very difficult under any of the following circumstances:

When there is more than one SME to talk to.
When the team are geographically separated.
When there is significant organization prior art that influences the space this problem resides in.

When you add more than one of these into the mix it becomes almost impossible!

One way to assist these separated or disparate groups come to a common understanding of the business problem at hand is to write User Stories in a workshop setting.

While their names are similar sounding, User Stories are not Use Cases. They differ in that Use Cases are typically highly structured and better suited for later in the requirements gathering process. Use Cases often involve a step-by-step flow through a particular usage which can easily start to describe the how rather than the why or what the problem is. A Use Case doesn't encourage discussion about a problem either… partly because they are so structured and partly because they are often either very complex (due to numerous alternate flows) or are very, very specific (an attempt to avoid the complexity of numerous alternate flows!).

This isn't to say that Use Cases are not of use in some circumstances! They can be very useful when you know exactly what the business problem is and wish to document the accepted solution at a low level.

Use Cases generally end up being very large and detailed. This makes estimating them or breaking them into manageable pieces more difficult. They also fail when the solution is not clear and conversation is required between multiple people to find out what the problem is from all angles.

User Stories are intended to state the goals (or the problem they wish to solve) for the users of the system in the language of the business. There should be one User Story per goal and as the User Stories are revised and broken down, they should become small enough and refined enough to be built in a single iteration.

So the idea of User Stories is to get a handle on the business goals and problems very early in the project in such a way that they're solution agnostic, estimate-able and understandable to everyone… especially the business.

User Stories should:

Describe the goal that the customer wants to fulfill.
Be short and represent small chunks of business value that can be implemented inside an iteration.
Avoid all technical detail.
Be easy and quick to read.
Not be hugely specific, thereby allowing lateral thinking while reading them.

A good way to remember the key qualities of a User Story is through the use of the INVEST acronym:

Independent

Negotiable

Valuable

Estimable

Small

Testable

Independent - User Stories work best when they don't overlap and therefore can be implemented in any order or not at all.

Negotiable - User Stories are intended to capture the essence, not be a contract. Hence they are negotiable.

Valuable - A User Story must be valuable to the customer. If it doesn't create value, why build it? This is related to the Extreme Programming idea of slicing a problem so that every piece of work adds value as opposed to building layers (e.g. a persistence layer) that is feature complete but is unusable because a portion of it isn't required yet.

Estimable - A User Story must be estimable. If the team can't estimate the effort required, the story is too vague or big and needs further refinement.

Small - A User Story must be small. If it looks like it'll be a lot of work, it may indicate a lack of understanding about the story which should be addressed.

Testable - A User Story must be testable. If the customer can't help define how to test it, then perhaps the story isn't clear enough.

A User Story actually consists of three parts:

The Story - This is the written description of the story which describes the goal.
Conversations - Conversations about the story that flesh out the details and get everyone on the same page about the story.
Acceptance Tests - These document details that can be used to determine when the goal of a story has been fulfilled.

The most common form to write a User Story card in, as popularized by Mike Cohn, is:

As a <type of user>

I want <some goal>

so that <some reason>

e.g.

As a cab driver

I want to be able to quickly find my way to the 10 most frequent destinations

so that organization knowledge on the quickest way to a destination is shared among drivers thereby saving myself and my customer time and money.

This story succinctly documents who needs what and why. In this example, the cab drivers' business problem is that he needs to know how to get to the top 10 destinations quickly and efficiently for his customers. The organization has knowledge about this problem which can to be shared amongst the drivers.

You'll also notice that the story avoids all mention of existing processes, methods or technology. In fact, it is devoid of all significant detail! This user story is a great way to start a conversation between the SME's on a project; since nothing is discounted off hand, everyone can offer their own perspective and bring their own understanding of the problem domain to the conversation where they can be openly debated.

A conversation also avoids one of the limitations of the written language i.e. imprecision. Because there is a conversation, people can ask for clarification on points at the time to get the idea across quickly.

This is the reason for a User Story workshop. It is a workshop to get all the SME's and key representatives of the project team together to define, discuss and document the User Stories. It is strongly recommended that these workshops be the first, second and third ways to gather user stories. Writing them in isolation after individual conversations is nowhere near as effective as it neglects the group conversation aspect.

Ideally you need the following participants in your workshop:

The business customer(s)
The product owner
As many SME's on the topic as practical
The project team test lead
The project team technical lead
The project manager
Any key organization members with knowledge of existing prior art

With all these people involved in writing, discussing and refining the user stories, you get excellent knowledge coverage across the team and a clear common understanding of the piece of work.

Going back to our example, our SME's have come up with the following options:

A paper map
A gps unit
More extensive driver are training.
A paper list of the 10 most common locations and quick directions to each

Each has their benefits and problems… so how do we choose one and how do we prove we've found a solution that is fit-for-purpose under real world conditions?

This is where the Acceptance Tests come in. This list of tests will often be scenarios in the real world that affect the user in reaching their goal. They normally line up with what would have been the alternate flows in a Use Case.

Going back to our cab drivers goal, some tests might be:

Test when it is dark outside.
Test when the driver is en-route but needs the next step or a reminder.
Test when the passenger only knows the name of a place and not the address or general location.
Test when the driver is not familiar with where they are starting from.
Test when a local event is taking place this week that is very popular (e.g. the World Cup).

With the goal, the conversations that generated some options and the tests, selecting and then estimating the solution should be much easier.

In conclusion, User Stories are a great way to capture the business goal or problem and encourage discussion about the topic. This conversation leads to a shared understanding and a more complete picture of the problem and the acceptance criteria that will prove success.

CRM 2011 - Visualise Business Data Using Charts

Fri, 11 Nov 2011 09:21:23 +1300

In the first "where's the business value" article we looked at how the breadth and depth of data visualisations within CRM 2011 provide a powerful tool for gaining business advantage by providing the ability to bubble to the surface critical business information, or alternatively allow users to spot trends and drill down for more information.

One of the visualisation options mentioned was the CRM 2011 charting facilities, which are powerful enough to warrant a dedicated post… so here it is.

One of the most powerful facilities in CRM 2011 is the charting functionality. For me what makes it so powerful, is that it is also very easy to use, customise and extend (for non-technical end users), making it very accessible to the end user. So many systems provide a lot of advanced functionality, but unless you combine that with ease of use to your end users, you will always have a more limited uptake and reduced business value.

CRM allows for charts to be viewed inline (on the same screen) as any data view/list, the chart itself being built in real time based on the data being displayed in the current view. This means that by using the filtering facilities in CRM 2011 (think Excel column filters), the chart also updates based on the filtered view data.

This means the user can easily work with the business data, and use the charts to quickly provide visualisation of the data and spot business trends. To make this even more powerful, users can click on any segment of data in the chart and drill down in more detail on that. For example you have a chart showing the sales pipeline broken by opportunity stage, you can click on any stage, and within seconds produce a new chart of the selected stage broken down by opportunity owner.

In addition to providing powerful data visualisation control to end users, the CRM 2011 charting facilities can also be extended further by power users using the chart export facilities. Any chart can be exported, which produces an XML file containing the chart definition information, including the FetchXML used to define the data query. Any of the data in the XML definition file can be customised, allowing a power user to further extend the charting facilities beyond what is provided by the built in charting wizards.

A couple of examples of how you can edit the definition file to further extend charts include:

The ability to scale break a chart. This is useful where you have extended the chart to display multiple series, and the series may differ in scale. By adding a scale break you can ensure the chart still displays both series in a readable format.
Adding multiple series to a chart. This is useful where you might have something like a revenue by customer chart, which could be extended to display both estimated revenue and actual revenue.

There are many other extensions that can be made to charts by editing the XML Chart definitions, and the charting visualisations can be viewed in any of the CRM clients (online, on-premises, Outlook CRM Plug-in). By providing a combination of easy to use (and create), plus powerful real time charting with drill down facilities, CRM 2011 allows end users to gain that big picture visualisation of critical business information and trends.

CRM 2011 – Where’s the Business Value (part 1)

Thu, 08 Sep 2011 17:22:46 +1200

Use CRM 2011's built in custom charting to quickly and easily visualise data the way you want.

In this series of posts we'll look at some of the different areas within CRM 2011 where genuine business value can be derived. There's currently a lot of information out there on all the features and functions of CRM 2011, but I'd like to focus on the business value that these features can provide.

There are a lot of facilities within the new Microsoft CRM 2011 platform that can add value to your business, whether it be better visibility of what is happening within the organisation (Dashboards, Reports and Charting), cost savings due to greater control and automation of business processes (Workflows and Dialogues), or the consolidation of various disparate Line of Business solutions onto a single cohesive platform (building an xRM system).

That's a lot of value to cover, and information visibility is what is covered in this post, but watch this space for additional posts which will cover some of the other areas.

Firstly, one of the nice aspects of the latest Microsoft CRM offering is that CRM Online now offers an identical experience and benefits to the on-premises version. So you're not sacrificing anything by going to the cloud based version. And taking that further, all the options for viewing data are also available from within the Outlook CRM plug-in module, allowing users who live and breath Outlook to view all lists, data, forms, charts and dashboards directly from within their Outlook client.

This flexibility means businesses have options as to how they deploy CRM to their end users, and can pick a CRM client based on how the user works, rather than forcing users to use a specific tool to access the information. This in turn helps with user adoption by ensuring a better fit to user needs.

Most CRM solutions offer some degree of visualisation or reporting facilities, and will include various pre-canned charts, reports and dashboards. Inevitably these will not match your actual day to day business needs and may be lacking in providing users, team managers, and senior management with the critical information they need for making day-to-day, and strategic, business decisions. Traditionally you've needed to pay for an external CRM expert to come in and build the various custom data views and reports the business needs (or hire someone to work full time on this).

With Microsoft's CRM 2011 users are able to view the same data in multiple ways, as well as being able to easily customise each of the viewing options, which means the business has better access to the critical data necessary for making the correct business decisions.

Some of the various options for data access and visualisation include:

Views - Much more than just a list of search results, views can provide customised lists of data based on business criteria defined by the user, who can also control what data (i.e. columns) is shown based on the query logic. For example a view can be created (in seconds) to display all opportunities due to close in the current fiscal period with a status of "qualified".

Charts - the power of CRM 2011 charts is twofold, firstly they can operate on any view (system or user created), so the above example view can be visually enhanced by looking at it via a view. Secondly the user has the ability to drill down into the view (real time) to isolate any section of data and chart it in more detail based on different criteria. This makes charts a powerful and flexible tool for quickly spotting various business trends and drilling down into a trend for more information.

Dashboards - allow users to view from a single screen a collection of different visualisations including charts, views, and custom web resources. Administrators can create predefined dashboards, and users can also create and share their own personal dashboards.

Reports - the standard selection of pre-canned reports, however the report creation tools are extensive with options to build your own report using the built in CRM Report Wizard, or generate more complex reports using the full power of SQL Server Reporting Services.

View Filtering - any view can be further filtered on the fly by using the filter capabilities in CRM 2011. This works in exactly the same manner as filtering in a MS Excel spreadsheet, allowing the user to quickly and easily drilldown to the specific information they require.

Excel Exports - where users are more comfortable working with MS Excel when slicing and dicing large sets of data, CRM 2011 makes it very easy to export any view/list into excel and then work with the data within excel. As you would expect from Microsoft, the integration between the products goes further by also allowing Excel to be linked to CRM 2011, so the user can update the exported data (directly from CRM) by a CRM data export toolbar added directly into Excel.

Conclusion

The breadth of options CRM provides users with for viewing and visualising data means that critical business information is much more readily accessible, and at users fingertips, which in turn means users (whether a sales person orsenior manager) are able to make much more informed decisions for the business.

Having recently completed a CRM migration from a competing platform, one of the things that repeatedly struck me with CRM 2011, was how much more easily important information was visible, without having to make a concerted effort to dig it up. A good part of this is simply due to the range and ease of use of the data visualisation options described above, particularly when you combine various options such as the chart and filtering facilities.

At the end of the day, quicker access to critical business information means more informed workers, who are spending less time searching for the information they need, and spending more time acting on the information they have received.

Agile for integration projects?

Mon, 15 Aug 2011 09:30:24 +1200

In recent years the agile approach to software development has become increasingly used in the IT industry, and as experience has been gained and the approach has matured it is now often the preferred approach for software development projects. Of course with success in one area (software development), there can be a desire to replicate that success, using that approach, in other types of projects - for example projects that are predominantly software integration oriented. This leads to some interesting questions - is the agile approach appropriate for other types of projects such as integation projects? Are there good examples of success out there?

To elaborate, a core part of the agile approach is a core set of disciplines and working practices that help make agile software development successful including:

Co-location of developers
Continuous testing
Daily stand up meetings
Short delivery iterations

These practices work well for straight software development projects, especially those that are developing a product where a single party is responsible for the development. However, projects that have a greater degree of integration involved can run into challenges if the same disciplines aim to be applied:

Integration projects typically require multiple vendors. If those vendors need to make changes to their products, they are typically doing this where their software development takes place, this is invariably not the same location as the integration project. Of course, not all integration projects require vendors to change products but for large projects this is often the case.
Because there are often multiple vendors involved, ownership of a particular area of a solution can be a problem. This typically becomes a problem when a given integration point is not working as expected and can end up with vendors pointing at each other and saying "it's not our problem".
For a given solution, one vendor may be heavily involved and another may only have a small part. This typically means that some vendors will be involved in the project on a continuous basis while others will drop in and out depending on their part in the project. This can make it difficult to coordinate stand up meetings and team related activities.
Testing is typically a lot more challenging because it doesn't just involve deploying a software release into an environment, it requires multiple vendors to coordinate their releases.

Looking at these challenges more carefully, communication is a key aspect - something that traditional agile addresses by forming a strong team and keeping it together in the same location. Optimation's experience with integration oriented projects, and in particular our clients preferences about how those projects are run, is that they tend to fall back into a waterfall process.

With instant messaging, social networking & collaboration tools, perhaps integration projects can be run in a (more) agile manner? Feedback from anyone out there with experience & ideas in this area would be welcome.

What makes a great tech team?

Tue, 09 Aug 2011 11:06:30 +1200

There’s been an interesting debate raging over at the Harvard Business Review lately on what makes a great team (and I don’t use the word ‘raging’ lightly - check out some of the comments!). In short, the argument on one side says a team of good performers is going to achieve better over the long term than one or two stars. On the other side, from people in the tech industry in particular, the counter-argument is that one great performer - a star, if you will - is going to be many times more productive than a whole team of ‘pretty good’ people. The counter-arguers point to studies of programmers, in particular, that show ‘great’ programmers can be five-to-10 times more productive than ‘good’ or even ‘very good’ programmers. As Otago University head of Computer Science Brendan McCane pointed out in this week’s Computerworld, this sort of differential is even found amongst third year students at Yale University, where presumably just about everyone is already well above average before even getting out of the starting blocks.

Now, I’m not a technical person (though I work with many and I also live with a software developer), but it seems to me there are a couple of fishhooks in this whole productivity argument. The most obvious being, how is productivity measured? Is it simply the time taken to write functional code? The efficiency and elegance with which the code produced solves the given problem? Or even how well the software written fits the expectations and needs of its eventual end users? Human beings being what we are, on this last point in particular the programmer may think they have produced the best thing since sliced bread but the customer may still be disappointed in the outcome. It’s here, I think, where the false dichotomy of comparing individual brilliance to a great team becomes most glaring. Because in my observation, in the real world programmers very rarely work alone. Here at Optimation, the typical team on a client development project will include one or more developers, but it will also include a range of other skills, such as a solutions architect, business analyst, testers, and a project manager. The client - usually someone from the business side rather than the technical side - is also directly involved at various points (often quite deeply in the case of projects done using agile methodologies). Depending on the project, a variety of other skills, from data analysts to web designers, may also be pulled in at various stages.

For this team as a whole to be productive – that is, to deliver the outcome the client wants, on time and within budget - many more ingredients need to go into the mix than the pure programming productivity of a single ‘star’. (Reminds me of that old Crusaders ad, ‘A champion team will always beat a team of champions’. Something we Hurricanes fans know only too well.) Given the diversity of skills and personality types teams typically need to effectively deliver results, the stereotypical ‘star’ programmer who is utterly convinced of their own unchallengeable superiority [disclaimer: I’ve never met anyone with this level of arrogance in real life but there do seem to be a few chiming in on the HBR comments section] may in fact derail the team’s overall performance.

As I see it, as with any team, there’s a certain alchemy involved in building great tech teams. You need both the creative types and the number-crunching analytical types. You need someone with enough empathy to get inside the client’s head and figure out what they really want, and someone who can help them come to terms with the sometimes painful recognition of what’s necessary versus what’s a ‘nice to have’. When things go pear-shaped and everyone has to work late, you need that person who’s going to bring the music, the jokes and an endless supply of Red Bull. Occasionally, you need the misfit, the person whose first question isn’t ‘how?’ but ‘why?’, and who can radically redesign a business process by questioning its very existence.

From this perspective, are great technical teams really that much different from other great teams? And what does it take to create one?

A Kiwi take on Microsoft WPC

Mon, 25 Jul 2011 11:58:03 +1200

Neil and Sharon are on their way back from the US, where they attended Microsoft’s Worldwide Partner Conference on behalf of Optimation. At huge global events like this, it can be tough to cut through the clutter and find the topics that are most relevant to us in NZ (plus, it’s easy to get sidetracked by all the cool gadgets and toys!). To make it easier, here’s a brief take on what really stood out.

In between trips to LA's famous amusement parks, Sharon and Neil attended a wide range of sessions covering:

The productivity suite including Office 365
SharePoint
SQLServer Denali
HR - The 2020 Workplace and Developing People
Dynamics CRM
Private Cloud
Lync

The biggest theme of the conference was the cloud – public, private and hybrid. The cloud is at the core of Microsoft’s strategy with offerings including Office 365 and Hyper-V Cloud, and there were some tantalising hints dropped around Windows Server 8 and Microsoft’s Private Cloud plans.

COO Kevin Turner outlined Microsoft’s five big bets for FY12 in a punchy speech that took aim squarely at competitors like Google, which the company is targeting with its new Office 365 offering. Microsoft also promised a vision of ‘Windows everywhere’, unifying ecosystems across a broad range of platforms and devices.

Another theme that came through this conference is the consumerisation of IT. The main message is keep up or you will be left behind in this area. With the proliferation of devices available now, “bring your own device to the workplace” is a reality and infrastructure needs to keep up. Microsoft’s response to the consumerisation of IT is through these four pillars:

Reimagining of Windows across all platforms/devices
Secure and manage your infrastructure via Intune and System Center 2012, both now available in beta
Best Productivity Experience on Microsoft devices and others
Unified Application Development across Azure and Office marketplaces

Neil and Sharon will no doubt return with a whole swag of information and copies of the various presentations, so if you want any more detail on things Microsoft, please ask and we’ll see what we can do!

News from Microsoft's world partner conference

Tue, 12 Jul 2011 12:27:47 +1200

While those of us stuck in New Zealand shiver through the first big winter storms, Optimation's own Neil and Sharon Butler are off in sunnier climes, enjoying the Microsoft Worldwide Partner Conference. This is a massive event, with around 12,000 attendees and a dizzying array of sessions to choose from. Given our roots and our focus on the software development life cycle, we Optimates are particularly interested in the latest developments related to the core MS application platform, including things like SQL Server's BI capabilities, BizTalk for integration, and Visual Studio.

On the solution side, we provide consultancy around SharePoint Server and Microsoft Dynamics CRM for our clients as well as using these platforms internally, so Neil and Sharon have instructions to bring us back all the intelligence they can on best practice and on the ways other companies are using CRM and SharePoint to improve their businesses. We're also very interested to hear what Microsoft has to say about the progress of its mobile and cloud offerings.

Neil and Sharon will be sharing the highlights of the conference from a Kiwi perspective when they return, but in the meantime here are a few interesting snippets from the first day -

Microsoft CEO Steve Ballmer used his opening keynote address to highlight the company's on-going success with Microsoft Office and Windows 7, but admitted there is still a way to go with Windows Phone 7.

He also focused heavily on integration opportunities, particularly between on-premise and in-the cloud offerings such as the Azure public cloud and Office 365. The latter seems to be off to a good start since its June launch, with 50,000 businesses signing up in the first two weeks and some positive reviews out of the gate.

In other news, Microsoft has now sold 400 million Windows 7 licenses so the drive is on to move all users off XP. The company took the opportunity at the conference to announce that as of July 11, customers have 1000 days until support for Windows XP ends.

Object Role Modelling

Wed, 01 Jun 2011 11:17:35 +1200

In preparation for a consulting assignment I’m about to begin, I’ve been reading up and refreshing myself on the techniques available out there for modelling information. One of those techniques is one I first came across a few years ago called Object Role Modelling and in the intervening time, it seems to have become more widely accepted.

Object Role Modelling (ORM) is a technique for conceptual data modelling and takes a more granular and more expressive way of modelling information than more traditional techniques such as ER modelling or UML Class diagrams. Below is an example of what an ORM diagram looks like:

This particular diagram has been taken from an example on the ORM Foundation website (http://www.ormfoundation.org/Default.aspx) - a very good place to start if you’re looking for tools or information on this technique. On first glance it doesn’t look hugely different to an ER diagram, but there are some significant differences.

The first one is that the concept of an entity has been replaced with the concept of a ‘fact type’. In the diagram above these are the Employee, Agency, Office etc and they are linked to one another via roles. For example in the diagram above, an Employee works at an Agency and reading it the other way, an Agency employs an Employee. Agency & Employee are the fact types and ‘works at/employs’ represents the roles each fact plays in the relationship.

The really important thing that this type of modelling allows you to do is to focus on the concepts and relationships without getting tied up with what is an entity and what is an attribute/column. You begin by focusing on fact types and the role they play in their relationship to one another and because you’re not bound by what is an entity and what isn’t, you can focus on modelling the concepts.

If you do decide that a fact will ultimately be represented as an attribute, you can change it to a dashed line. In the diagram above, ‘Name’ is a good example of this as it is related to an Agency, but it is ultimately a value associated with the Agency fact type.

Another important difference from traditional ER diagrams is that ORM diagrams can be easily turned into structured english sentences, that accurately describe the facts, relationships and the roles being played across the model. This is something that is much more difficult to achieve in ER diagrams, especially when you start adding constraints and rules into the mix. The ability to easily translate the diagram into english means that you have another way of communicating with your audience - especially the business oriented part of your audience.

In fact business rules and the ability to describe these in a structured manner is where ORM really comes into it’s own. Because you are modelling at a more granular level and have a rich set of modelling constructs to choose from that allow you to describe constraints as well as relationships, you can use ORM as the basis for collecting business rules in a structured manner.

If you’re involved in modelling information and data, it would be well worth your while taking a look at the wealth of information that now exists on ORM, Microsoft is well behind this approach now and as a modelling technique it has some very definite advantages over ER or Class diagrams when working at the conceptual stage of modelling.

This particular diagram has been taken from an example on the ORM Foundation website (http://www.ormfoundation.org) - a very good place to start if you’re looking for tools or information on this technique. On first glance it doesn’t look hugely different to an ER diagram, but there are some significant differences.

The XRM trend

Mon, 09 May 2011 10:06:06 +1200

XRM, or eXtended Relationship Management, is one of the many buzzwords currently floating around the CRM space, along with the likes of Social CRM, cloud computing, SaaS and many more.

As with most of these terms, there is genuine business value to be obtained - but first you must cut through the hyperbole and come to grips with what specifically is being referred to; is it relevant to my business? And where do the key benefits lie?

XRM solutions have evolved over the last several years as your "classic CRM" platforms have grown beyond the traditional 3 pillars of CRM - Sales, Marketing and Service. As CRM became more popular and the CRM platforms became more flexible, their use expanded into managing relationships in other areas such as partner relationship management, employee relationship management, and a range of miscellaneous subjects such as events, tasks, fleet, property, asset… the list goes on. As the diversity of solutions has grown, so has entered the phrase extended relationship management (XRM).

XRM solutions are able to take your Line of Business (LoB) applications (often things that have been managed using paper forms, spreadsheets and Access databases) and implement them using a CRM platform as the framework to build the solution on.

I've spent many years working with various "application builders" which all aimed to work in a similar fashion to XRM whereby a platform is used to define data capture forms, processes and reports with the aim of allowing an entire application to be built by end users, or business analysts with minimal coding effort - in theory enabling reduced implementation costs and risks.

It has been interesting coming back into the CRM space recently and seeing how it has matured and grown over the last 5 - 7 years. The CRM based approach to this "application builder" model, XRM, looks pretty solid and certainly provides many benefits over other application building platforms. In addition to providing flexibility to quickly build specialised LoB solutions, the XRM approach allows multiple solutions to all sit within a single platform, providing integration options and the ability to leverage all the facilities of the underlying CRM platform (e.g. dashboards, workflows, activity management).

Microsoft's latest offering, CRM 2011, is certainly no exception, and it looks to provide the full range of expected CRM functionality, as well as a great deal of flexibility for not just extending the solution, but for using it as a platform for building whole new systems upon. While you probably won't want to try building a new accounting system on CRM 2011, it is the ideal the platform to introduce when you have multiple small applications that may have organically grown within your business over time.

Now the benefit of being able to bring these various applications onto a single consistent platform is significant, and this is something that most CRM/XRM platforms can offer. But when looking at Microsoft CRM 2011, what stood out were 3 core benefits that look to be unique to CRM 2011, these being:

Familiar Office style interface
Online / on-premise options
Outlook integration

A common problem with introducing new solutions is ensuring user uptake and use of the system, and trying to minimise training requirements and the time it takes users to get up to speed with the new system. As CRM 2011 uses the familiar Office ribbon style toolbar, any custom XRM solution built on top of CRM 2011 will also use these, meaning all your custom LoB solutions can utilise to standard, common navigation and toolbar style based around the Office ribbon.

Most custom developed solutions, and XRM based solutions will give you one of two options; depending on the developer (or XRM platform) you will either need to deploy and run the solution on your own infrastructure and servers, or you will have a web based solution hosted on someone else's infrastructure, commonly called cloud computing. Something unique to CRM 2011, is that it is offered in both of these flavours so you can choose whichever suits your own business needs.

You can also take it a step further and start with one hosting option, then move to the other if your situation changes. For example, you can use the Online version to start with for building early prototypes and test versions without the need to invest in additional servers, and only pay for a few user licenses for test users. Once the new solution proves itself and is ready for a larger deployment, you can either ramp up the number of online licenses, or package the solution and install it on your own infrastructure.

The CRM 2011 Outlook add-on is probably my favourite as this gives a whole new twist to XRM and custom developed solutions; by giving end users the ability to interact with your custom XRM solution directly from within Outlook (without the need to actually develop a separate custom Outlook module as the CRM / Outlook plugin takes care of this automatically).

All the data, customised views, and forms built for your XRM solution can be accessed directly from within Outlook simply by installing the CRM Outlook client. Without any additional work, you've gained a whole new interface, including off line capabilities (using the standard Outlook offline facilities) and even better at no additional cost as the Outlook plug-in is a free add-on to any CRM 2011 solution.

Many business users spend a large part of their day "living" within Outlook, and being able to view and interact with an XRM solution directly from Outlook makes it that much easier to obtain buy-in and usage of any new system - no more "not another application I have to learn and use" excuses.

There are many options and app builder platforms out there offering similar frameworks for quickly building LOB applications at lower cost and risk than traditional custom development. But the 'out of the box', deep Outlook integration, familiar ribbon style toolbars, and flexible cloud or on-premises hosting options add together to make CRM 2011 something to be considered carefully as an option for any CRM, XRM, or custom Line of Business solution you're looking at implementing.

Oracle & cloud computing

Mon, 18 Apr 2011 17:03:02 +1200

What is Oracles cloud strategy? Having wondered this myself I decided to attend the Oracle Enterprise Cloud Computing Seminar in Wellington.

What grabbed my attention straight away was how Oracle now seems to be taking a firm position on the cloud computing trend, reversing its earlier scepticism around cloud computing and its viability.

Oracle CEO Larry Ellison famously denigrated cloud computing, declaring it a new name for existing technologies and not a fundamental shift, all the while ignoring the success of Amazon Web Services and companies like Google and Salesforce.com in delivering Cloud services. Despite that, it appears that Oracle has decided there is value in adopting what is quickly becoming the dominant IT trend for the next decade and re-inventing itself as a cloud computing player.

Oracles cloud computing strategy is now to offer a breadth of alternatives both for private and public cloud adopters. Oracle’s offering provides enterprises with a high degree of flexibility which is comforting for organisations that have not yet determined their cloud strategy.

Oracle demonstrates its breadth and flexibility by offering ‘Oracle on Demand’ (Software as a Service (Saas)) and Oracle PaaS (Oracle VM, Weblogic, Oracle Database and RAC), provided by the likes of Amazon and Rackspace. Oracle also says that its technology can be either deployed in private or public clouds. An example of this is the Amazon Machine Image (AMI) configured with Oracle.

Where I see Oracle gaining an advantage over its competition is in the private cloud. I believe this is also where Oracle sees they currently have the most to offer. I base this observation on how the seminar was heavily focused on private cloud computing. Platform as a Service (Paas) and Infrastructure as a Service (IaaS) were heavily focused on with little time given to SaaS and the offerings from Oracle in this space.

Oracle's acquisition of Sun has provided Oracle with the capability to deliver Infrastructure as a Service (IaaS) and Platform as a Service (PaaS) installations that are 100% Oracle, and it will deliver its applications and middleware as either "private Software as a Service" inside the enterprise or as services consumed directly from Oracle. This is further evidence of the direction Oracle seem to be taking.

It is in the PaaS and IaaS layers of cloud computing that Oracle now has what I consider a compelling proposition for enterprises wishing to implement a private cloud. Enterprises are able to purchase hardware pre-installed, configured and optimised for Oracle software providing lower costs, greater control over security, compliance and quality of service.

Oracle has used its acquisition of Sun and the IP gained to fill out its cloud computing portfolio, using Sun servers for hardware and making Java a linchpin of programming between infrastructure, Oracle middleware and applications. This now provides Oracle with control of the entire stack enabling Oracle to provide the Exadata and Exologic machines as “Cloud in a Box” that are preconfigured and optimised for the Oracle stack claiming performances increases of 10x.

Oracle ‘Cloud in a Box’

Cloud in a Box certainly makes sense from a cloud IaaS providers or Enterprise IT teams point of view where there are plans to build new dynamic data centers. It does provide a higher level of unit abstraction to build next generation data centers. It takes care of lot of IT dirty work of connecting servers, storage and networking pieces along with back chassis and channels and certainly offers a lot of OpEx savings.

From a SaaS and PaaS point of view internal IT are able to offer resources in similar way to the public cloud and charge back the business units based on usage. This presents a shift in budgetary responsibility as IT budgets can be assigned by business unit, project or overall annual requirements. But cloud computing, as a centralized IT resource charged on a usage-based billing model, blurs traditional budgetary lines, reviving the concept of the IT function charging its costs back to individual departments. As to how charge back can be implemented for a private cloud I will save that for a future blog, but what is clear is that providers can offer their internal services and charge in a similar way to the public cloud and Oracle provide measuring capabilities to enable this.

Google Powermeter continued

Thu, 07 Apr 2011 10:38:34 +1200

I briefly discussed my experience on going greener with Google Powermeter in a previous blog post and will explore the actual solution in a bit more detail.

I use an inexpensive socket power meter to measure usage of individual devices and appliances. They are especially useful to measure power consumption for plugged in devices like TV’s, gaming consoles, etc but can’t be used to monitor overall consumption or for devices that are directly connected to the power system like lights, ovens etc. The most effective method of measuring these is by using a central power meter connected to your mains power supply. There are a variety of commercial non-invasive (you don’t need an electrician for installation) devices available from about $100 upward that are fully compatible with Google Powermeter. I personally enjoy fiddling with anything technology and decided to build my own solution using a few electronic components, my home Ubuntu server and Google Powermeter.

The solution described below is not at all a reflection on how to get started with a commercial off-the-shelf Google powermeter and I hope does not deter readers from investing in such devices themselves

My homemade solution is based on two hardware devices, one for measuring the current at the main connecting board housed outside (1 on the diagram) and a receiver (2) to capture the signal broadcasted from the first. The receiver is connected to a low power Ubuntu server that captures the incoming data every 10 seconds and stores it in a PostgreSQL DB. The Java service responsible for capturing the incoming data also calculates the aggregated data every 10 minutes and uploads it to Google Powermeter.

Google powermeter requires you to register your device before uploading any data. The registration process provides you with a SSL security hash and authentication key that is needed for uploading usage data. The registration process is explained in detail in the Google powermeter API documentation. (http://code.google.com/apis/powermeter/docs/powermeter_device_activation.html)

Data logging is implemented by posting an atom xml feed to a predefined URL that contain both user and device information. The authorisation information obtained from the registration process also needs to be included in the http header of the request. Google currently only allows uploads every 10 minutes and will block more frequent updates. A typical data upload will look as follows:

<?xml version="1.0" encoding="UTF-8"?>

<entry xmlns="http://www.w3.org/2005/Atom"

xmlns:meter="http://schemas.google.com/meter/2008">

<meter:startTime meter:uncertainty="1.0">2011-03-08T02:56:58Z</meter:startTime>

<meter:endTime meter:uncertainty="1.0">2011-03-08T03:06:49Z</meter:endTime>

<meter:quantity meter:uncertainty="0.1" meter:unit="kW h">0.05800</meter:quantity>

</entry>

The hardware I used is based on the Arduino platform connected to two XBee communication modules and an AC current sensor. Arduino is an open source physical computing platform based on a simple microcontroller board and a development environment for writing software. I would highly recommend looking at this platform to any aspiring electronics hobbyist since it’s so easy to start with. See http://www.arduino.cc/en/Guide/Introduction for more information on the platform.

Implementing your own power meter solution is not recommended for most people and probably not worth the effort or cost if compared to off the shelf products, but I personally found it a highly stimulating and satisfactory activity. I highly recommend investing in such a device and estimate that most people will see a return on investment in only a few months (if conscience of their energy usage).

Feel free to contact me if you want more information on the implementation or discuss other ideas.

CRM 2011 filtered lookups - allowing you build more flexible data structures

Wed, 30 Mar 2011 16:52:53 +1300

A common issue encountered with CRM systems is that while modern CRM systems can be very flexible, and allow easy construction of new entities for storing data, the facilities they provide for accessing this customised data structures tend to be limited in subtle ways that may not be immediately apparent, but can have negative impacts on the system over time.

What tends to happen is common entities that could easily be grouped into a single structure (say a contact), can end up being separated into customised individual entities for each type/category of that entity. For example you see separate custom entities for client, partner, supplier etc.

The disadvantage of this that you now have multiple entities to maintain and any changes (e.g. additional fields) need to be rolled out to each of these entities. A better structure is to try and consolidate where practical into a single abstracted entity and simply classify and categorise as necessary. And where major differences in the type of data being collected exists you can create separate extension entities, but still maintain you single repository of core common data.

A simple principle, but with some of the subtle forces at play within many CRM/xRM system, something that is not always easy to apply.

It was interesting to observe that with Microsoft's new CRM 2011, they've introduced a several new facilities that while simple in nature, go a long way to making it easy to avoid the above mentioned pitfall.

One of these is the concept of the filtered lookup. This allows links between different entities to be defined, and when you "lookup" a record on the related entity you can have that lookup predefined to show only results matching a certain criteria.

A simple of example of this might be a health care provider storing all contacts in a single "contacts" entity, but categorising them into contact types such as client, doctor, staff, and facility manager.

On the client application form they may want to record the client, their doctor, and perhaps the manager of the facility they're assigned to. So on the application form we can create three separate filtered lookup fields, which all reference back to the single contact entity, and each of these lookups will use a predefined filter to show only the relevant type of contact. Thus the user will only see "clients" when they use the client lookup field, and will only see doctors when they use the doctors lookup field.

Sounds fairly obvious right? Well it is, and there were ways of achieving this in previous CRM versions, however it generally involved writing large amounts of reasonably complex code to hand build this sort of facility.

This additional complexity in trying to achieve a more flexible data model often lead to the CRM system being setup differently whereby each of these contact types (e.g. client, doctor etc) would be setup as separate entities in the system. While this would work, it meant the CRM system quickly becomes difficult to manage, with large amounts of duplicated structures and reports. For example if you ever wanted to start capturing additional contact information, such as an additional phone number, you would have to go in and edit each of the individual entities and reports separately.

With the new features such as the filtered lookups in CRM 2011 making it much easier to setup a more manageable and flexible data structures (e.g. a single contact entity for all types of contacts), the number and complexity of reports is simplified, and you only have a single contact entity to edit whenever changes are required (and they inevitably are). This allows the business analysts and system designers to build the correct structures to support a much more flexible and easier to maintain system.

There's a lot of value to be gained from seemingly small CRM 2011 features such as this, by making it is easier to design and build simpler and more flexible data structures, the business gains a CRM/xRM system that over time can much more easily cope with the growing and changing demands of the organisation.

Trend watching: SOA and the Cloud

Thu, 24 Mar 2011 14:26:20 +1300

Is SOA dead? Lately, there has been a lot of talk that SOA is dead.

SOA is definitely not dead – but it can be expensive to implement and it has not been adapted as widely as perhaps some commentators have predicted. Claims that it is are either coming from organisations that are far enough advanced in their SOA implementation that it is therefore well integrated in the business, or those organisations that have failed in their adoption and in the best case are only seeing some minor benefits for what may have been a high cost.

SOA investments will grow by 25% worldwide by 2013, says IDC. IDC believes this increased growth will largely come from Cloud computing.

Over the next few years many businesses will be faced with the challenges of replacing existing services with Cloud services and integrating those with existing internal services and other cloud services. A sound SOA architecture can help to ease those challenges and enable the business to gain market advantage.

The operative words are ‘sound SOA architecture’. Today, SOA is mature both from a technology and governance perspective with well established best practices and a wealth of literature. Yet, there are still organisations that ignore this knowledge and experience and end up with SOA spaghetti, higher costs and a low return on investment. These are typically organisations that haven’t bought into some fundamental SOA concepts.

SOA is a long term investment and requires organisational and business process changes to deliver open standard enterprise services. These are built on adaptable components that can participate in BPM processes all of which are crucial for a well constructed service oriented architecture. BPM is a core element of SOA strategy and a sound SOA strategy forms the basis for BPM strategies.

Cloud computing in simple terms is just another sourcing option for selecting and replacing existing systems and to enable BPM processes to be developed that utilise new Cloud based services. Thus, there is a strong link between SOA and cloud computing.

Going forward

Today, organisations could be seen to fall into the following categories when it comes to SOA and Cloud computing:

A – Have sound SOA architecture and is well prepared for Cloud adoption, or has already embraced Cloud computing.

B – Have adopted SOA but have not gained the promised benefits from the investments. Planning to invest or has already invested in Cloud computing.

C – No SOA platform but are planning to embrace Cloud computing.

D – No SOA platform and have no Cloud computing plans.

Organisation from category B and C are those that will largely contribute to the 25% growth over the next 2 years. Category C type organizations have the opportunity to deliver a business case that is focused on real business benefits in terms of (Cloud) Services. SOA forms an enabling architectural technology to achieve the objectives. In this scenario SOA is part of something bigger than just SOA – it provides real tangible business benefits and cost models. i.e. SOA by itself is not the main objective. It’s the Services delivered on the (SOA) architecture that define the value. These services support the business, including:

- Bringing new services to the internet (C2B, B2B),

- Integrating internal enterprise services (often participating in a BPM process),

- Migrating services from one source to another (cloud) source.

Category B organisations have the opportunity to re-define their original business case.

In the local New Zealand market

The IDC numbers are mainly a reflection of the future trends in the US and European markets.

How will the IDC numbers translate into the NZ market?

Today, we find that SOA has been adopted by many larger organizations in NZ and to a lesser extent across SME organizations. Cloud services (will) offer great benefits to SMEs.

Perhaps Cloud computing be the trigger point that increases the growth of SOA across the NZ SME organisations?

Footnote

Optimation has many years’ of SOA (and BPM) experience with a team of architects, developers and test engineers that are involved with SOA on a daily basis. We are excited about the potential uptake of investments into the NZ SOA space. In particular those in conjunction with Cloud computing that are backed by a business case with real tangible value.

Cloud computing for the non-believer

Tue, 15 Mar 2011 11:03:04 +1300

There's a lot of hype and attention currently around so called "cloud" computing, how it's going to revolutionise the computer industry as we know it, and possibly even cure world hunger. When you look more closely at what is (and more importantly what is not) cloud computing, it does seem to be more an evolution of technologies and how we're using them, rather than some sort of revolution that has (or is) going to change the face of computer systems forever.

However that particular discussion could fill several blog posts and is not what I wanted to cover in this article, and there's more than enough discussion on this already as well as what defines cloud computing and how businesses can take advantage of it.

What has gotten my interest is Microsoft's reasonably unique approach, and surprisingly strong (albeit late) entry into the cloud computing arena. With the likes of Google, Salesforce.com and Amazon all having had strong cloud based offerings for some time, many figured Microsoft were too late to the party, are too stuck in the "software on the desktop" model, and did not have the sort of cohesive vision to be able to come up with any sort of compelling offering to compete with the current big names in the cloud space.

Taking a step back, there are still key aspects to cloud computing that many business (possibly even most businesses) are still very uncomfortable with, and these are a cause of resistance to moving critical business systems (e.g. Financial, ERP, CRM etc.) into the cloud. This resistance does appear to be slowly changing as businesses experiment with less critical systems such as ecommerce, extended intranets, and productivity tools like Google Docs and encounter a positive experience. But when it comes to critical business data, issues such as security, IP and retention of data onsite are still large roadblocks for many business decision makers.

Microsoft look to have formed a strategy that not only offers a wider range of cloud services than any other single provider, but they have packaged their offering to provide a unique degree of flexibility that is likely to resonate with those still uncomfortable with throwing their entire business into the cloud.

The key aspects of their strategy that are particularly impressive, and give Microsoft a strong point of difference to the rest of the market (important when you're late to the party as Microsoft so often is) are twofold:

An approach that offers the ability to slice and dice how far you want to go with putting your systems into the Cloud. Want to jump straight in and move everything to the cloud, no problems. Not convinced the cloud is nothing more than a passing fad? That’s fine, these system will run just as well on-premises (e.g. CRM 2011 uses the same code base for the on-premises and online versions). Interested in the cloud but not quite prepared to bet the whole horse and cart? That’s fine too, Microsoft offering is a fairly unique ability to combine and run your systems partly in the cloud and partly on-premises.

Microsoft have a very broad coverage of business applications and platforms, from Office and the Dynamics family through to Server platforms such as Exchange, SQL Server and Windows Server. Microsoft looks to be one of the few providers who can provide such a broad range of systems via the cloud as well. While most can offer one or two of the three cloud based "...as a service" verticals (Infrastructure, Platform, and Software), Microsoft have managed to leverage all their platforms to be able to provide significant enterprise offerings in the "Infrastructure as a Service" (think virtualised servers), "Platform as a Server" (think Azure and Azure SQL), and "Software as a Service" (think CRM Online or Office 365).

This broad coverage in the cloud space has come as a somewhat of a surprise to me, over the years I've become used to seeing Microsoft throwing all sorts of stuff out there, some good some not so good - but usually there has always being a feeling of disconnect and lack of inherent integration between much of their software and platforms.

Not so with their cloud offerings, now you can pick and choose pieces from any of these cloud offerings, and even combine them transparently with any internal systems you want to keep as well. For example maybe you already run SharePoint internally, but want to try out CRM Online. No problems, sign up for CRM Online (takes less than 5 minutes) and you can configure your SharePoint Server to access your new CRM Online and display data from CRM. If you're using SharePoint for document management, you can also assign documents against various account or contacts (or anything else) within CRM Online - and then access and edit these documents from CRM Online or SharePoint.

Another example to illustrate this flexibility; you already have a website hosted with a web hosting company (and you're happy with their service and don't wish to move your site), but you'd like to allow your customers to login using their Windows Live logins. You're also thinking of going with CRM, and would like to take information from CRM and make it available to customers on your existing website - should you go with CRM Online or install CRM on premises?

The answer is it doesn't matter; either option will work just fine. You can use Azure to provide the Windows Live login facilities (it will link directly back to your existing website), and both CRM Online or CRM on-premises will allow you to access and pull out CRM data and display it on your current website.

Thus with Microsoft's approach you are not forced down a particular route, and don't have to engage in an all or nothing battle in order to start your cloud computing experience. Ultimately this level of flexibility is good for businesses as it allows them to build IT Strategies based on their own business needs and requirements, and not be forced into what some vendor might consider to be best practice.

Power conservation with the help of Google Powermeter

Tue, 08 Mar 2011 17:06:42 +1300

We here at Optimation like to do our bit for the environment, here is what one of our architects - Lodewyk Potgieter - has done at home using Google Powermeter to make a difference:

I love the environment and the outdoors and recently decided to pay a bit more attention on my energy consumption. I also belief you can’t improve something that you cannot measure, so I embarked on a quest to find out exactly what my families home consumption pattern is.

There is a wide spectrum of options for measuring electrical power consumption that range from the inexpensive $20 power adapters model to a thousands of dollar fully automated home automation system. The challenge for me was to keep it inexpensive but still get enough information to achieve my end goal.

After a bit of research I decided to focus on Google Powermeter and being the nerd that I am I opted to roll out my own solution by constructing the required devices to capture my energy consumption, but more on that later.

We as a family have always tried to be conservative with our energy consumption so my expectation on real saving was fairly conservative. But I got very excited after getting my new energy bill with a reduction of about 33%. I can contribute a big portion of the saving to reducing “Always On” power. Always On power is the sustained power used during a 24h period that are normally consumed by appliances that don’t really turn off, even when you not actively using it. Examples of these are refrigerators, water heating, TV’s, gaming consoles, DVR’s and computers. Refrigerators and water heaters can obviously not be switch off but the last group of devices is especially bad since they actively draw power without fulfilling any purpose at all. Our treadmill (have not seen a activity for the last six months) was actively using 40 Watts of constant power while switched off. This does not sound like a lot when you consider it cost around $0.25 for a KW/h of electricity but adding it up over a month adds up. Also bring all the other devices into the equation and you can immediately see the difference. The table below will give you an estimate of cost per device over a month when in sleep mode. I estimate my “Always on” reduction to be about 30% of the total saving.

	Watt	KW/h	KW/H per month	Cost ($0.25 per KW/h)	Annual Cost
TV	40	0.04	28.8	$7.20	$86.40
Treadmill	40	0.04	28.8	$7.20	$86.40
Xbox 360	9	0.009	6.48	$1.62	$19.44
Wii	11	0.011	7.92	$1.98	$23.76
PS3	18	0.018	12.96	$3.24	$38.88
Total	118	0.118	84.96	$21.24	$254.88

Power consumption of devices in sleep mode.

The next area to address was other household devices/appliances like lights, dish washer, dryer etc. The first change we made was to change the light bulbs in high traffic areas of the house to energy saving bulbs. Our main family room where we spend most of our evenings for instance had 6 x 75W incandescent bulbs. This equates to a total of 54KW/h ($13.5) per month or $162 over a year if we use the room for 4 hours on average per day. By replacing the lights with 17W compact fluorescent lights we decreased consumption to 12.24KW/h ($3.06) per month or $36.72 per year. That’s a saving of over $100 per year and it does not even consider savings due to the fact that energy saving light bulbs last 10 times longer than incandescent bulbs. Appliances like dryers and dishwashers also use a lot of energy, so only using the dishwasher when it’s fully loaded and limiting usage of the dryer when it’s raining could have a positive impact on your overall consumption but are more difficult to reduce and justify. Some appliances like dishwashers, heat pumps etc. have a “Economy” settings that could also reduce energy usage.

The “Recent usage” graph of Google Powermonitor clearly displays usage patterns during the day as can be seen in the above diagram. The fridge thermostat switching on and off as it regulates temperature is responsible for the little spikes in the graph. Our house use gas water heating but an electric water heating system will have the same effect. The Dishwasher is also clearly visible as it wash and then dries the dishes. The solid dark green bar at the bottom of the graph is “Always on” power used by devices like the PVR, alarm system etc.

The following graph displays our daily usage for a two-week period. The reduction in “Always on” power is clearly visible when looking at the first four days, as we discovered and switched off the offending devices.

I would estimate that the biggest saving and probably the hardest to achieve was due to behavior changes. Switching off TV’s when nobody is watching and turning off lights if not used, does not always come natural especially when you six years of age, but constant reminders and profit sharing (on the actual saving) when it’s pocket money time seemed to return positive results.

I will explore the actual power meter solution in more detail in a follow-up post.

Open Source - Faceted Searching using Solr and SolrJ

Tue, 01 Mar 2011 12:55:18 +1300

Faceted Searching using Solr and SolrJ

Have you ever wondered how some sales websites can provide faceted searches based on things like colour and price ranges, and provide counts of these within a defined search criteria, and doing this with blistering speed.

In this article I will show one approach in achieving this using the open source proct Solr and the java client SolrJ. This approach will make use of the Solr facet queries to provide data back to the web application.

What is Solr

Solr is an open source enterprise search platform from the Apache Lucene project. It has full-text search, hit highlighting, faceted search, dynamic clustering, database integration, and rich document (e.g., Word, PDF) handling.

Written in Java, it can be used as an embedded server or deployed as a web application within a servlet container. As a web application it provides an XML/HTTP API, as well as JASN APIs.

What is SolrJ

SolrJ is the Java client interface for accessing and managing Solr. SolrJ can either interface with Solr deployed as a web application or use it as an embedded server.

In addition to the SolrJ java client, Solr has clients for Ruby - Ruby Response Format, PHP - SolPHP, Python - SolPython, JSON - SolJSON, .NET - solrnet SolrSharp Deveel Solr Client, Perl - SolPerl, Solr.pm, JavaScript - AJAX Solr (a framework-agnostic JavaScript library for creating Solr user interfaces).

Solr Approach to faceted searching

The web store web application will provide the data for displaying items on the web store by querying the Solr index and not the database. This will provide an extremely fast interface for the users due to the overheads that interfacing directly with a database has for this type of application.

There is some basic steps that have to followed

Creating the Solr schema. The schema defines all of the details about which fields your documents can contain, and how those fields should be dealt with when adding documents to the index, or when querying those fields.
Indexing the data. This can be achieved by programmatically adding the items into the index, importing the data as a CSV, or by having Solr directly querying the database using the DataImportHandler (http://wiki.apache.org/solr/DataImportHandler).
Building the interface between the web interface and Solr.

Specifing the facets

The following example will provide counts for every colour, manufactorer and individual price ranges (full source)

query.addFacetField("colour");

query.addFacetField("manu");

query.addFacetQuery("price:[* TO 9999]");

query.addFacetQuery("price:[10000 TO 19999]");

query.addFacetQuery("price:[20000 TO 29999]");

query.addFacetQuery("price:[30000 TO *]");

The Search Term

The actual query is synonomous with most web search engines' query language

Adding the search term (full source)

query.setQuery("motor vehicle");

In my testing against 100000 data items the search was taking about 50 milliseconds

My Output

Showing the output of the facets and 2 rows of results

NUMBER OF RESULTS : 100000
QUERY TIME (msec) : 26
FACET FIELD RESULT : [colour:[green (20117), red (20089), black (19974), white (19971), blue (19849)], manu:[volvo (3701),
jeep (3690), land (3684), rover (3684), suzuki (3673), mazda (3668), kia (3623), lamborghini (3621), lexus (3610),
saturn (3608), infiniti (3600), mercury (3599), bmw (3564), lincoln (3562), ford (3560), nissan (3557), audi (3546), pontiac (3546),
saab (3545), toyota (3542), subaru (3541), gmc (3537), honda (3536), hyundai (3513), mitsubishi (3505), jaguar (3502),
porsche (3486), buick (3481), volkswagen (3400)]]
FACET QUERY RESULT : {price:[* TO 9999]=0, price:[30000 TO *]=80214, price:[10000 TO 19999]=9931, price:[20000 TO 29999]=9855}
id : 0
sku : CAR00-00
name : A wheeled motor vehicle used for transporting passengers, which also carries its own engine or motor. To run primarily on roads,
have four wheels, and constructed principally for the transport of people rather than goods.
manu : Mercury
colour : red
price : 54766.0
id : 1
sku : CAR00-01
name : A wheeled motor vehicle used for transporting passengers, which also carries its own engine or motor. To run primarily on roads
have four wheels, and constructed principally for the transport of people rather than goods.
manu : Subaru
colour : red
price : 83441.0

A More complex query

This shows a query which demonstrates a user clicking on a couple of ranges within the search - of course the implementation would bulid the AND portions of the query on behalf of the user.

query.setQuery("motor vehicle AND price:[10000 TO 19999] AND colour:red");

My Output

NUMBER OF RESULTS : 1989
QUERY TIME (msec) : 51
FACET FIELD RESULT : [colour:[red (1989), black (0), blue (0), green (0), white (0)], manu:[saturn (92), mercury (85),
gmc (83), volkswagen (82), buick (81), hyundai (78), toyota (77), subaru (76), mitsubishi (74), porsche (74), ford (73), jeep (73),
mazda (73), pontiac (73), lexus (71), jaguar (70), suzuki (70), bmw (68), volvo (67), land (66), rover (66), saab (65), infiniti (64),
kia (64), audi (63), lincoln (60), nissan (60), honda (56), lamborghini (51)]]
FACET QUERY RESULT : {price:[* TO 9999]=0, price:[30000 TO *]=0, price:[10000 TO 19999]=1989, price:[20000 TO 29999]=0}
id : 10006
sku : CAR00-010006
name : A wheeled motor vehicle used for transporting passengers, which also carries its own engine or motor. To run primarily on roads,
have four wheels, and constructed principally for the transport of people rather than goods.
manu : Jeep
colour : red
price : 12286.0
id : 10047
sku : CAR00-010047
name : A wheeled motor vehicle used for transporting passengers, which also carries its own engine or motor. To run primarily on roads,
have four wheels, and constructed principally for the transport of people rather than goods.
manu : Jeep
colour : red
price : 16456.0

Simple example source code

In this example I will demonstrate faceted searches that will return back counts of price ranges for a specific query. The Solr server is an embedded server with 100000 data items added to the index.

The Search Example

Java Example to create the sample data set

Getting started with Earned Value

Thu, 25 Nov 2010 14:46:00 +1300

In September, I spoke at the New Zealand Computer Society’s 50th Anniversary Conference. I presented the “lite” approach to Earned Value Management, which we have developed here at Optimation over the past five years. In case you haven’t heard about it, Earned Value is the best way to keep track of the progress of your project, and to compare that progress to the allocated schedule and budget. The problem is that Earned Value is usually seen as very complicated, due to its roots in the US defence and aerospace industries. Here at Optimation, we now have five years’ experience in adapting Earned Value to suit New Zealand conditions and it has worked very well for us. People seemed to like my talk at the conference, particularly the live interactive graphics. So, I’m going to be presenting it again in Wellington on December 9th, as part of the NZCS Innovation Day. It should be a great day, with keynote talks from Rod Drury (of Xero) and Ian Taylor (of Animation Research, makers of the cool America’s Cup graphics). At the end of the talk, I’ll be giving away copies of our “Earned Value Starter Kit”, which is a spreadsheet that makes it really easy to get started with basic Earned Value. If you can’t get a copy from me on the day, you can download it here instead. I hope to see you on the 9th.

Sharpening up our competitive edge

Tue, 27 Jul 2010 12:22:00 +1200

Here at Optimation, we’re sharpening up our competitive edge in more ways than one with the roll out of a company-wide wellness programme and a Summer Shape Up contest. Exercise, eating and sleeping are three basics of maintaining our bodies. You would think these things are obvious. We all know we need sleep, that we need to eat well, and that exercise is vital. But the realities of life mean that actually achieving all three consistently can be tough. With work, families, illness and injuries, it’s easy to fall off the rails. For most of us, work gets squarely in the way of exercise, especially when you’re flat out on a client project. As employers, we know we need to help ourselves and our people find the time and motivation to do something good for themselves. So in Auckland, we’ve partnered with Les Mills to offer a three month Summer Shape Up programme for all of our employees, contractors or permanents. This is not just a gym membership – it’s a carefully constructed health and wellness programme where each person is given their own fitness plan that includes nutrition advice, personal training and group training sessions.

The Wellington team are taking a slightly more independent approach, scheduling regular sessions with workplace wellness expert and fitness diva Mish McCormack of Mishfitness. She’s helping each person design a fitness and nutrition plan that fits their lifestyle. Participants are taking part in a variety of activities including gym-based sessions, home exercise DVDs, and cycling, walking or running in Wellington’s great outdoors (where wind resistance certainly ups the fitness factor). We even have a pole dancing enthusiast! Oh, and Mish is also doing all that scary stuff like taking blood pressure and body measurements, so everyone can track their progress. (Mish has worked as a stunt double on Xena Warrior Princess, so no one is brave enough to slack off on her watch!) To up the motivation to take part (especially since we’re kicking off in the dark winter months, when it’s oh-so-tempting to skip the workout and head straight for the couch), we’ve set up a team competition so everyone gets lots of support. We’re also running weekly group training sessions during lunch breaks, and there are some great prizes for the winning team and individuals. The response to the programme has been fantastic, with around half our Optimates signing up and ten teams taking part. It’s going to be interesting to monitor the results, but hopefully by giving our people every opportunity to take care of their eating and exercise routines, we should have a happier, more motivated workforce. So this leaves sleep as the final part of the trifecta of the three basics of good health. Our theory is that if we give the team the opportunity and the incentive to exercise and eat right (well, better than before, at least), then sleep’s going to be no problem at all.... Let me know what you think of our Summer Shape Up and any ideas that you may have.

People skills for geeks

Thu, 01 Jul 2010 13:40:00 +1200

Last year, I was fortunate enough to attend, and speak at, the inaugural AgileRoots conference in Salt Lake City, Utah. The conference was organised by the outstanding Agile community that has developed in the Salt Lake area. It was a great privilege to be able to visit the official birthplace of Agile (the Agile Manifesto was written at a ski resort near Salt Lake City) and to meet so many of the people I have corresponded with online. The core of my presentation was "people skills for geeks". When I first entered the industry, I had little skill or confidence in dealing with other people, but the ability to present clearly and communicate confidently with both technical and non-technical audiences, is now a required skill for all of us working in IT. In this videoed extract from my presentation, I outline my own personal journey, talk about the fact that "people skills" can be learned, and re-iterate the original Agile goal of valuing people and their interactions. I hope you find it useful and invite you to share any other communication tips that have worked well for you. (And a big thanks to my colleagues at Optimation over the past five years for helping me learn the things I describe in this talk. Thanks also to Optimation for sponsoring my travel :-). )