Performance Metrics

Today was one of our busiest days of the year - we delivered excellent levels of service and I’ve captured some key metrics that might be of interest to you below.

The number of unique users and logins (sessions) we served over the course of the day. By New Zealand standards these are pretty high usage numbers: nearly 85,000 users and 160,000 logins.

image


Breakdown of user logins, by hour.

image



The average time to load the user page was just over 0.35 sec average over the course of the day. That metric puts us on a par (or better) with any other pages served online at the same time, including Google at ~0.8 sec.

Our expanded call centred handled an additional 7600 calls, over and above the 2000 handled by our usual call centre operation.

Of all the request to our website over 55% came from a mobile device with around 30% coming off an Apple device.

Unsurprisingly most of our requests originated from Auckland.

image

All in all a very successful day.

Reflections on ReInvent 2017

This years AWS ReInvent was bigger than ever before - the conference spanned five different hotels/conference centres along the Las Vegas strip so getting your daily allocation of steps in was no challenge. Roughly 43,000 people attended the event this year! If you’ve attended ReInvent in previous years you would know that there is a fair amount of walking involved - this year really kicked it up a notch with people shredding shoes in a matter of days

Physical exercise and footwear aside this year event provided what seemed like an endless list of new services in almost every category. CEO Andy Jassy kicked things off with his keynote presentation which riffled through a bunch of new service announcements. They key ones for me were:

  • Amazon Elastic Container Service for Kubernetes (EKS), a managed Kubernetes service running on top of AWS. Simplifying running and managing containers.
  • Aurora Serverless—on-demand, auto-scaling Amazon Aurora. This service eliminates the need to provision instances, automatically scales up/down, and starts up and shuts down automatically. It was very clear that AWS is keep to liberate customers from the tyranny of their existing database vendors (leave you to guess who they mean…)
  • In the Machine Learning space Andy introduced Amazon SageMaker (leverages open source Jupyter project). SageMaker provides built-in, high performance algorithms, but doesn’t prevent users from bringing their own algorithms and frameworks. SageMaker also greatly simplifies training and tuning, and helps automate the deployment/operation of machine learning in production.
  • DeepLens, the world’s first HD video camera with built-in machine learning support. This technology is incredible - I attended the workshop session and walked away with a DeepLens unit so expect more detail on this front in the coming few weeks/months.
  • Amazon Translate, which does real-time language translation as well as batch translation.

Andy’s keynote focused on what ‘builders’ wanted and how they would build the organisations and societies of the future. It’s very clear that AWS is trying to take the heavy lifting out of technology, making it simpler for anyone to be a builder.

It’s very clear that AWS is sticking to it’s ‘customer obsessed’ mantra, not only in terms of how it delivers services to its customers but also in the types/range of services its bringing to market for AWS users to utilise to improve the experience of their customers. Investments in voice technologies, AI and machine learning are all geared towards re-inventing how organisations interact with their customers.

In contrast Werner Vogels’ keynote was light on service announcements and more focused on 21st century architectures and how technology will shape (and will be shaped) by the world in the coming 5-10 years. Werner’s presentation also showcased a number of female techies doing some impressive things in their respective organisations/industries - pretty inspiring stuff.

Werner did announce a couple of key services which stood out for me:

  • Alexa for Business is a fully managed service for Alexa voice-controlled devices at work.
  • AWS Cloud9, a cloud-based IDE which AWS acquired last year. Cloud9 is a clean and feature rich IDE but the ‘killer app’ is collaboration. You can invite other AWS users to join your project for pair programming sessions with a nice little chat box to help you work through bugs (it comes with a full debugger for solo projects as well).
  • Lambda language support for .Net and Go meeting a long requested feature request.

Serverless architectures and services were definite a headline topic this year. A number of the presentations included case studies of AWS customer leveraging serverless technologies to deliver on-demand applications and services. This is consistent with the AWS strategy of ‘business rules being the only thing you will need to code’ in the future.

One of my special interest categories this year was around artificial intelligence and machine learning. It’s clear that AI/ML will bring about unprecedented workforce/job changes in the coming decade. I think a lot of people assume that AI is coming when in reality it’s already here and getting better every day.

A number of services announced were intended to make AI/ML accessible to a wider user base - to take it out of labs and into the hands of people building front-line products and services. These AI/ML developments - paired with things like DeepLens - will pave the way for potentially changing the way we interact with technology in every aspect of our lives.

Cloud adoption still seems to be variable - based on the people I talked to and the round-table sessions I attended. Many organisations are still pursuing the ‘lift and shift’ approach with variable benefits. There are organisations re-engineering their processes and applications as part of the move to cloud but they are still the exception. Worryingly I was actually part of a couple of round-table sessions where some people seemed to be advocating for the on-premise model as a better option.

On a global scale what we are doing around cloud adoption in New Zealand still seems to be on par with what leaders in other parts of the world are doing.

In terms of logistics, you could perhaps argue that ReInvent got too large this year. The travel times between venues were high and I know lots of people missed sessions they wanted to attend due to travel times or popular sessions not offering any walk up spots. From what I remember the 2016 event (which was all at the Venetian) seemed to flow more smoothly with fewer frustrations from attendees. Perhaps it’s time to split ReInvent into two events - one in the US and one in Europe?

The Verdict on Agile

This week Te Papa in Wellington played host to the annual Agile NZ Conference. The team from Assurity have done a great job of hosting this conference over the past few years and its gone from strength to strength - attracting some headline international Agile experts and speakers.

At the same time I have noticed an increase in the number of articles questioning the value of Agile and whether the methodology actually delivers the benefits it promises. Many people I’ve talked to recently have been asking the same questions. After a few years of adoption many organisations are now getting to a stage where they are taking stock of how things are going.

The interest of executives and senior managers in Agile probably peaked about 5 years ago. Everyone was disillusioned with the lack of delivery and cost blow outs associated with the traditional IT project delivery approach. They thought surely there must be a better way - so they (or their consultants) went looking.

What they found was that teams who have a clear purpose, complimentary competencies and communicate well were using Agile practices. Enter a classic case of ‘correlation versus causation’. The assumption people made was that Agile was the key to improving delivery. And why not - iterative value based on more frequent releases, makes logical sense right?

And so happiness reigned in the land again - executives and managers had found a new approach which they had seen delivered value and a whole industry of Agile coaches & experts emerged to help organisation adopt Agile practices. Silver bullet found.

Unfortunately the reality is that most organisations are bad at change, and so what followed was years of largely only the IT function adopting Agile practices - usually with the support of a handful of forward thinking business product owners or stakeholders - whilst the wider organisation around them didn’t shift. Most organisations ended up with what I call the Water-Agile-Fall methodology.

Upstream activities such as governance, portfolio management, resourcing/funding and prioritisation continued to operate in the usual waterfall manner which meant that product ownership/the product backlog were constrained by the same funding/resourcing/changing priorities issues that sank the SS Waterfall.

The adoption of product ownership as a business discipline has also been patchy - so often you end up with product owners who act as proxies for a range of business stakeholders, and who end up being torn between the competing needs of thoss stakeholders without having the mandate to actually make decisions. I think too many business owners see product ownership as an ‘IT thing’ when really it’s about being the responsible owner of the product or service on behalf of the whole organisation.

One of the other major oversights of the ‘Agile scouts’ was that the companies they studied were predominately technology-centric (usually software development) companies or ones that delivered digital products or services. They weren’t based on a classic asset-centric, project cycle based business model - they were based on the construct of product teams and lifecycles. That shift - from project to product - was a crucial part of the Agile recipe which most organisations left out.

Some business models or activity types simply don’t lend themselves to iteration and incremental development - there’s too much rework and wastage introduced. There’s nothing wrong with that - your organisation should be able to pick the right tool (approach/framework) for the job at hand.

So is all lost - do we need the next generation of scouts to go out to the valleys and find us another methodology or framework? My sense is that rather than repeating the cycle organisations should perhaps genuinely commit to change (including difficult conversations around leadership, structures and roles) and utilise frameworks such as Agile to re-engineer their operating models to be fit for purpose.

For me there are a few key things to get you started down that path:

  • The adoption of any framework like Agile needs to be business focused - led by the CEO and executive - with the intention of changing the way everybody works and the way the organisation delivers products & services to its customers. In fact I would start by re-engineering the customer engagement side first and then work backwards.
  • Frameworks and methodologies are no replacement for culture and people. The idea that changing frameworks will improve delivery is so appealing. The reality is that there are no silver bullets; you can’t simply ‘import’ something into your organisation - you have to do the hard yards, figure out what works in your context and then work tirelessly to make it happen.
  • Your primary focus should be on people, not frameworks. Hire and train the best possible people you can find, create a clear purpose for them and then empower them to do awesome things. Creating a culture where people want to stay will ensure that even as you upskills them (and they get offers from other places) they will want to stay. Culture is king.
  • Now is the time for some difficult conversations - using yesterday’s thinking, governance or processes to manage today’s delivery simply isn’t going to work. It’s time for change; for some people that might mean upskilling or career change.
  • Rebuild using the product mindset - nothing stands still anymore and as soon as your product or service hits the market there will be a backlog of enhancement and fix requests. Writing a business case to run a project every few years won’t allow you to keep up with those customer demands. Products require a life-cycle approach and constant stream of investment.

The principles, spirit and practices behind Agile are still solid. It’s the application of those that has let most organisations down over the last few years. The adoption of Agile has been too focused on the framework & tools rather than on the people & practices.

I think the experience of Agile adoption has taught us some valuable lessons. The good news is that all is not lost and with the right leadership and focus organisations can still achieve what they set out to achieve.