Ecommerce Platform and Architecture Trends
The way we architect eCommerce solutions is evolving rapidly. The need to innovate faster, integrate rapidly into new systems and deploy quality code at a high frequency are all demanding new, modern approaches to architecture.
These changes in how we provide eCommerce services go hand in hand with the rise of cloud computing, devops, and other advances in development methodologies which promise to provide much greater business agility.
This is in turn is changing the way we view eCommerce platforms; historically the centre piece of an eCommerce strategy, the role of a platform is changing, even diminishing.
Platform vendors themselves are changing the way platforms are delivered to provide lower TCO, greater flexibility and be more suited to SaaS offerings.
This presents both an opportunity and a risk to businesses where online is becoming increasingly competitive. Adoption of modern technology solutions and practices can provide an edge over competitors in the short term, but without a clear direction and buy in from stakeholders to achieve it, businesses could soon fall behind.
In this article, we will explore the key components of a future-proofed eCommerce strategy and also look at how you can begin to build these approaches into your systems today.
History of eCommerce Platforms
In the early days of eCommerce (in the late 90s and early 2000s), the first platforms for eCommerce provided very basic functionality to allow a simple browse and search capability (often just using basic database queries), with the ability to place an order, take payment, and trigger logistical fulfilment. In those days, there was no concept of search engine optimisation, marketing and merchandising tools, facets for filtering etc.
As online transactions grew, so did competition between eCommerce platform vendors as they attempted to acquire the largest market share. Increasing online revenue and therefore platform vendor licensing fees, paid for large ongoing investment in the platforms.
The result was an eCommerce arms race during the 2000s, with the major players (IBM, Hybris, ATG, Magento) adding more and more features to increase the value of their software packages; foundational features such as multi-language / price / currency, through to support for multi storefronts, B2B capability, wish lists, advanced search, search engine optimisation, management tools, basic product and content management, starter store fronts, multi-channel support, preview where added; and the list goes on.
This resulted in extremely functional eCommerce applications that could support a huge variety of complex business requirements. Not only that, frameworks were included to allow customisation to the platforms, so that integrators could fulfil unique business requirements, or develop new eCommerce features not yet available out-of-the-box.
Functionally for retailers, this was fantastic. But drawbacks began to emerge.
Scaling the platform for large traffic volumes became difficult; typically involved running multiple copies of the platform in a highly available configuration, backed by a large database.
An eCommerce platform selection process would be a soul-destroying task of measuring hundreds (or thousands!) of functional and non-functional requirements against the different eCommerce vendors capabilities to identify what was “out of the box” and what would need to be developed or configured.
Deploying updates to the platform to provide new features became a large and complex operation, where even a small change mandated the retesting of the entire application, with the rollout process sometimes taking days, often involving “down time” with a maintenance page displayed.
Significantly, eCommerce platforms became viewed as monolithic applications that are cumbersome and expensive to install, extend and run. Implementing an eCommerce platform from the ground up, could be a large investment taking 12 or more months in many cases. Upgrading to a new major version could also be a significant undertaking.
Image credit: ROELBOB
In the meantime, market pressures with regards to cost and speed of change have led to a new wave of interrelated technology and development methodologies which are reshaping the way we deliver eCommerce features and turning the tide towards more agile architectures.
Organisations and eCommerce delivery teams have gradually started to value speed to market, and a lower total cost of ownership (TCO) and agility, above out of the box capability.
For most organisations, adopting new ways of working is not a case of starting from a blank sheet, as there is too much historical investment in current systems. Therefore, architects need to understand how they can build a roadmap to modernise legacy platform implementations to position businesses for the future.
In the following sections, we look at the key trends in eCommerce delivery, how the platform vendors themselves are adapting, and some strategies for incremental adoption.
Trends in eCommerce Architecture and Delivery
In the next sections we will examine the main areas of evolution in eCommerce architecture and delivery practices.
Many of these trends are interrelated and often complement each other. For example, a headless SPA architecture will be made much easier by adopting a good API strategy, devops will be well supported in the context of cloud hosting, continuous integration methodologies rely heavily on devops automation, etc.
The good news is, many of these strategies can be adopted incrementally, by using Martin Fowler’s Strangler Application approach , or by simply implementing a technical solution a functional area at a time. For example, a headless strategy could be adopted on the home page, following by catalogue pages, and finally account & checkout.
The most important aspect however, is to have a clear vision of the end-state and to have stakeholder buy-in to deliver it; otherwise you may end up with a mixture of solutions across the stack, which can be more costly to support in the long term.
Each of the major platform vendors has their own priorities and roadmap, but there are a few common trends across the major players.
Firstly, there has been a big push towards vendors providing their platforms as a fully managed service, rather than simply selling a software package to be implemented by businesses and their partners.
This appears designed to compete with established platform-as-a-service providers such as Salesforce Commerce Cloud (formally Demandware).
The change in focus from on-premise or private cloud deployments to a fully managed service offering has changed the investment profile.
There has been a marked slowdown in new feature development available out-of-the-box. Using IBM as an example, we have seen very little no feature enhancements in the last two major releases.
A similar trend can be observed with other major platform vendors.
In some cases, the vendors are in fact removing capability from the core platform, preferring integration with a micro-service or other product, to reduce the complexity of the solution.
Instead of feature investment, there has been heavy investment in cloud optimising the platforms, reducing TCO, producing increased API coverage and allowing simpler environment builds / upgrades and improving the integration with the platform and other parts of the vendor’s portfolio.
In part, this is a reaction to the market pressures to reduce the total cost of ownership of the solutions, but also this is designed to make the management of vendor provided fully managed service offerings more achievable.
That is not to say that vendors are not developing new features; new functionality is being delivered as new products, or extensions to parts of the existing portfolio, integrating with the eCommerce platform through APIs or other means.
Another key focus for the vendors, is the ability to externalise any business specific customisation, away from the core platform. Historically, although eCommerce platforms are designed to be extended through customisation, these extensions can introduce additional overhead when upgrades to the platform are applied.
By providing methods for extending the platform, without directly changing the core application, updates to the platform can be rolled out across a large customer base at much less cost.
Decomposing the Monolithic eCommerce Platform
Most organisations recognise the need for a set of core eCommerce services that can provide basic eCommerce capability (customer account, checkout, etc) globally, securely and at scale.
However, the approach of delivering all eCommerce services from a single platform is changing. Architects of eCommerce solutions are increasingly looking at ways of decomposing eCommerce platforms into a set of component parts or services, for a number of good architectural and business reasons: -
- Reduced dependence on a single technology partner and associated roadmap.
- It’s easier in the future to replace a component rather than a monolithic application.
- The technology of choice for a particular component can be chosen to suit its purpose, rather than having to adopt the technology of the platform.
- It is easier to test a component in isolation (rather than a full system) resulting in an improved delivery pipeline (see devops section later).
- Individual components can be scaled for traffic individually.
Note that this diagram
is intended to show the principle of reducing the dependence of an eCommerce
platform by decomposing it into separate architectural components or services,
rather than a full description of eCommerce features. Each organisation will
have its own unique drivers and priorities for decomposition.
The realisation of this approach, is closely aligned with an API first and headless strategy, as described in later sections. Newer cloud-based platforms such as commercetools are capitalising on this trend by providing eCommerce services on demand as APIs in a SaaS model, allowing companies to stitch together a full eCommerce solution using its services and other component parts.
One concrete example of eCommerce platform decomposition i.e. reducing the size and complexity of the platform, is to introduce a headless architecture that splits the user interface out of the eCommerce platform.
In this case, the user-interface (UI) to end customers (e.g. HTML web pages) is implemented in a separate system, using an appropriate technology separate to the platform. Integration between this “head” and the eCommerce platform is via REST APIs.
The choice of system to deliver the UI tier varies across different organisations, with some opting for a commercial package like Adobe AEM or Acquia, and others developing lightweight “heads” using technologies favoured by front-end developers such as NodeJS, and others still using Single Page App (SPA) frameworks such as React or AngularJS, or a combination of the above.
There are a number of advantages to a headless approach: -
The choice of integration approach is an important one, as each has its own challenges. There are two major patterns for headless integration being adopted today. Let’s take a closer look….
Head integration with eCommerce services
This approach involves integrating a commercial CMS (e.g. Adobe AEM, Acquia or other), or custom-built application with eCommerce services and is a relatively common approach.
Note that the services called may not necessarily be just for the eCommerce platform, but could now begin to incorporate other services build for specific parts of the user journey.
Clients (browsers) interact directly with the head, which in turn access eCommerce services to retrieve catalogue data, access account details, create a shopping cart, place orders, etc.
Whilst this is a popular approach and has been successfully adopted by many organisations, it is not a trivial undertaking, and there are a number of factors to consider: -
It is worth noting that it is perfectly possible to achieve a headless architecture in an incremental fashion e.g. delivering first the home page, followed by catalogue pages, and finally account and checkout. This provides earlier business benefit (e.g. better tooling) and reduced risk. However, without strong governance, this can mean that over time the style and experience of the pages served from the head and the eCommerce platform can drift, leading to a disjointed user experience.
Headless Single Page App (SPA)
The term “App” means that the client browser is running an intelligent application (rather than downloading HTML pages) and the term “Single Page” comes from the fact that user interactions (clicking on a link) do not load full web page each time, instead the result of the API call is injected data directly into the browser DOM (document object model).
The result is an extremely fast user interface – only the data required is transferred from the origin of the eCommerce services, and there is no full page reload.
There are a number of benefits from this approach: -
- Increased UI development velocity (compared with coding pages in a platform)
- Frequent UI releases made easy, simply pushing a new version of the JS app
- Much faster response times for end-users
- Promotes an API first strategy (see later)
- Simpler integration with eCommerce services (than a services based integration with the head)
- Ability to support “offline” usage, especially useful for mobile
The offline support is provided through a complementary technology Progressive Web App technologies (PWA) which provides fast initial (first visit) page load time and a Service Worker which can intercept network calls and provide offline storage. This allows the PWA to cache data offline, pre-load data data, etc.
The result is a super-fast modern approach to serving end-users, whilst providing more resilience for intermittent networks (i.e. mobile).
SPA is being adopted quite rapidly across many web sites. In fact, all platform vendors that we work with are in the process of providing a starter store / storefront accelerator using this technology.
SPA can be even more suited (compared to traditional headless approaches) to an incremental rollout with many organisations we work with choosing areas such as store locators, checkout etc for first delivery.
Devops (a culture of unifying development and operations) promotes automation at all stages of software development, from compilation and packaging, through to release management, testing and infrastructure.
Many organisations still lack maturity in devops, mostly due to the constraints of legacy software, platforms and infrastructure: -
- Large eCommerce platforms are more difficult to automate and deploy than smaller services / components.
- Legacy infrastructure was often manually provisioned, including the core software installations, leading to poor configuration management and lack of environment consistency.
- Little investment in automated test packages.
- Relatively high technical debt that has not been addressed.
Most organisations with legacy eCommerce systems will trend from basic through to moderate maturity. Advances in hosting and delivery methodologies make leading devops capabilities within the grasp of most delivery teams, given sufficient governance and investment.
The result can be vastly improved quality and speed of delivery.
There is a strong trend towards public cloud for eCommerce services hosting, particularly with large organisations ($bns) looking to reduce cost and improve agility. Mid-sized retailers have been more opportunistic, looking at cloud hosting at the point that costly private hosting contracts are coming to an end, or a major platform upgrade helps cover the cost of a migration.
The primary reasons for adopting public cloud are: -
- Reduced infrastructure cost. Note that not all cloud usage will necessarily be cheaper that privately hosted alternatives, but typically a reasonably managed cloud deployment can make significant savings.
- Security. Traffic segregation by moving off the corporate network, along with the stringent security standards of major cloud providers provide greater security assurance. Again, cloud deployments are not automatically more secure, and need to be carefully managed and tested to ensure a good level of security is achieved.
- Scalability. The ability to scale infrastructure to meet peak demand can provide greater business assurance that customer demand can be met. Of course, cloud licensing models are important (typically a revenue share) to ensure that platforms can be contractually scaled on demand in cloud deployments.
- Infrastructure as code. Cloud and devops go hand in hand, in terms of providing infrastructure that can be automated through code. With sufficient engineering teams can produce environments on demand for functional or non-functional (e.g. performance) testing that are identical to production environments, thereby increasing quality and assurance.
One major enabler and driver towards cloud deployments, in more recent times, is the move by major platform vendors towards containerised applications. Unlike virtual machines, which provide abstraction at the hardware level, containers provide abstraction at the operating system level, making them extremely lightweight and fast to start and deploy.
The fact that Google runs most of its application infrastructure on containers is testament to the efficiencies that can be achieved using this alternative approach to virtualisation. In fact, Google pioneered the Kubernetes open source framework which is a leading system for the management of containerised environments and provides native support in its cloud platform. Amazon has followed suit recently announcing support for Kubernetes in its AWS platform.
As such, public cloud is fast becoming the de facto choice for hosting, as it provides not only the scalable infrastructure at low cost, but also removes the burden from operations teams of running the services required to support containerised applications in production scenarios.
A containerised environment also provides the ideal ecosystem to deploy other custom (or off the shelf) services in their containers as (micro) services, to assist with decomposition of the eCommerce platform as discussed earlier in this article.
Microservice architectures are the extreme opposite of monolithic applications, which each service (typically a distinct feature) exists in complete isolation, exposing an API. Services are joined in orchestration to achieve given task where necessary or fronted with a component to deliver a customer UI (like a headless architecture).
Micro-services provide a number of advantages, but also present a new set of challenges.
Many large organisations such as Netfix and Amazon have widely published success stories related to micro-service applications but there has been less adoption in eCommerce scenarios. But this approach is not for the faint hearted. Martin Fowler summed it up best in saying: -
“New techniques tend to be adopted by more skillful teams. But a technique that is more effective for a more skillful team isn't necessarily going to work for less skillful teams. We've seen plenty of cases of less skillful teams building messy monolithic architectures, but it takes time to see what happens when this kind of mess occurs with microservices.”
The real challenge with embarking on a pure microservice eCommerce architecture is that is requires writing a lot of code to implement features that are readily available via commercial off the shelf applications. Instead, it would make more sense to integrate a set of commercially available services (e.g. commercetools APIs) into a larger set of services, build new features alongside your existing eCommerce systems as microservices, or replace existing eCommerce features with microservices over time.
Our view is that microservices should be seen as simply another architectural tool to be used when the appropriate need arises, rather than a purest crusade. That said, in the context of eCommerce architecture, adopting the strategies outlined in this article (headless, API first, cloud, devops automation) will create an environment where microservices can more easily deployed when appropriate.
An important change in design approach, is that delivery teams when faced with a business feature request, should no longer ask “how do I best extend the eCommerce platform”, but instead should consider whether to extend the platform, integrate into an available service or deploy a new microservice. This is a complete shift in software development culture and approach.
All major modern platforms provide a comprehensive set of REST APIs that cover the full customer journey and increasingly many marketing, merchandising and admin functions as well.
An API first strategy is one in which all new functionality is first delivered as an API, as opposed to a UI which a human would interact with. Any customer facing interactions are developed as a layer on top of the API e.g. using a headless architecture discussed earlier, a native app, or something entirely different that hasn’t even been thought about yet!
Why is this important? An API first strategy creates standard eCommerce features that can be shared by all channels – regardless of the user interface.
Increasingly, your customers will not just interact via HTML web pages, but use apps, voice, even gesture and channels yet to be defined. Creating functionality foremost as an API allows any channel to interact with your eCommerce features.
In addition, REST APIs provide a standard way for your systems to interact with other internal or third-party systems, including omnichannel initiatives.
This approach is closely aligned to a headless architecture in that it decouples the user interface from the feature itself.
As part of an API first strategy, in the introduction of an API Gateway is an important consideration. A gateway can present a consistent view of all of your APIs, regardless of the back-end system providing the feature, and also help with non-functional requirements such as transformation, security, throttling, auditing etc.
In conclusion, eCommerce delivery teams and architect have many opportunities to improve existing eco-systems, to adopt newer technologies and ways of working, and help deliver a differentiated experience for businesses and end-users.
Every organisation is different, and there is certainly no single architecture that perfectly fits all needs. However, the technical trends described in this article can be used to help shape an architecture and delivery roadmap, to ensure you are best placed to adapt to the rapidly changing landscape of eCommerce.
Salmon has a track record of over two decades helping enterprise clients through eCommerce platforms and to understand tech architecture trends, with hundreds of successful deployments to date. Find out more on related services here.