Platform Engineering to Empower Developers: A Better Way to Meet Current Challenges

By Matt Miller | Trace3 Innovation Principal

In a quest to create and deliver value faster, development teams of all sizes face a shared problem: cognitive overload.

Can DevOps help solve this problem? What is DevOps?

On the surface, DevOps is about breaking the barriers between development and IT operations teams, and bringing together the technologists to automate testing, integrations, and deployment of software. Originally coined over 15 years ago, the approach has grown from its agile roots to a methodology focused on continuous integrations (CI), continuous delivery (CD), infrastructure as code (IaC), and a renewed focus on metrics such as:

While the concepts have gained traction and lots of organizations have implemented team changes to foster this approach, many have hit a stall at the tools and process level because they have not adopted the culture changes of the practice. Failure to lean into the DevOps culture has led to anti-patterns, hustle culture, and hero work. The real goal of DevOps is to improve the quality and delivery of business value. This requires a cultural shift in the IT organization to align the Ops and development team’s goals to business goals. DevOps does not solve or address a development team’s cognitive load but focuses on goal alignment between two traditionally separate teams to deliver streamlined business value.

Why change anything? The teams just need to get it done.

Traditionally, when only a few in an IT organization can deploy environments or even support changes, there becomes a bottle neck to the flow of change. Reducing this friction to the flow of change will increase the speed of delivering value. This also decreases the “throw it over the wall” approach seen in many development vs operations units.

Conversely, setting up environments or parts of the “app stack” to deploy and support a developed product is a significant overhead to a development team. For instance, time spent on container orchestration or troubleshooting environment variables before code deployment reduces the time available for development. Development time is value add time and environment setups or configuration tweaks are considered an extraneous cognitive overhead for developers.

Reimagining a team’s structure with a focus on intrinsic and business-aligned cognitive load can not only increase productivity but can allow for business goals to be met faster.

The journey from DevOps to Platform Engineering

Many attempts to adopt DevOps have led to unintended outcomes, including anti-patterns to which IT Ops responsibilities are absorbed by the development team. Over time this usually leads to the development team's most experienced and expensive senior resources being responsible for deployment, maintenance, and monitoring of environments. Junior developers become dependent on Senior developers to add and deliver value. Some IT organizations see this as the most undesirable situation and abandon or recoil their DevOps initiates, while others use it as a chance to explore a new platform strategy that is known today as Platform Engineering.

Platform teams are established to treat development, product, and service engineering teams as a customer, collecting feedback and taking a product approach to automation, deployment, toolchains, and all the glue to make it work. This adds layers of abstraction allowing for the developer and developer team’s cognitive overhead to decrease, which ultimately increases the delivery of value. Developers now have the room to concentrate more on their value-stream aligned domain and solve problems and use cases for the business while leaving the nuances of deploying containers, clusters, and web farms up to the platforms.

What is Platform Engineering?

Platform engineering is the practice of constructing and managing self-service internal developer platforms. Each platform layer is managed as a compelling product to optimize developer experience and productivity. In software engineering, the goal of platform teams is to optimize the developer experience and accelerate the product team’s delivery of customer value.

Successful platform teams lower the developer cognitive overhead by providing golden paths and paved roads published by internal developer platforms (IDP). The aim of these patterns is to provide an abstraction of the tools and automation while preserving repeatable and trusted security standards.

The Benefits of Platform Engineering

An organization can gain many advantages from Platform Engineering, such as:

• Faster time to market: By providing reusable components and automation, platforms can reduce the development and deployment time of applications.

• Higher quality: By enforcing best practices and standards, platforms can improve the quality and consistency of applications.

• Lower costs: By leveraging economies of scale and shared resources, platforms can reduce the operational and maintenance costs of applications.

• Greater innovation: By enabling feedback, platforms can foster a culture of innovation and continual feedback.

The Challenges of Platform Engineering

Platform Engineering also has some difficulties, such as:

• Complexity: Platforms can introduce additional complexity and dependencies, which can affect the performance and reliability of applications.

• Trade-offs: Platforms can impose trade-offs between flexibility and efficiency, which can limit the customization and optimization of applications.

• Governance: Platforms can require governance and coordination, which can affect the autonomy and agility of developers.

The Tooling Layers of Platform Engineering

Platform Engineering is not a single or one-size-fits-all solution, but rather a context layered approach that requires careful planning and execution. An example of common Platform Engineering layers:

The Developer Control Plane hosts the Developer Portal, integrated development environment (IDE), distributed version control system, infrastructure as code, and application specific source code. Specifically, the internal developer portal provides the users with an enhanced self-service experience, a key component to Platform Engineering.

The Delivery and Integration Plane hosts the CI/CD pipelines, orchestration platforms, and image registries. A few emerging solutions provide capabilities that span across these features.

The Logging and Monitoring Plane hosts the observability tools. The concept is not that a platform deploys these specific tools, even though it could, it is more that the platform onboards services and resources into these observability tools based on standard metrics and SLOs.

The Security Plane hosts the security-centric tools including solutions that provide static application security testing (SAST), software composition analysis (SCA), secrets and certificate vaults, cloud native service gateways, and policy-based access management platforms.

The Resource Plane hosts the main compute, networking, data, and services to support the core technology services for the platforms.

The Best Practices of Platform Engineering

Platform Engineering involves some of the following best practices:

• Understand the needs: Platforms should be designed and built based on the needs and expectations of the developers and product teams.

• Deliver value: Platforms should deliver value to the consumers, both in terms of functionality and user experience.

• Evolve iteratively: Platforms should evolve iteratively, based on feedback and data, rather than being built upfront and fixed.

• Empower the users: Platforms should empower the users to be self-servicing, rather than being restrictive.

The Future of Platform Engineering 

Platform Engineering will continue to evolve and incorporate innovative services as development and product teams need current and future capabilities. Some of the current innovations being incorporated into Platform Engineering are:

• Cloud-native: Platforms will leverage the capabilities and benefits of cloud computing, such as scalability, elasticity, and resilience.

• Microservices: Platforms will adopt the microservices architecture, which enables modularity, decoupling, and independence of applications.

• Serverless: Platforms will embrace the serverless paradigm, which abstracts away the infrastructure and management of applications.

• AI/ML: Platforms will incorporate artificial intelligence and machine learning, which enable intelligence and automation of applications.

This blog post is a continuation from Trace3’s Innovation Team 2024 Enterprise Technology Themes series.