IDP – unifying development and operations not only for cloud natives

Software development projects often start with the daunting task of setting up a functional development environment. Developers typically spend days installing tools, resolving environment-specific issues, and aligning configurations with organizational policies or personal preferences. This process delays productivity and wastes valuable time for the most critical resource: developer. Internal Developer Platforms (IDPs) address these inefficiencies by providing developers with bypassing repetitive setup tasks and begin coding almost immediately – that’s the dream.

In analogy: On the left is the developers (the machinery) and on the right is the customer (deliverable vehicle, ready for operation), in between is the IDP. Image source: https://www.freepik.com/free-vector/industrial-automated-machinery-manufacturing-template_9586813.htm

Another aspect is the gap between developers and the later operation. A term used the last decade was “dogfooding”, often forcing the developers to take care of its own created software while operation. This often forced to abstract infrastructure complexities, but also caused mental overload and hided the needed automation of respective processes. IDPs are more the principle, called “division of labor,” allowed workers to focus on doing one thing very well, rather than being responsible for a number of tasks. By standardizing workflows across teams, IDPs ensure consistency throughout the Software Development Lifecycle (SDLC) and reducing recurring, equal investments. This results in faster release cycles, improved software quality and reduces research and development costs.

The Software Development Lifecycle (SDLC) from helicopter perspective

Term often use in this approach is “shift left”. For example, developers can automate unit testing, do static code analysis reducing the risk of issues escalating later. In context of continuous integration (CI) we often address this as “fast feedback”, because later is worse. Finally it is about early quality which is get mapped into pre-defined templates as best practice. And here is the essential aspect. Organizations which work product oriented more often have recurring tasks, allowing templating. Consider a web application stored in GitHub, for instance, might need a database, DNS configurations, and storage volumes. An IDP can provision these resources automatically while enforcing organizational policies. For a platform (product) the inconsistencies across environments are reduced with an IDP, making it easier to troubleshoot and maintain applications. This directly reduces service “service management” aka operational effort. However, challenges arise in project-based organizations, where diverse teams require customized configurations. Platform engineers tackle these issues, especially in project oriented organization, by aligning architectures with continuous integration and continuous deployment (CI/CD) pipelines and developing reusable tools, ensuring smooth deployment without developers being bogged down by infrastructure complexities. Note, deployable or even deployed is not delivered. Only when it is used, it is operated.

IDP as central portal for all involved parties

As pipelines and toolchains grow longer with DevOps and continuous delivery, developers and especially platform engineers must balance “shift left” and “shift right” strategies. “Shift left” involves addressing quality early. Conversely, “shift right” focuses on monitoring and validating performance in production. Practices like A/B testing and fault injection help teams analyze real-world behavior and improve system reliability. Site Reliability Engineers (SREs) play a critical role here, ensuring stability and performance under varying conditions. According to the statement “if we can break the system also a 3rd party can break it”. I like this statement because it well describes the nature of a SRE. SREs are not waiting until a request happened to the operated service, it is more a proactive role trying to corrupt the operated system, introducing resolutions before they happen.

Jointly improve automation

Policies like “start right” templates and “stay right” automation ensure developers follow best practices without running into mental overload, because performance indicators (KPIs) like deployment frequency, lead time, and mean time to recovery (MTTR) are not their business. Organizations often struggle to balance the flexibility required by diverse teams with the need for standardization. Value flow focused measures such as “project 2 product” takle the right aspect but in my view lack the view on the individual developer tasks. Identify bottlenecks and optimize workflows continuously are appropriate measures, nevertheless they ignore the specific “engineer” tasks and expertise. Here automation tools for scaling and cost optimization come into the play.

If we combine automation with collaboration IDPs ensures reliability and scalability, even in non-cloud-native environments. By simplifying the complexities of development, platform engineering empowers teams to focus on their core mission: building exceptional software. But beware for the cargo culture. Just having a tooling such as Jenkins, working with pull requests (PRs) as well as having a single repository approach will not by itself lead to better code quality, faster time to market or continues “whatever”.

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *