At the end of 2020 we have a quite good amount of backend services, frontend components, .net libraries and of course lot’s of internal systems and tools. It could be around 400 pieces of mostly unique entities. Since we have used many tools over the years it was kind of difficult to quickly find out following questions.

• Who is the owner team?
• What is the component good for?
• Is it still under development, ready to be used in production or already obsolete?
• Where is the technical, business or operational documentation?
• Where is the source code for this nuget, npm or tool?
• Where is the backend service deployed, like to which environment or region and in what version?
• What are the dependencies of the service or component?
• How much does it cost to run?

Answers for the before mentioned questions were scattered around all kind of sources… Enterprise Architect, SharePoint wikis, sites, Azure DevOps project descriptions and wikis, README files…even Excel files…lot’s of excel files. And sometimes only in someone’s head.

Usually it took even hours to get to the correct team, wiki pages were without structure, not all have rights or license to open all tools… And not only it was hard to find it. When you finally shout “Heuréka!” you realized quite often the information found was already outdated.

At that time Vladimir Pindura returned from some meeting with Microsoft and showed me Backstage from Spotify. And I found out it might be the answers for our pains. I have hacked together some working prototype and our journey started.

Why Backstage and what is it?

Why Backstage? It’s answering the before mentioned questions. It’s aggregating data in favor of duplicating them: it has lots of integration possibilities. Manually entered catalog information are close to the source visible to developers: in most cases in the same repository as the source code. It defines clear schema for the business & entities description. It has a full text search among all entities (meta data & technical documentation etc). It’s a very flexible both data and visualization and integration wise. It’s open source with really lot’s of contributors.

What is it? It’s open source portal written in typescript running in Node.js with React frontend. In case you want to implement it you would need at least one typescript (full stack) developer. It’s not install and configure tool yet. It’s more about creating custom skeleton app and integrating those plugins you want to use. Also it’s about integrations, about data. You will probably spend lot’s of time cleaning up your data sources, refreshing them, preparing integrations with your identity provider, source control management system and building catalog data. Spoiler alert: there is also SaaS version of Backstage, check out the web https://roadie.io/! To sneak peak of about 10% features you can also check out https://demo.backstage.io/. There is no registration required at the moment for the demo.

Implementation in Oriflame

The most important we needed to do on our journey was to talk. Talk with all current and future stakeholders, with other architects, management, team leaders, product owners, developers,… Explain to them how the Backstage might help them. Ask them for feedback, ideas, questions.

Maybe the same important is to have the documentation ready: what is Backstage, how to prepare the definitions. To evangelize the Backstage ideas to the internal audience. To undergo few rounds of workshops to let the developers understand the Backstage, get to know it and finally love it.

For us it was also important to find out the “killer feature“. The feature people would find the most useful for them and the main reason they would open Backstage at the first place. Also it might be quite specific. For team leaders it was a working hierarchy tree with the description, links and members of the team loaded and refreshed from the Active Directory (that is no need for them to keeping up to date some already outdated Excel sheets with team structure). For QA and release team it was the dependency visualization of dependencies. For management it was probably the cost insights plugin that visualize how much each team spends daily of cloud costs.

Architecture

Let’s talk about our current architecture for Backstage:

At Oriflame we are using Azure Active Directory for authentication and Azure DevOps as Source Control Management. Both is supported by Backstage: Microsoft Azure Authentication Provider, azure-devops plugin.

I won’t write a lot about where and how we run the backstage itself or how the CI/CD pipeline works. As you can see from the above diagram we do deploy it on Azure Kubernetes with PostgreSQL flexible server and as for CI/CD we have prepared our own Azure DevOps pipelines.

Catalog data

What I want to highlight is the way how we work with catalog data. We have one repository that have multiple index files with respective entities. We have following entities in the catalog:

• Backend components index contains mainly backend services that are either auto-generated based on our configuration manager or linked from the service repository in case the custom catalog definition exists there
• Frontend components index contains single page applications (SPAs) and other frontend components (mainly React). Either auto-generated based on package.json in the (mono) repo or again, custom definition is possible
• Active Directory index contains entities such as teams (aka groups), streams/departments, users… since we do not have our AD group hierarchy in a best shape and since the catalog-backend-module-msgraph plugin was not ready/useful when we needed to implement it, we did our custom integration via Azure DevOps pipelines. This way we can also enrich the groups with custom links to the Azure DevOps backlogs, team channels and more.
• Azure Resources index contains entities such as subscriptions, SQL Servers/Databases, REDIS caches, key vaults, app insights,… we have custom tags defined in the Azure and thanks to them we are able to link the resources with respective Backstage entity and also specify the component and/or environment, region etc. Thanks to this we can generate deployment diagrams automatically, we do have links to the actual resources and we can track the costs down to the resource type level.

To help teams jump on the Backstage train and prepare their own definition we have prepared comprehensive documentation, sample definitions and also incorporated the process of Backstage definition to the overall process of service/component creation.

Integrated plugins

There are plugins we were able to integrate out of the box quite easily:

• Azure AD plugin that helps with authentication and teams hierarchy and users
• Azure DevOps plugin that allows source code integration, Pipelines and Pull requests overview
• Kubernetes plugin that allows to see status of all pods deployed for the service over all AKS clusters at once
• Sonarqube plugin showing SonarCloud overview for backend and frontend components
• Lighthouse plugin with lighthouse container calculating and showing lighthouse score for websites or SPAs
• API plugin easily integrates documentation for API built from OpenAPI specification with the help of swagger
• Explore page and org plugins with team hierarchy overview

Plugins with custom integrations

Following plugins we needed to somehow integrate / put more effort to make them run:

• For tech-radar only the front end component is available. We needed to somehow store and load the data by ourselves.
• For cost-insights only the front end component is available. We needed to implement integration with Azure Cost Management API. In Backstage we do have only quick overview about the system costs with a link to PowerBI report with comprehensive filtering and grouping, month to month comparison etc.

Our own plugins

Few plugins we needed to implement by ourselves. It was:

• Dependency board. This plugin allows teams to show released version of apps and their dependencies across environment, regions. It visualize eventual issues and possibilities for release. It has a frontend part integrated to Backstage and then a backend service that aggregates data from Azure DevOps and Kubernetes about deployed versions of applications.
• Changelog plugin shows published versions of services/components and the changes (work items from Azure DevOps) associated with each release.
• Score-card plugin. Main idea behind it comes from a need to somehow visualize maturity of our services and to establish a process how to improve it (discuss with the teams what to focus on next). So it’s more about a review process and how to visualize the outcome for other teams for inspiration.

Screenshots

Final word

What did Backstage bring to Oriflame? Mainly

• More steep learning curve for new engineers
• More effective engineers in terms of looking for information they need and also working on daily (devops) tasks
• Costs in Azure under more control thanks to cost-insights plugin and powerBI integration
• Documentation automation = no need to manually define resources nor API documentation.
• Access to organization meta-data (teams, hierarchy, ownership, systems descriptions/tags..) via API
• Data democratization. We have now open data about our services, teams, costs for all Oriflamians. Everyone can now build solutions on top of them helping others.

Hope you have enjoyed the reading! In case you have any feedback/comments/questions don’t hesitate to ping me.