When we discuss designing a new application or about IT services in general we talk a lot about end user interface, end user experience, cost of downtime and a thousand other things. But I don’t remember having too many discussions about developer, infrastructure engineer or IT consultant experience and how they deal with all the processes and tools surrounding each and every step of developing and implementation of the application or infrastructure. Let me explain what I mean.
You can be a brilliant developer and expert in the code, platform or database administration (I don’t claim to be either of those) but when you land on a new workspace either as in-house staff or as a temporary consultant you quickly realize that those skills don’t cover everything.
First you need to understand the toolset and what tools you are actually able to use. It is not so straightforward as it seems from the first glance. In many cases even a familiar app has to be used by a special way because you need to pass through a special proxy server for outside connections, your IDE or preferable tool for coding has to be set up differently or cannot be used at all due to some restriction or incompatibility with the OS on your provided laptop. And it is probably not a big problem if you are, for example, an Oracle DBA with decades of experience of using SQL*Plus for everything. And you might recall when and why you ditched out all other tools because it was taking too much time to onboard even a simple GUI based interface. But if you are a developer you need a bit more than that and sometimes even Vi or Emacs is not sufficient (can you believe that?).
After installing all the tools to your fresh laptop and going through the endless change-management process of getting approval to get access to the Git and other resources you think you are ready and can start to be useful. But that impression will be short-lived since you find out that you need to read tons of internal documents about the process around every step and action you plan to do. How to get access to the project or code repository. How to get access to the test environment. How to merge your code and deploy it. You find that in one case you need to create a fork of a repository and in another only branch (sorry for technical slang) . In one case you have to use one program for “linting” (checking for syntax) and a totally different one in another. And at least some of that knowledge is coming as a hard experience from mistakes and errors because it’s been kept as a tribal knowledge or instruction for that particular step is hidden deep away in the vastness of the Confluence knowledge-base.
A dedicated long book can be written about the change-management process which can be quite complex and involve multiple steps, different teams and workflows to be approved. For example, in one case it took almost two weeks , 2 different change tickets and two meetings to run a couple of commands in a test environment. In another case it required 5 different tickets in 3 different systems and 4 weeks to get the job done. And in every case you might have a good reason and a history of explanations why it is done this way. In my experience of being DBA, developer, architect the change management process never works as intended and all new changes applied with good intentions make the process even more difficult.
I can go deeper and give more examples but I am going to stop here and think about cost and impact to the job experience. It is fair to say that for some of the challenges it is a one-time activity and has relatively small impact if you are talking about in-house staff. But if you invite somebody to work on a problem for a couple of months it can have significant impact and cost overhead. All the time engineers spent figuring out why a tool doesn’t work or searching through documents for a set of rules or a guide to implement the changes transformed to money and negative experience for the workforce. How much money? It can be significant enough to start thinking about how to fix it. Let’s imagine how much work could be done in a couple of weeks when somebody was trying to push one change in the environment.
So far the entire blog feels like a rant of an old engineer annoyed by the change management process. This is true to some extent and I can’t claim I have a perfect solution for the problem. What I can suggest is a set of potential actions dictated by common sense.
- When you design a new workflow, CI/CD or rules for development try to simplify it for developers and engineers. Think about end user experience for your engineers. They are people too.
- Minimize the number of tools an engineer needs to use for development and deployment.
- Automate installation of tools to the engineer’s laptop or to a VM or any other work environment.
- Avoid deviation of a standard deployment process. Each exception leads to lost time because people will try to search if some kind of exception is applied to every process.
- If you need more than two different tools to apply changes to your code then probably it is not the best approach.
- Periodically revise your change management process to make sure it doesn’t have any unnecessary approvals or steps. Make it reasonable and don’t try to overdue “just in case”.
- If you need more than one change management ticket for one task then it might be a sign that the process is not right.
I genuinely hope the blog is not a totally useless rant but can help to look at the development and engineering process from a slightly different angle. Even small first steps toward removing internal hurdles inside a company can make a big difference for people making your product.
Good reminder Gleb- Real life examples of challenges and resolutions would be helpful if you intend to continue this subject which I will be your reader if you do so,