EWA — a support’s best (non human) friend
It is software development’s fate to reinvent itself constantly. Striving for even more performance, better maintainability and less costs. In the recent decade this led to a strong movement towards dividing formerly monolithic systems into clusters of smaller, more handy services. Every service fulfills its designated task in a blink of an eye. Queues and streams assure events are sent wherever they are needed. Services are working isolated and are easy to deploy on production systems without noteworthy impacts on their users.
Undoubtedly this brought large, thriving scaling effects into software development departments all over the world. But what if I tell you, this not just has positive impacts?
User support teams are in the urge to work through hundreds or even thousands of user requests, and sometimes also complaints, every day. Have you ever worked in a support department, facing thousands of support tickets every day? Then you probably know that a user support member would never switch from its leading, ticketing system to another administration surface gratuitously. So you might wonder why one should do so. The answer is: they don’t. But if they have to, they only accept one administration surface. Not several.
Well I can tell you of our journey from a monolithic system to a service driven system as mentioned above. And what we found applicable for us and our user support team.
Before we splitted our monolith, user support had one admin surface where they were able to find and edit all relevant user information they needed to work with. Throughout our journey towards a service driven architecture that legacy admin became more and more starved and consequently replaced by several smaller admins, ready to serve only the scope of each service they represent. Hence many different admin surfaces, doing different, but only small chunks on a user support journey, appeared and leaving user support with a bunch of login credentials to many different administration tools, and little overview of how these tools fit into their daily use cases. Thus support ticket lead times increased while user satisfaction decreased, as well as for support team members.
So how to solve this problem, knowing there is no backdoor to the almighty legacy admin of the monolith era?
Well, one strength of human beings is to adapt to new situations. The bad thing about this capability is we are sometimes too adaptive, meaning we tend to get used to new situations and see no urge to change them. That happened to us in a stealthy manner.
This then led to dramatically increased workload in our tech support team, causing them to find a way to get rid of the tremendously increased workload of collecting data that the user support team could no longer manage.
We started with the intention to provide us with a little helper that eases our repetitive and brainless tasks of simply searching for some IDs and setting them into a certain status within one of the new non monolithic admin surfaces. EWA our Efficiency Workflow Automation was born. Once that task was automated we found the next use case to automate and then the next and so on.
Eventually we saw the huge potential our, now called workflow efficiency tool could release in the daily interaction between our customers and user support. But yet we are not there at the moment. Technically we are quite clear what the next steps might be. But handing over such a powerful tool into the hands of constantly overcharged user support members might not be the best decision at that point and need some more refinement and more security checks to undergo before we can take the final step and boost efficiency in user support.
Let’s come to the technical implementation and challenges we faced.
The Workflows Awaken: EWA’s Epic Journey from Chaos to… Slightly Less Chaos
In the Beginning: When Simple Scripts Dared to Dream Bigger
Picture this: You write a Python script. It’s beautiful. It’s perfect. It works like a charm… until your coworkers get their grubby hands on it. Suddenly, your neat little script collection turns into the digital equivalent of a teenager’s bedroom. Time to adult up and organise this mess, right? Enter version control — because nothing says “I’m a real developer now” quite like a GitLab repository. Oh, and while we’re at it, why not slap a UI on this bad boy? How hard could it be?
The Great Infrastructure Facepalm
In our infinite wisdom, we thought, “Hey, let’s just throw this on a web server in AWS. Easy peasy!” Spoiler alert: It wasn’t. Our SRE, bless their patient soul, gently suggested (read: insisted) we use a Docker container instead. Because apparently, our little project needed to play nice with the big kids in our infrastructure playground.
We picked Flask for our web framework. Why? It’s straightforward, well-documented, and let’s be honest, it was the first result in our Google search. MVP time, baby! A dash of HTML here, a sprinkle of CSS there, and voilà! We had a form with an input and a send button. Web design, we’ve conquered you!
The Deployment Saga: It Works on My Machine™
Ah, the age-old developer battle cry: “But it works on my local machine!” Cue the endless hours of staring at logs, questioning our life choices, and wondering if we should have become baristas instead. We asked ChatGPT, Claude, and Gemini for help. They were about as useful as a chocolate teapot, but hey, at least we now know which AI to trust with our future existential crises.
SRE: The Unsung Hero (Don’t Tell Them We Said That)
When all hope seemed lost, our SRE swooped in like a caffeinated superhero. Turns out, we were dealing with not one, but two villains:
- The 30-second Timeout Terror: Because apparently, our load balancers have the patience of a toddler.
- The Inconsistent Upload Mystery: A thrilling game of “server roulette” that nobody asked for.
Solutions? WebSockets and JWT, because nothing says “I know what I’m doing” like throwing more acronyms at the problem.
Automation: Our New Robot Overlords
After what felt like a century (but was probably just a few weeks), we finally stumbled into the promised land of automation. And just in time! An incident hit, and our newly minted EWA saved the day, sparing countless support agents from carpal tunnel syndrome.
Growing Pains: From Zero to Monolith in 60 Seconds
Riding high on our success, EWA grew faster than a beanstalk on steroids. Before we knew it, we’d created a monolith. Oops. But fear not! Our next exciting adventure awaits: “Breaking the Monolith: How Many Microservices Are Too Many?” Coming soon to a blog near you!
Pearls of Wisdom (or, Things We Wish We’d Known Before We Started)
- Infrastructure matters. Who knew?
- Never approve your own code. Your future self will thank you.
- Logging is your friend. A nosy, oversharing friend, but a friend nonetheless.
- AI is cool, but your brain is cooler. Use it occasionally.
- When in doubt, bug your SRE. They love that. (Narrator: They don’t.)
And there you have it, folks. The tale of EWA: a journey of triumph, tears, and far too much caffeine. May your own coding adventures be just as chaotic and rewarding. Now, if you’ll excuse us, we have a monolith to break and a therapy session to attend.
EWA — a support’s best (non human) friend was originally published in willhaben Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.