With Halloween coming up, would you like to share an IT Horror Story?
Here's Mine:
About 3 years ago, I was contracting with a firm who had a major application written in AngularJS. The product owner had risen up in the corporation from entry level position to owning this company wide application. The product owner was a 30+ something genius (many years younger than me). To him, everything should be easy, he probably had a cell phone within hours of being born.
This application had a daily use of over 50,000 users using it for product information in sites across the U.S. But it's display wasn't right in many parts of the application, and it was flaky.
The authors of this application had been contractors too, but were let-go for reasons I was not told. I guessed the reason, by looking at the horrible huge monolith code patterns. Ahh yes, code comments? "We don't need no stinkin' code comments!"
I had not yet learned how directives in Angular could be used for styling. The app had a 3rd party whiz-bang style component which would hide then slide-in when a particular trigger event occurred, all using directives. Of course trying to set a breakpoint for me at the time was horrible because I didn't know about the directive thing. I kept trying to capture the break based on the display part showing. I made almost no progress for a long time.
Directives are invoked by anonymous Angular Core Functions during the render cycle. This means you must set a breakpoint in the Directive, or dive deep into Angular rendering logic to find Directive Initialization.
In the meantime, this also was my first foray into Less. Was Less causing this? I simply didn't know, so to find out I was doing the change-this-did-it-work? technique to dive deeper. A time consuming horror.
This task was on the board for 5 weeks, I kept putting it off, then picking it up, putting it down etc. as time permitted. Other tasks were picked up, but the worst of them were style changes. The reason turned out to be the CSS was 100% non responsive horror, any change here, knocked out something over there. A mess which screamed of redoing layout based on Grid.
In the product owner eyes, all the tasks were small display changes. In reality it was a total CSS layout redesign. The target dates all fell little by little and repeatedly into subsequent sprints. I missed about 4 or 5 target dates for simple work to be done. A total horror.
Not all of the work was CSS as there were other changes to be made. This is where the environmental issues hit. The application could not be hot-reloaded for browser refreshes after changes. It had 108 endpoints that were hit every-time the browser loaded. About 50 of them had very high security features.
Those endpoints could only be hit if the admins of granted specific machine accessibility. This did not include my desktop/laptop. So to get a fully functioning app, I had to deploy to dev environment for every change. I worked around that by knowing when I had to deploy changes (which used those servers) and when I could just use hot-reloading for other parts. As well as code up automated deploy logic. Another total horror.
Finally the application was flaky, it turned out to be due to a 3rd party endpoint that would not respond properly, causing frequent but different errors. One of them was something like "The endpoint could not be reached" After getting the Vendor on the line it turned out that their software had a memory leak, which occasionally tipped over the entire machine. To us, things would kind-of-sort-of work and then later on would never work for up to 1 hour or more. This was due to it having to be rebooted but I suspect someone had to do it manually. They never told us why it took so long to reboot.
We had almost no 3rd party support, in addition to extremely siloed areas there (such as the network team). The network team was unable to run communication traces due to security, but more than that, they were almost never available to debug an issue and pinpoint the root causes. All of this fell back on my shoulder because the product owner just wanted some small changes.
Top comments (0)