Create Chaos In Your Company Today, No I am Serious!

Joe Hobot on January 14, 2019

I am tagging this post as #productivity because I believe that is best way to describe something when you are in state of a chaos. Nothing fills... [Read Full]
markdown guide
 

This is so true, we often don't take care of scenarios like what if the database is not available, if a certain validation fails, etc... and we need to make sure the user sees a reasonable message to let them know whats going on.

I think productivity fits to this topic :)

 

A good reminder that we need to be ready/aware for/of possible chaos. I res about that recently in DevOps handbook. It really makes sense if you have many microservices running.

 

Hvala.

See recently I got Kafka/Zookeeper on K8s running and things went smooth. Now I played that Chaos and went ahead and deleted random zk kf pods. Things were good until I found out about wait times and rebalancing etc. Now imagine introducing a lag of 1-2min in prod with say 500 pods. Disaster. :)

 

Hey there,

Some good points in this article, no doubt. I like the more abstract concept of intentionally putting your system in less-than-ideal circumstances, observing the outcomes and acting accordingly. It could help to deliver a really robust system.

I do think, though, that this post is very heavily aimed towards people who work in CI/CD or DevOps. As a web developer, there are many chaotic situations that I could intentionally trigger, but would be unable to resolve. Sometimes, the solution lies with a third party or is restricted by technology.

I think we need to take these principles, but keep them flexible and configure them to match a given discipline.

All in all - great article 😊

 

True, this was more from DevOps prospective, however you could/can work along the DevOps (if you don't have own full access dev env) in where you could observe as to what happens if you shut off network , stop access to database, provide wrong username, dependencies on other apps and such.

When I do things like one above, I work with Devs because they help me provide better insights into app and or logs. So that Noc has a notification at least that an app restarted and healed it self by running script xyz that also restarted dependencies..

Thank you for commenting, much appreciated thoughts from different prospective..

code of conduct - report abuse