I think the biggest thing that snipes a lot of technology teams is some notion that production can never ever go down no matter what. Every byte must be synchronously replicated to 2+ cross-cloud regions, etc. Not a single customer can ever become impacted by a hacker, DDOS, or other attack.
Anyone in this industry is prone to these absolutist ideologies. I wasted a half-decade chasing perfection myself. In reality, there are very few real world systems that cannot go down. One example of a "cannot fail" I'd provide is debit & credit processing networks. The DoD operates most of the other examples.
The most skilled developer will look at a 100% uptime guarantee, laugh for a few moments, and then spin up an email to the customer in hopes of better understanding the nature of their business. We've been able to negotiate a substantially smaller operational footprint with all of our customers by being realistic with the nature and impact of failure.
If you can negotiate to operate your product on a single VM (ideally with the database being hosted on the same box), then you should absolutely do this and take the win. Even if you think you'll have to rewrite due to scale in the future, this will get you to the future.
Periodic, crash-consistent snapshots of block storage devices is a completely valid backup option. Many times it is perfectly OK to lose data. In most cases, you will need to reach a small compromise with the business owner where you develop an actual product feature to compensate for failure modes. An example of this for us would be emailing of important items to a special mailbox for recovery from a back-office perspective. The amount of time it took to develop this small product feature is not even .01% of the amount of time it would have taken to develop a multi-cloud, explosion-proof product.
Anyone in this industry is prone to these absolutist ideologies. I wasted a half-decade chasing perfection myself. In reality, there are very few real world systems that cannot go down. One example of a "cannot fail" I'd provide is debit & credit processing networks. The DoD operates most of the other examples.
The most skilled developer will look at a 100% uptime guarantee, laugh for a few moments, and then spin up an email to the customer in hopes of better understanding the nature of their business. We've been able to negotiate a substantially smaller operational footprint with all of our customers by being realistic with the nature and impact of failure.
If you can negotiate to operate your product on a single VM (ideally with the database being hosted on the same box), then you should absolutely do this and take the win. Even if you think you'll have to rewrite due to scale in the future, this will get you to the future.
Periodic, crash-consistent snapshots of block storage devices is a completely valid backup option. Many times it is perfectly OK to lose data. In most cases, you will need to reach a small compromise with the business owner where you develop an actual product feature to compensate for failure modes. An example of this for us would be emailing of important items to a special mailbox for recovery from a back-office perspective. The amount of time it took to develop this small product feature is not even .01% of the amount of time it would have taken to develop a multi-cloud, explosion-proof product.