Even for twitter, how can they afford it? It's the best in the game but charging by ingestion volume is not the best business model especially when they are not paying for storage. They could charge as an alternate based on users and endpoint entities/app instances that are logging instead. Now that Cisco is taking over Splunk, I don't expect it to stay great. But if they make it work it could save Cisco!
Worked for a company that used Splunk... I was told to log less.
That was real strange coming from Amazon, where I was told to log anything and everything... And often, without the logs, it would have been impossible to determine root cause of some issues.
The difference was that logging was super cheap at Amazon, because we'd store all the data, but no indexing on the contents of logs.
If your service is small, or if you are only interested in logs for a specific hosts, you could download, and grep through it... Or run some sort of mapreduce jobs against it. But that was a long time ago. Surely they have better tooling now.
For security logging, Google cloud security (formerly Chronicle) lets you send unlimited data (but less control of the data or what you can do with it of course).
It has competition left and right from sumologic to elastic cloud, perfect time to sell to Cisco!
I've used Kibana and BigQuery, Splunk is lightyears ahead. It's not just for logging, it is excellent at big data analytics and visualization. This is what I struggle to communicate with people that develop and deploy these products. I can write a stupid front end to grep too if I just want to query and regex. I want the query language to let me extract and manipulate fields and their values very easily , let me measure all kinds of stats, control the output, pipeline between outputs and then visualize that data where possible.
You wanna see how manu unique users of Chrome 99.x.y.z transfer how much traffic and how frequently they see what page in your web logs? That's like 3-4 quick SPL lines in Splunk. Everyone else expects you to write parsing somewhere, stats elsewhere and visualization some other place and even then so many limits. Non-splunk users write pages of Jupyter notebook to replicate a short |stats splunk command.
