For me personally, the ability to write custom authentication plugins for HTTPie is the reason I use it. For my use case I wrote a simple AWS Sigv4 [0] plugin to simplify calling AWS API Gateway endpoints during development/debugging.
I'm in the same boat - looking to consolidate a few smaller data pipelines that have organically grown over the years into a single pipeline and taking the opportunity to try and get the org to rethink how we emit events in various parts of the stack. As teams operate fairly independently, we need something that enables a flexible data structure whilst still enabling the ability to join data on common well-defined fields.
The current path we are going down is looking at using a 'nested' JSON structure that enables sub-processes in the various systems to inherit values from the parent. Something similar to:
The structure itself will be documented using JSONSchema and hopefully we will be able to verify the validity of events as they are processed, though this might be too expensive to do in high volume scenarios.
The goal is to then build a low latency router that takes in routing policies to forward subsets of events to further data pipelines. The policies themselves will be defined using some kind of DSL (possibly using JSONPath?).
As a whole, this seems to be a fairly common problem[0] that other companies are trying to solve, but a lot of the low level details are not spoken about. One thing that does seem to be a common component to a service like this is Apache Flink.[1]
Thanks for sharing! So it seems you may end up using an incremental counter as a version – or was that a placeholder example?
What are the main use cases for the data pipelines you're forwarding your data into? Are the events fairly rich or fairly simple? Do you plan on doing any type validation as well?
if it's a "json" like data that is sent to you (i.e a structure made out of only primitive types and list/dict) it may be not that hard, but I think it will be a work with very few value outside of this very specific use case.
It looks like the compiled binary just gets exec'd by the Lambda function and listens on a TCP port (specified by the environment variable _LAMBDA_SERVER_PORT) which accepts RPC calls to execute the handler using net/rpc.
Interesting concept, but an unauthenticated SMS gateway might not be a good idea. I hope there is throttling and other means of abuse prevention in place.
I recently purchased Dropshare[0] when it was on 'special' for $0.99 on the App store. This is more targeted at being able to send the screenshots to your own server (via scp) but has screenshot and video recording support.
Though, I imagine you could easily achieve the basic functionality by writing a quick script to run the cmd line 'screencapture' and tying it to a hot key combo.
As you say, I'd prefer to run this sort of thing locally - half the point of running your own CA is that you have full control over it.
I wrote caman (https://github.com/radiac/caman), a bash script wrapper for openssl with what looks like a similar syntax to etcd-ca. I posted about it on HN a while back, but it now also supports SAN certificates and intermediate CAs.
[0] https://github.com/docker-slim/docker-slim#debugging-minifie...