Yep. Using DuckDB against conventional data sources such as JSON and CSV or even Parquet has amazing potential. Paired with manifold[1]--a Java library that makes it all type safe--and it's a game changer.
I designed and built an end-to-end application exactly like this about 5 years ago.
All the API calls do are post and get. Nothing processed in transit - all payloads are stored then processed using SQL.
Stable as a rock. Never breaks, one place to go to make mods.
Of course the real reason I built it this way is because SQL is pretty much the only language I know. Glad to finally learn I'm not an outcast after all :-)
There seems to be a few typos in the article, unless I don't understand what's going on.
In step 1, the register uses the source path 'github-events' but the payload is using 'github-pushes'.
In step 3 it says filter_query passes since payload->>'ref' is 'refs/heads/main' but we never see that filter being inserted in the service, the only filter we see is type IN (\'PushEvent\', \'PullRequestEvent\').
Was the article edited midway through and missed some pieces?
Yep. Using DuckDB against conventional data sources such as JSON and CSV or even Parquet has amazing potential. Paired with manifold[1]--a Java library that makes it all type safe--and it's a game changer.
1. https://github.com/manifold-systems/manifold/blob/master/doc...
I designed and built an end-to-end application exactly like this about 5 years ago.
All the API calls do are post and get. Nothing processed in transit - all payloads are stored then processed using SQL.
Stable as a rock. Never breaks, one place to go to make mods.
Of course the real reason I built it this way is because SQL is pretty much the only language I know. Glad to finally learn I'm not an outcast after all :-)
I used to use PostgreSQL as my authentication layer. - Creating users - Hashing & salting passwords - Logins
It was really fast.
There seems to be a few typos in the article, unless I don't understand what's going on.
In step 1, the register uses the source path 'github-events' but the payload is using 'github-pushes'.
In step 3 it says filter_query passes since payload->>'ref' is 'refs/heads/main' but we never see that filter being inserted in the service, the only filter we see is type IN (\'PushEvent\', \'PullRequestEvent\').
Was the article edited midway through and missed some pieces?