23 points | by apwheele 3 days ago ago
4 comments
It would be nice if R2 supported Requester Pays like S3. In the past there's various data/files I've wanted to make public but not at my own expense.
Does that approach really work for a 72GB dataset? I assume that means DuckDB will need to load all that data in the browser?
Client side it is aggregations of the data, (so yes you could run out of memory). DuckDB does not load all of the data client side.
I didn’t think it was possible to host such amount of data for free. I have not tried duck db but this gives me the reason
It would be nice if R2 supported Requester Pays like S3. In the past there's various data/files I've wanted to make public but not at my own expense.
Does that approach really work for a 72GB dataset? I assume that means DuckDB will need to load all that data in the browser?
Client side it is aggregations of the data, (so yes you could run out of memory). DuckDB does not load all of the data client side.
I didn’t think it was possible to host such amount of data for free. I have not tried duck db but this gives me the reason