I currently only have my phone, so i can't judge the API. From my point a full scrape at regular intervals is not that bad. Its only 1000 requests. Depending on the data and querymethods you xan make fresh data appear sooner than removing old data.
The major question is: how fresh do you need your data?
Not every application needs realtime data, querying it only on occasion or every few hours can be good enough.
Thanks for your point of view, I appreciate very much.
About data freshness, we would like to serve on our system the same information that you can find on the source website, and we have a few tickets pointing the difference to solve, so maybe the continuous strategy each hour may minimize it as today we do it once a day.
I currently only have my phone, so i can't judge the API. From my point a full scrape at regular intervals is not that bad. Its only 1000 requests. Depending on the data and querymethods you xan make fresh data appear sooner than removing old data. The major question is: how fresh do you need your data?
Not every application needs realtime data, querying it only on occasion or every few hours can be good enough.
Thanks for your point of view, I appreciate very much. About data freshness, we would like to serve on our system the same information that you can find on the source website, and we have a few tickets pointing the difference to solve, so maybe the continuous strategy each hour may minimize it as today we do it once a day.