Sorry for the delays here. All the stories and comments missed should be backfilling as we speak. It's almost done. I posted updates further down, but wanted to post here since it's the top comment. We'll get an update on what happened and our plan to make sure we are on top of this in the coming days.
Btw, HN Replies also stopped working for me a month or two ago. Not sure why (I imagine something to do with my email or domain - re-registering didn't help), but if you find a chance to look into it I'd be grateful.
Thanks for the heads-up! HN Replies is up and running though, and notified me of a test comment I just made. If anyone would like to troubleshoot their subscription, feel free to email me.
Thanks for replying! Is the email necessary? It seems it's at least affecting 2-3 folks who replied here, so if you might be able to just look into email sending/delivery for us, that would be wonderful.
I sent you an email. Notification emails through the service seem to have stopped on my end on or around the 13th, if that helps, though I don’t post every day or get replies every day.
Try https://hcker.news (I made it), the front page interface is made for hn. It captures pretty much all you could do with the algolia site as a FP and then some.
Unlike most hn frontends out there i don’t rely on algolia’s api for any of its functionality, so it’s resilient to their outages.
I became aware of it last night when a few users emailed me of chronological data in my app being outdated since Friday. Then went to GitHub and saw an issue was already open and upvoted by 8 others.
Hey folks, sorry for the delay on this. We're investigating and I promise a response on what happened here. Will update in a bit when I have more, and will make sure to let you all know what happened.
We are backfilling the comments and stories we missed. You should start seeing them in the search results page. I will update when it's all caught up.
Apologies for this again, didn't want to send an update until I had something and every time it felt so close.
We'll work on an HN update on what happened here and lessons learned. (spoiler alert: ownership has become muddled over 10 years so we had to figure out how the portion that stopped crawling worked. That was fun. We'll make sure to work on a clear plan here and be transparent about it. So sorry :( )
I just happened to be checking the website "manually" because hnrss.org stopped updating, this explains why lol. Glad to know it'll hopefully be resolved soon.
Glad there is a thread about it as it seems to have happened before. Also if using certain IP ranges, sometimes the search is forbidden. It would be great to have a more robust search without the blocking.
Sorry all. We've let the Algolia team know and they're working on it.
Sorry for the delays here. All the stories and comments missed should be backfilling as we speak. It's almost done. I posted updates further down, but wanted to post here since it's the top comment. We'll get an update on what happened and our plan to make sure we are on top of this in the coming days.
How much does this service "cost"? What's it run on?
Btw, HN Replies also stopped working for me a month or two ago. Not sure why (I imagine something to do with my email or domain - re-registering didn't help), but if you find a chance to look into it I'd be grateful.
That's a third-party project but I've emailed Dan to let him know.
(That's a different Dan. A different Dan G in fact! but not a dang as far as I know.)
Thanks for the heads-up! HN Replies is up and running though, and notified me of a test comment I just made. If anyone would like to troubleshoot their subscription, feel free to email me.
Thanks for replying! Is the email necessary? It seems it's at least affecting 2-3 folks who replied here, so if you might be able to just look into email sending/delivery for us, that would be wonderful.
I sent you an email. Notification emails through the service seem to have stopped on my end on or around the 13th, if that helps, though I don’t post every day or get replies every day.
Oh I had no idea that's not you. Thank you!
I normally use the Harmonic app and it hasn't been loading post replies the last 2 days or so, citing a likely Algolia issue. Maybe a related issue.
They have released a fix for that and now comments are properly displayed again
HACK app on iOS, macOS and android works along with the push notifications for replies:
https://apps.apple.com/app/id1464477788
https://play.google.com/store/apps/details?id=com.pranapps.h...
I am the developer.
I have found HN replies always just stop, after some time. I resubbed a couple of times, and never bothered, after the last one stopped working.
This has been going on for me, for a couple of years, I think.
What do you use now for replies? Because the RSS I was using stopped working too.
I don’t use anything. Not the end of the world. Most of the replies I miss are probably shade, anyway.
Same!
This has happened before and it will happen again. Outages happen.
Can you share with us what the best point of contact or escalation path to reach out to the next time it happens on the Algolia side?
I'm not sure what the best point of public contact is, but maybe nickv can say something about that when there's a minute...
Someone else posted this as well, but ... you wouldn't be able to find it.
There are dozens, dozens of us that use hn.aloglia, much better interface than the front page.
Try https://hcker.news (I made it), the front page interface is made for hn. It captures pretty much all you could do with the algolia site as a FP and then some.
Unlike most hn frontends out there i don’t rely on algolia’s api for any of its functionality, so it’s resilient to their outages.
I'm getting 500 after visiting
Shouldn't be happening, please try again!
I use it as a front page, because it shows flagged posts and ranks them by total number of votes, not by arcane algorithms.
You can also try https://news.ycombinator.com/active
I became aware of it last night when a few users emailed me of chronological data in my app being outdated since Friday. Then went to GitHub and saw an issue was already open and upvoted by 8 others.
Hey folks, sorry for the delay on this. We're investigating and I promise a response on what happened here. Will update in a bit when I have more, and will make sure to let you all know what happened.
Quick update.
We are backfilling the comments and stories we missed. You should start seeing them in the search results page. I will update when it's all caught up.
Apologies for this again, didn't want to send an update until I had something and every time it felt so close.
We'll work on an HN update on what happened here and lessons learned. (spoiler alert: ownership has become muddled over 10 years so we had to figure out how the portion that stopped crawling worked. That was fun. We'll make sure to work on a clear plan here and be transparent about it. So sorry :( )
Was this the same issue as on February 1st? Or are these outages unrelated?
Ah man then how do I know if I beat the other person for the latest simonw or rachelbythebay submission?
The "from site" feature is native to HN.
You're probably kidding, but it's a cool feature and maybe someone doesn't know about it yet.
When you click the url next to a submission, it shows all the submissions from that website.
I just happened to be checking the website "manually" because hnrss.org stopped updating, this explains why lol. Glad to know it'll hopefully be resolved soon.
You can Google or use LLMs they index HN too.
Glad there is a thread about it as it seems to have happened before. Also if using certain IP ranges, sometimes the search is forbidden. It would be great to have a more robust search without the blocking.
RSS was broken too? At least https://hnrss.github.io/ was broken.
Yup, I noticed yesterday that all items were a few days old. It seems to work now.
Seems so, because I don't get notifications for rss anymore
Yes, hn-rss is powered by algolia search as well
It's back up
HN Search has poor defaults: "stories by popularity for all time". Almost never what you want which is recent content, and often comments and stories.
I noticed this too, using it for https://kindred.fit.
Hope this gets fixed soon. I've had to write a fallback for my extension to use the HN firebase but it's a lot less efficient for the data I need.
This outage helped me work through my backlog of articles. I've also reduced the frequency that I refresh the feed. Silver linings :)
Ok. I thought it was some change in the algorithm.
The change is that the algorithm is off vacationing in Bali.
He's in Bala.
https://m.youtube.com/watch?v=SRvUxIaD-cY&pp=0gcJCf8Ao7VqN5t...