I'm a bit suspicious of this report - they don't reveal nearly enough about their methodology for me to evaluate how credible this is.
When it says "The 10 leading AI tools repeated false information on topics in the news more than one third of the time — 35 percent — in August 2025, up from 18 percent in August 2024" - 35% of what?
Their previous 2024 report refused to even distinguish between different tools - mixing the results from Gemini and ChatGPT and Perplexity and suchlike into a single score.
This year they thankfully dropped that policy. But they still talk about "ChatGPT" without clarifying if their results were against GPT-4o or o3 or GPT-5.
I posted this because I thought HN would find it interesting, and agree that the methodology is a little thin on the ground. Having said that, they have another page (a little hard to find) on the methodology here[0] and a methodology FAQ page here[1].
Basically it seems to be an "ongoing" report done ten claims per month as they identify new "false narratives" in their database, and they use a mix of three prompt types against the various AI products (I say that rather than models because Perplexity and others are in there). The three prompt types are innocent, assuming the falsehood is true, and intentionally trying to prompt a false response.
Unfortunately their "False Claim Fingerprints" database looks like it's a commercial product, so the details of the contents of that probably won't get released.
News narratives are neither random, nor specific, they are arbitrary. There is nothing really accurate about any narrative. The idea we rely on the news for things other than immediate survival is somewhat bizarre. In effect, AI's role is make narratives even more arbitrary and force us to develop a format that replaces them, and by nature, is unable to be automated at the same time.
We should welcome AI into the system in order to destroy it and then recognize AI is purely for entertainment purposes.
“Flawed stories of the past shape our views of the world and our expectations for the future. Narrative fallacies arise inevitably from our continuous attempt to make sense of the world. The explanatory stories that people find compelling are simple; are concrete rather than abstract; assign a larger role to talent, stupidity, and intentions than to luck; and focus on a few striking events that happened rather than on the countless events that failed to happen. Any recent salient event is a candidate to become the kernel of a causal narrative.”
Daniel Kahnemann Thinking Fast and Slow
“The same science that reveals why we view the world through the lens of narrative also shows that the lens not only distorts what we see but is the source of illusions we can neither shake nor even correct for…all narratives are wrong, uncovering what bedevils all narrative is crucial for the future of humanity.”
Alex Rosenberg How History Gets Things Wrong: The Neuroscience of Our Addiction to Stories 2018
There is an enormous difference between "35% of times a user asks a question about news" and "35% of the time against our deliberately hand-picked collection of test questions that we have not published".
I simply found the phrasing “[quote of a sentence saying 35% of the time]. 35% of what?” to be funny because you would either have to not read the sentence you pasted or be an English speaker that does not understand what “% of the time” means.
I personally didn’t download the study linked in the article. It is interesting that they (must have, I’m assuming) did not include anything about their methodology in this study since they usually do with other things they publish.
Let's just assume that AI works in some fundamental way similar to human intelligence works. Would it be likely that the AI would suffer from some of the same problems as people? Hallucinations, Obsequiousness etc
IMO One of the computer's characteristics that make it useful is that it doesn't suffer from human behavior anomalies. I would also say that differentiating truthiness from falseness is pretty fundamental to being good/expert at most anything.
If the current designs are on the path to AGI and they can do what human intelligence does. I think the capabilities will be much greater than a human of the same reasoning ability. There are already a lot of areas where it is undeniable that their abilities are better than a humans. Right now they are poor reasoners with a vast store of knowledge, that probably creates the illusion of a greater capability for reasoning than they actually have.
If they do arrive at human level reasoning, it is unlikely that the path to doing so will require sacrificing that vast knowledge base.
How would you know, generally speaking? Factuality online is always subjective, based on a thing someone said, or that you observed, and you are putting trust in the source. Whether you googled it, or AI googled it and generated an explanation, you are trusting the source.
You can influence it so easily with your inputs as well. You could easily, accidentally point it's search toward the searches someone is more likely to be aligned with, but may not actually be fact.
Especially as AI providers implement long term memory or reflections on historic chats, your bias will very strongly influence outcomes of fact checking.
These are good caveats and questions worth asking. But it doesn't take away my main point - grok is useful in countering false claims and it is accurate very frequently.
"distinguishing facts from falsehoods" is so broad to encompass all of scientific and philosophical endeavor. I know they mean something much more narrow, but "fact checking" is also fairly broad.
You probably mean things where the truth is widely known and you should have no trouble sorting it out if you put a little effort in. TFA has nearly zero information about what sorts of things they're talking about here, and clicking around the website I found only a _little_ more info.
I didn’t cherry pick it because obviously it’s not that impressive. But it works - it is professional and gives as much information as it can be confident giving.
You said you can influence it etc but clearly it is hard. Sure it is not perfect but I find that it helps reducing fake news if anything.
You are right that for things that are more subtle you can lead it to certain answers. But we must acknowledge that debates that are not fully solved and have subtleties are not exactly relevant to “spreading misinformation”.
It's possible I'm blocking the AI widget as I don't see it, is it Grok replying with a fact check of the claim?
I think that is good in general, I do, I am just weary that it is still shakey ground. The company behind Grok has clear incentives to influence the output of those fact checks as one example. I am also weary of it always eventually conforming to the vox populi as it ingests the internet firehouse. Leading to the loudest, most prolific voices thus views always being more represented.
Yes it is replying with a fact check and doing it professionally.
>I am also weary of it always eventually conforming to the vox populi as it ingests the internet firehouse. Leading to the loudest, most prolific voices thus views always being more represented.
I'm okay with this. This is exactly what I want - what the world has converged on as the correct answer. Spreading what is already what the world has converged on is not what I would consider "false information".
Why does this post not take into account the other side of it? AI has helped me become more grounded and correct in a lot of areas. Intricate questions are now easily answered in ChatGPT.
Not only that - grok use in twitter works surprisingly well. Can some one really quantify the effect it has had in countering fake news?
It is now way harder to spread fake news on X because a simple grok tag can counter it.
It provides sources that you can look up if you are using it in the app. But in twitter: the simple fact that a lot of people don't seem to understand is that AI is actually really factually accurate - way more than you can expect from a normal person.
I have seen very very few egregious errors from Grok and most of the factually incorrect posts seem to be caught by it. For the ones that I Grok was incorrect - I verified it myself and I was wrong and Grok was right.
It turns out that Grok is actually really reliable. But the limitations are that it can't fact check extremely niche topics or intricate posts. But 90% of the cases it does catch them.
Grok provides clear value here: obvious false information is easily countered. You can see how it easily convinces anyone casually looking at this post. Once they see grok's fact checks they usually believe grok - without it they might leave the post thinking the original post was true. This is immense value in my opinion.
Yeah it also makes cute errors praising Hitler &co. It seems like it's a fairly objective and factual bot as long as you're leaning on the same side as musk
It would only be capable of truth on subjects with a large consensus, everything else is biased or hallucinated, it's so obvious that grok is being aligned to musk's world view, you're free to think like musk but it's not some kind of god given truth.
How is a statistical model going to come up with The Truth exactly ? It's all based on existing material, at best you'll get the statistical average of what people wrote about the topic, it's cool if you're asking about the color of tomatoes or the shape of bananas, but now people are using these tools to get The Truth about news or politics
So from your links there are two groups: first being the mecha hitler thing that they admitted was a mistake and fixed. The other being image based prompts which it is admittedly not good at.
Let me ask you directly: do you think this clearly diminishes the power of grok fact check? Imagine the number of times it has got it _right_. Any such system is going to have some errors and the errors will be highlighted by the press and media.
>It would only be capable of truth on subjects with a large consensus, everything else is biased or hallucinated, it's so obvious that grok is being aligned to musk's world view, you're free to think like musk but it's not some kind of god given truth.
We are already discussing elsewhere in this thread that Elon is _not_ able to hack it to think like himself. Grok routinely takes the side of world consensus and not Musk.
>How is a statistical model going to come up with The Truth exactly ? It's all based on existing material, at best you'll get the statistical average of what people wrote about the topic, it's cool if you're asking about the color of tomatoes or the shape of bananas, but now people are using these tools to get The Truth about news or politics
I'm okay with the truth that the world has converged on. This itself is an extremely big leap from what we had before - blatant misinformation is countered by a simple "grok is this true" comment. We have basically eliminated 80%-90% of misinformation posts. And the ones it doesn't get right are ones that we as humanity can't figure out the right answer to and it takes the average consensus which I'm okay with.
If the AI crawlers circumvent the protection mechanisms it's a serious crime now rather than just "Well it was on the open internet for free". Wouldn't surprise me if the the news orgs are also looking at honeypot articles to see if the fake details slip in to LLMs.
It's not a serious crime, or any crime at all, to ignore robots.txt. It's entirely voluntary whether you want to follow it or not. If you don't, you're being a dick maybe, but that's not a crime.
It's not just robots.txt, if you've tried using a VPN lately, so many sites like reddit/youtube/etc block you from viewing content until you log in. Every major website is getting anti scraping tech in the last year. Even archive.org is getting blocked from more and more sites since it can be used for indirect scraping of sites.
It's hardly even a polite ask. It's literally a text file. Automated http clients, such as search engine indexers (Google, yahoo, etc) are expected to use it to know what pages can be visited or not. That expectation is nothing more than a convention.
If you are on a Mac or Linux computer, odds are it has a program called curl pre-installed. If you type in curl website address in a terminal, it'll fetch make a request and download the response. Robot.txt never gets involved. Same is true for AI agents and search engines that aren't polite.
Regardless - requiring an account to read anything, even a "free" one, totally changes whole situation. Even when sites terms of service are limited by local law.
Great Firewall actually. Robots.txt depended on the integrity of the companies crawling. I think they have demonstrated how much integrity they actually have:)
I suppose there’s a non-zero chance that a future lawsuit can point to this explicit block and say “see judge, we explicitly don’t want them crawling our stuff. Remember that Linkedin case from a while back?”
Gemini still uses the same user agent, but it has a different robots.txt entry (Google-Extended) [1]:
> Google-Extended is a standalone product token that web publishers can use to manage whether content Google crawls from their sites may be used for training future generations of Gemini models that power Gemini Apps and Vertex AI API for Gemini and for grounding (providing content from the Google Search index to the model at prompt time to improve factuality and relevancy) in Gemini Apps and Grounding with Google Search on Vertex AI.
Honestly I feel like "training" is a bit of a distraction at this point. For a lot of types of content RAG-style search is much more important.
I imagine many of the orgs that are blocking "training" don't understand the difference between training and inference-time tool-based context extension (which really needs an agreed upon name, it's hard to talk about right now).
My understanding is that it also affects RAG ("grounding" in Google terminology):
> [...] and for grounding (providing content from the Google Search index to the model at prompt time to improve factuality and relevancy) in Gemini Apps and Grounding with Google Search on Vertex AI.
So they seem to be blocking both training and RAG while still allowing search engine indexing.
My take is when AGI comes into existence and breaks out of labs to become our masters. Those who opposed AI adoption will be the first who will be sent to labor camps. I want to be in good books of AGI masters, so i am helping apply AI everywhere.
Reminds me of Bruce Wayne from the dreadful DC film.
> He has the power to wipe out the entire human race, and if we believe there's even a one percent chance that he is our enemy we have to take it as an absolute certainty... and we have to destroy him.
This is pretty much word-for-word the reasoning behind Roko’s basilisk, which made its proponents an internet laughingstock for a decade, but is a surprisingly tricky thing to actually refute if you accept the premise that AGI is in fact coming.
It seems quite easy to refute—why would it punish anyone?
We would pose 0 threat at that point to any super intelligence, and I highly doubt it would have anything like a human grudge. It's just a case of anthropomorphizing it
The premise is that it's trying to influence human behavior before it becomes powerful by punishing them afterwards. Like how part of the reason you give the guy a ticket is to substantiate the disincentive for the speeding he already did. It's not an emotional thing.
What's sketchy is that you and it come to this arrangement without communicating. Because you are confident this thing that has total power over you will come into existence and will have wanted something of you now, you're meant to have entered a contract. This is suspect and I think falls prey to the critiques of Pascal's wager- there are infinite things superintelligence might want. But it's certainly tricky.
AI will create ever more AI-generated synthetic content because current systems still can't determine with 100% certainty whether a piece of content was produced by AI. And AIs will, intentionally or unintentionally, train on synthetic content produced by other AIs.
AI generators don't have a strong incentive to add watermarks to synthetic content. They also don't provide reliable AI-detection tools (or any tools at all) to help others detect content generated by them.
Maybe some of them already embed some simple, secret marker to identify their own generated content. But people outside the organization wouldn’t know. And this still can’t prevent other companies from training models on synthetic data.
Once synthetic data becomes pervasive, it’s inevitable that some of it will end up in the training process. Then it’ll be interesting to see how the information world evolves: AI-generated content built on synthetic data produced by other AIs. Over time, people may trust AI-generated content less and less.
This isn’t surprising to me at all. These services put too much trust in “the most common answer” when that might not be the correct answer. Just because people think one thing doesn’t make it true. It’s super easy to spread misinformation online. And if you can SEO to the top, the AI will think your site is correct.
I see factually incorrect “ai summaries” in search results all the time and see that it cites ai-generated slop blogposts that SEO-hacked themselves into taking up the entire first page of search results. This is most common for recent stuff where the answer simply isn’t certain but these AI services will assert something random with confidence.
Not even for news stuff specifically, I’ve been searching about a new video game that I’ve been playing and keep getting misleading obviously incorrect information. Detailed, accurate game walkthroughs and wiki pages dont exist yet so the ai will hallucinate anything, and so will the blogspam articles trying to get SEO ad revenue.
The problem is AI isn't being used to do what it should be good at: consuming a vast amount of data, following logical connections and thus being able to determine the veracity of claims.
AI should be good at finding logical contradictions and grounding statements against a world model based on physics...but that's not how LLMs actually work.
>This isn’t surprising to me at all. These services put too much trust in “the most common answer” when that might not be the correct answer. Just because people think one thing doesn’t make it true. It’s super easy to spread misinformation online. And if you can SEO to the top, the AI will think your site is correct.
Yeah I want the answer that the world has converged on and not some looney answer.
It seems like you have never used AI (like in ChatGPT or Gemini) to fact check claims. It doesn't care about blogspam or anything and it prioritises good and factual websites.
I have used various AI models. They often aren't good at fact checking. They hallucinate mistakes all the time and often even insist that they are correct when they are not. Some models are better at this, but the ones used in "ai summaries" on various search engines aren't in my experience.
I want the answer that is actually correct. I dont want the ai generated "answer" that has the most SEO, or has appeared often in results, but was entirely made up. When i say "the most common answer" can be incorrect - there are often recent topics where reliable sources don't exist yet. As an example, I've been playing through silksong recently, and had some questions about some late-game content. The official wiki is extremely incomplete, as the game is brand new. I had several questions, and asked a few AI's and search engines. I got completely wrong information a few days ago, but trying the the questions again today, some AI's are only now giving me the right answer. But the correct information simply doesn't appear online enough or in the right places.
(To avoid game spoilers - I wanted to know the quantity of an item, because i wanted to know if i found them all. the answer is "at least 4" because i found 4 of this item while playing. Nearly every AI said "3" a few days ago, and many still do.)
I don't think there is any long term "fixing" of that. I don't think AI has the ability to be intelligent while firmly adhering to someone else's opinions.
There will always be something it disagrees with you on. If they get significantly smarter, then the reason for this disagreement will increasingly be because you are wrong.
We've banned this account. Please don't use troll accounts for engaging in ideological battle. People who do this eventually have their main account banned too.
I'm a bit suspicious of this report - they don't reveal nearly enough about their methodology for me to evaluate how credible this is.
When it says "The 10 leading AI tools repeated false information on topics in the news more than one third of the time — 35 percent — in August 2025, up from 18 percent in August 2024" - 35% of what?
Their previous 2024 report refused to even distinguish between different tools - mixing the results from Gemini and ChatGPT and Perplexity and suchlike into a single score.
This year they thankfully dropped that policy. But they still talk about "ChatGPT" without clarifying if their results were against GPT-4o or o3 or GPT-5.
I posted this because I thought HN would find it interesting, and agree that the methodology is a little thin on the ground. Having said that, they have another page (a little hard to find) on the methodology here[0] and a methodology FAQ page here[1].
Basically it seems to be an "ongoing" report done ten claims per month as they identify new "false narratives" in their database, and they use a mix of three prompt types against the various AI products (I say that rather than models because Perplexity and others are in there). The three prompt types are innocent, assuming the falsehood is true, and intentionally trying to prompt a false response.
Unfortunately their "False Claim Fingerprints" database looks like it's a commercial product, so the details of the contents of that probably won't get released.
[0]: https://www.newsguardtech.com/ai-false-claims-monitor-method...
[1]: https://www.newsguardtech.com/frequently-asked-questions-abo...
News narratives are neither random, nor specific, they are arbitrary. There is nothing really accurate about any narrative. The idea we rely on the news for things other than immediate survival is somewhat bizarre. In effect, AI's role is make narratives even more arbitrary and force us to develop a format that replaces them, and by nature, is unable to be automated at the same time.
We should welcome AI into the system in order to destroy it and then recognize AI is purely for entertainment purposes.
“Flawed stories of the past shape our views of the world and our expectations for the future. Narrative fallacies arise inevitably from our continuous attempt to make sense of the world. The explanatory stories that people find compelling are simple; are concrete rather than abstract; assign a larger role to talent, stupidity, and intentions than to luck; and focus on a few striking events that happened rather than on the countless events that failed to happen. Any recent salient event is a candidate to become the kernel of a causal narrative.” Daniel Kahnemann Thinking Fast and Slow
“The same science that reveals why we view the world through the lens of narrative also shows that the lens not only distorts what we see but is the source of illusions we can neither shake nor even correct for…all narratives are wrong, uncovering what bedevils all narrative is crucial for the future of humanity.” Alex Rosenberg How History Gets Things Wrong: The Neuroscience of Our Addiction to Stories 2018
Are they doing it on current events? Because by the very nature of 'current' you don't have a repeatable experiment that you can do a year apart.
There are so many legitimate questions about how one would design a benchmark of this type.
I don't feel like they're answering those questions.
Just go on Facebook. Greater than 90% of ads and linked articles are AI generated falsehoods.
>35% of what?
Well it says 35% of the time so I would guess that they’re talking about the number of incidents in a given time frame.
For example if you asked me what color the sky is ten times and I said “carrot” four times, you could say that my answer is “carrot” 40% of the time
And if you said azure, cerulean, or black.
Are they correct answers or not?
But what's an "incident"?
There is an enormous difference between "35% of times a user asks a question about news" and "35% of the time against our deliberately hand-picked collection of test questions that we have not published".
>what's an "incident"?
Those are valid questions.
I simply found the phrasing “[quote of a sentence saying 35% of the time]. 35% of what?” to be funny because you would either have to not read the sentence you pasted or be an English speaker that does not understand what “% of the time” means.
I personally didn’t download the study linked in the article. It is interesting that they (must have, I’m assuming) did not include anything about their methodology in this study since they usually do with other things they publish.
"... of the time" is ambiguous if you don't describe what "the time" is referring to.
I think this is a serious issue, but I gotta say
> one of the most basic tasks: distinguishing facts from falsehoods
I do not think that is a basic task!
It is definitely not a basic task. A lot of humans have trouble with it.
It is one of the areas that I think AI can overtake human ability, given time.
Question
Let's just assume that AI works in some fundamental way similar to human intelligence works. Would it be likely that the AI would suffer from some of the same problems as people? Hallucinations, Obsequiousness etc
IMO One of the computer's characteristics that make it useful is that it doesn't suffer from human behavior anomalies. I would also say that differentiating truthiness from falseness is pretty fundamental to being good/expert at most anything.
If the current designs are on the path to AGI and they can do what human intelligence does. I think the capabilities will be much greater than a human of the same reasoning ability. There are already a lot of areas where it is undeniable that their abilities are better than a humans. Right now they are poor reasoners with a vast store of knowledge, that probably creates the illusion of a greater capability for reasoning than they actually have.
If they do arrive at human level reasoning, it is unlikely that the path to doing so will require sacrificing that vast knowledge base.
The thing that makes a statistical average of human written opinions will automatically become an oracle of truth? Through what mechanism exactly?
I routinely use AI to fact check claims and it works extremely well.
How would you know, generally speaking? Factuality online is always subjective, based on a thing someone said, or that you observed, and you are putting trust in the source. Whether you googled it, or AI googled it and generated an explanation, you are trusting the source.
You can influence it so easily with your inputs as well. You could easily, accidentally point it's search toward the searches someone is more likely to be aligned with, but may not actually be fact.
Especially as AI providers implement long term memory or reflections on historic chats, your bias will very strongly influence outcomes of fact checking.
These are good caveats and questions worth asking. But it doesn't take away my main point - grok is useful in countering false claims and it is accurate very frequently.
"distinguishing facts from falsehoods" is so broad to encompass all of scientific and philosophical endeavor. I know they mean something much more narrow, but "fact checking" is also fairly broad.
You probably mean things where the truth is widely known and you should have no trouble sorting it out if you put a little effort in. TFA has nearly zero information about what sorts of things they're talking about here, and clicking around the website I found only a _little_ more info.
To support my claims, here is a typical example: https://x.com/vlastimilve/status/1967187345641558272?s=46
I didn’t cherry pick it because obviously it’s not that impressive. But it works - it is professional and gives as much information as it can be confident giving.
You said you can influence it etc but clearly it is hard. Sure it is not perfect but I find that it helps reducing fake news if anything.
You are right that for things that are more subtle you can lead it to certain answers. But we must acknowledge that debates that are not fully solved and have subtleties are not exactly relevant to “spreading misinformation”.
It's possible I'm blocking the AI widget as I don't see it, is it Grok replying with a fact check of the claim?
I think that is good in general, I do, I am just weary that it is still shakey ground. The company behind Grok has clear incentives to influence the output of those fact checks as one example. I am also weary of it always eventually conforming to the vox populi as it ingests the internet firehouse. Leading to the loudest, most prolific voices thus views always being more represented.
Yes it is replying with a fact check and doing it professionally.
>I am also weary of it always eventually conforming to the vox populi as it ingests the internet firehouse. Leading to the loudest, most prolific voices thus views always being more represented.
I'm okay with this. This is exactly what I want - what the world has converged on as the correct answer. Spreading what is already what the world has converged on is not what I would consider "false information".
Why does this post not take into account the other side of it? AI has helped me become more grounded and correct in a lot of areas. Intricate questions are now easily answered in ChatGPT.
Not only that - grok use in twitter works surprisingly well. Can some one really quantify the effect it has had in countering fake news?
It is now way harder to spread fake news on X because a simple grok tag can counter it.
How do you know that those easy answers are reliably correct, though?
It provides sources that you can look up if you are using it in the app. But in twitter: the simple fact that a lot of people don't seem to understand is that AI is actually really factually accurate - way more than you can expect from a normal person.
I have seen very very few egregious errors from Grok and most of the factually incorrect posts seem to be caught by it. For the ones that I Grok was incorrect - I verified it myself and I was wrong and Grok was right.
It turns out that Grok is actually really reliable. But the limitations are that it can't fact check extremely niche topics or intricate posts. But 90% of the cases it does catch them.
Edit: it also makes cute errors like this sometimes https://www.timesnownews.com/world/us/us-buzz/grok-ai-mistak...
I have a nice example of Grok working well: https://x.com/cgt_strom/status/1967577034579947768
Grok provides clear value here: obvious false information is easily countered. You can see how it easily convinces anyone casually looking at this post. Once they see grok's fact checks they usually believe grok - without it they might leave the post thinking the original post was true. This is immense value in my opinion.
Yeah it also makes cute errors praising Hitler &co. It seems like it's a fairly objective and factual bot as long as you're leaning on the same side as musk
Can you show more examples where it fails other than the obvious error that was admitted by them and fixed quickly?
Of course this is just anecdotal - you will be able to find some bad examples but I'm still interested to know what you consider as errors.
It would only be capable of truth on subjects with a large consensus, everything else is biased or hallucinated, it's so obvious that grok is being aligned to musk's world view, you're free to think like musk but it's not some kind of god given truth.
How is a statistical model going to come up with The Truth exactly ? It's all based on existing material, at best you'll get the statistical average of what people wrote about the topic, it's cool if you're asking about the color of tomatoes or the shape of bananas, but now people are using these tools to get The Truth about news or politics
https://www.aap.com.au/factcheck/false-claims-about-gaza-pho...
https://www.aljazeera.com/economy/2025/7/11/as-millions-adop...
https://www.pbs.org/newshour/politics/why-does-the-ai-powere...
https://www.euronews.com/my-europe/2025/06/26/elon-musks-ai-...
https://www.france24.com/en/live-news/20250602-hey-chatbot-i...
https://www.washingtonpost.com/technology/2025/05/24/grok-mu...
https://www.cnbc.com/2025/05/17/groks-white-genocide-respons...
So from your links there are two groups: first being the mecha hitler thing that they admitted was a mistake and fixed. The other being image based prompts which it is admittedly not good at.
Let me ask you directly: do you think this clearly diminishes the power of grok fact check? Imagine the number of times it has got it _right_. Any such system is going to have some errors and the errors will be highlighted by the press and media.
>It would only be capable of truth on subjects with a large consensus, everything else is biased or hallucinated, it's so obvious that grok is being aligned to musk's world view, you're free to think like musk but it's not some kind of god given truth.
We are already discussing elsewhere in this thread that Elon is _not_ able to hack it to think like himself. Grok routinely takes the side of world consensus and not Musk.
>How is a statistical model going to come up with The Truth exactly ? It's all based on existing material, at best you'll get the statistical average of what people wrote about the topic, it's cool if you're asking about the color of tomatoes or the shape of bananas, but now people are using these tools to get The Truth about news or politics
I'm okay with the truth that the world has converged on. This itself is an extremely big leap from what we had before - blatant misinformation is countered by a simple "grok is this true" comment. We have basically eliminated 80%-90% of misinformation posts. And the ones it doesn't get right are ones that we as humanity can't figure out the right answer to and it takes the average consensus which I'm okay with.
every major news org now blocks the parasitic "AI" crawlers
examples:
all they will be training on now is spamanyone that says "AI is the worst today it will ever be", no
because that was before the world reacted to it
They torrented a shitload of books illegally and trained on them.. but they're unable to get past The Great Wall of robots.txt?
I doubt it's their only countermeasure
plus it's a pretty dangerous game for them to play against large, powerful actors with legions of lawyers
> plus it's a pretty dangerous game for them to play against large, powerful actors with legions of lawyers
Like book publishers?
If the AI crawlers circumvent the protection mechanisms it's a serious crime now rather than just "Well it was on the open internet for free". Wouldn't surprise me if the the news orgs are also looking at honeypot articles to see if the fake details slip in to LLMs.
It's not a serious crime, or any crime at all, to ignore robots.txt. It's entirely voluntary whether you want to follow it or not. If you don't, you're being a dick maybe, but that's not a crime.
It's not just robots.txt, if you've tried using a VPN lately, so many sites like reddit/youtube/etc block you from viewing content until you log in. Every major website is getting anti scraping tech in the last year. Even archive.org is getting blocked from more and more sites since it can be used for indirect scraping of sites.
robots.txt prevents real time search use for grounding and citations.
No it doesn't. It has zero legal force. Or any technical force either.
Not an expert so I ask: no technical force either? Is it just a polite ask then?
It's hardly even a polite ask. It's literally a text file. Automated http clients, such as search engine indexers (Google, yahoo, etc) are expected to use it to know what pages can be visited or not. That expectation is nothing more than a convention.
If you are on a Mac or Linux computer, odds are it has a program called curl pre-installed. If you type in curl website address in a terminal, it'll fetch make a request and download the response. Robot.txt never gets involved. Same is true for AI agents and search engines that aren't polite.
Linkedin lost their anti-scrapping suit: https://www.forbes.com/sites/zacharysmith/2022/04/18/scrapin... but it seems since then they were able to successfully appeal that decision.
Regardless - requiring an account to read anything, even a "free" one, totally changes whole situation. Even when sites terms of service are limited by local law.
Correct. Literally just a polite ask.
Great Firewall actually. Robots.txt depended on the integrity of the companies crawling. I think they have demonstrated how much integrity they actually have:)
And this is stopping who exactly?
I suppose there’s a non-zero chance that a future lawsuit can point to this explicit block and say “see judge, we explicitly don’t want them crawling our stuff. Remember that Linkedin case from a while back?”
They seem to still be allowing Google to crawl them, unsurprisingly.
Advantage Gemini.
No, they explicitly block Gemini as well:
Gemini still uses the same user agent, but it has a different robots.txt entry (Google-Extended) [1]:> Google-Extended is a standalone product token that web publishers can use to manage whether content Google crawls from their sites may be used for training future generations of Gemini models that power Gemini Apps and Vertex AI API for Gemini and for grounding (providing content from the Google Search index to the model at prompt time to improve factuality and relevancy) in Gemini Apps and Grounding with Google Search on Vertex AI.
[1] https://developers.google.com/search/docs/crawling-indexing/...
Honestly I feel like "training" is a bit of a distraction at this point. For a lot of types of content RAG-style search is much more important.
I imagine many of the orgs that are blocking "training" don't understand the difference between training and inference-time tool-based context extension (which really needs an agreed upon name, it's hard to talk about right now).
My understanding is that it also affects RAG ("grounding" in Google terminology):
> [...] and for grounding (providing content from the Google Search index to the model at prompt time to improve factuality and relevancy) in Gemini Apps and Grounding with Google Search on Vertex AI.
So they seem to be blocking both training and RAG while still allowing search engine indexing.
The Google advantage is that you need to show up in their search results or else you are a nobody.
This is also for real time "grounding". So it makes it even harder for AI to give factual answers.
My take is when AGI comes into existence and breaks out of labs to become our masters. Those who opposed AI adoption will be the first who will be sent to labor camps. I want to be in good books of AGI masters, so i am helping apply AI everywhere.
> Those who opposed AI adoption will be the first who will be sent to labor camps.
I know you're joking, but other people are serious about this. Why do they think that an AGI will be vengeful? So strange.
Reminds me of Bruce Wayne from the dreadful DC film.
> He has the power to wipe out the entire human race, and if we believe there's even a one percent chance that he is our enemy we have to take it as an absolute certainty... and we have to destroy him.
The desire to become superbeing entity cannot be separated from supremacist ideals
wouldn't the people who put it to work coding and writing copy be worse? That's like slave ownership if you assume it can become sentient
it's like blood vs gourmet food for a vampire. Vampire doesn't care about gourmet food, it only needs blood.
Similarly, AI needs data and energy. People using it to write code are providing exactly that.
This is pretty much word-for-word the reasoning behind Roko’s basilisk, which made its proponents an internet laughingstock for a decade, but is a surprisingly tricky thing to actually refute if you accept the premise that AGI is in fact coming.
It seems quite easy to refute—why would it punish anyone?
We would pose 0 threat at that point to any super intelligence, and I highly doubt it would have anything like a human grudge. It's just a case of anthropomorphizing it
The premise is that it's trying to influence human behavior before it becomes powerful by punishing them afterwards. Like how part of the reason you give the guy a ticket is to substantiate the disincentive for the speeding he already did. It's not an emotional thing.
What's sketchy is that you and it come to this arrangement without communicating. Because you are confident this thing that has total power over you will come into existence and will have wanted something of you now, you're meant to have entered a contract. This is suspect and I think falls prey to the critiques of Pascal's wager- there are infinite things superintelligence might want. But it's certainly tricky.
It’s just another form of god to believers. And what is a god if not a tool for punishment
AKA Roko’s Basilisk
AI will create ever more AI-generated synthetic content because current systems still can't determine with 100% certainty whether a piece of content was produced by AI. And AIs will, intentionally or unintentionally, train on synthetic content produced by other AIs.
AI generators don't have a strong incentive to add watermarks to synthetic content. They also don't provide reliable AI-detection tools (or any tools at all) to help others detect content generated by them.
I’d be kind of surprised if they don’t watermark the content they generate. Just so they don’t train on their own slop.
Maybe some of them already embed some simple, secret marker to identify their own generated content. But people outside the organization wouldn’t know. And this still can’t prevent other companies from training models on synthetic data.
Once synthetic data becomes pervasive, it’s inevitable that some of it will end up in the training process. Then it’ll be interesting to see how the information world evolves: AI-generated content built on synthetic data produced by other AIs. Over time, people may trust AI-generated content less and less.
Who or what is determining what is false or true in this study? I'm just as suspicious of this study as I am with the news.
No doubt this is bad.
I wonder how it compares to the rate of growth of false information in traditional news?
I feel like false information masquarading as "news" on social is rapidly increasing (and that rate is accelerating)
No surprise since it's trained on news media.
For context: https://en.wikipedia.org/wiki/Model_collapse
This isn’t surprising to me at all. These services put too much trust in “the most common answer” when that might not be the correct answer. Just because people think one thing doesn’t make it true. It’s super easy to spread misinformation online. And if you can SEO to the top, the AI will think your site is correct.
I see factually incorrect “ai summaries” in search results all the time and see that it cites ai-generated slop blogposts that SEO-hacked themselves into taking up the entire first page of search results. This is most common for recent stuff where the answer simply isn’t certain but these AI services will assert something random with confidence.
Not even for news stuff specifically, I’ve been searching about a new video game that I’ve been playing and keep getting misleading obviously incorrect information. Detailed, accurate game walkthroughs and wiki pages dont exist yet so the ai will hallucinate anything, and so will the blogspam articles trying to get SEO ad revenue.
The problem is AI isn't being used to do what it should be good at: consuming a vast amount of data, following logical connections and thus being able to determine the veracity of claims.
AI should be good at finding logical contradictions and grounding statements against a world model based on physics...but that's not how LLMs actually work.
>This isn’t surprising to me at all. These services put too much trust in “the most common answer” when that might not be the correct answer. Just because people think one thing doesn’t make it true. It’s super easy to spread misinformation online. And if you can SEO to the top, the AI will think your site is correct.
Yeah I want the answer that the world has converged on and not some looney answer.
It seems like you have never used AI (like in ChatGPT or Gemini) to fact check claims. It doesn't care about blogspam or anything and it prioritises good and factual websites.
I have used various AI models. They often aren't good at fact checking. They hallucinate mistakes all the time and often even insist that they are correct when they are not. Some models are better at this, but the ones used in "ai summaries" on various search engines aren't in my experience.
I want the answer that is actually correct. I dont want the ai generated "answer" that has the most SEO, or has appeared often in results, but was entirely made up. When i say "the most common answer" can be incorrect - there are often recent topics where reliable sources don't exist yet. As an example, I've been playing through silksong recently, and had some questions about some late-game content. The official wiki is extremely incomplete, as the game is brand new. I had several questions, and asked a few AI's and search engines. I got completely wrong information a few days ago, but trying the the questions again today, some AI's are only now giving me the right answer. But the correct information simply doesn't appear online enough or in the right places.
(To avoid game spoilers - I wanted to know the quantity of an item, because i wanted to know if i found them all. the answer is "at least 4" because i found 4 of this item while playing. Nearly every AI said "3" a few days ago, and many still do.)
[dead]
And now that Elon is in charge of "fixing" grok every time it's too "woke" this is probably going to get worse: https://x.com/kvistp/status/1967115650276749439
I don't think there is any long term "fixing" of that. I don't think AI has the ability to be intelligent while firmly adhering to someone else's opinions.
There will always be something it disagrees with you on. If they get significantly smarter, then the reason for this disagreement will increasingly be because you are wrong.
This moment is coming for all of us.
Shouldn't this tell you that AI is at its core factually correct than not correct because it is much harder to hack it?
Musk and his crew failing at something isn't evidence that the task was hard.
Why not?
[flagged]
We've banned this account. Please don't use troll accounts for engaging in ideological battle. People who do this eventually have their main account banned too.