Visa application is riddled with scams. From the simple website that charges you twice the price to websites that will tell you that you were rejected and then fake your documents to get in with your name.
So they're probably trying to see that you're not one of those web servers, a proxy for them or detect some known C2 channels.
That would be quite clever for an incredibly horrible website. The other day my SO, who is a Turkish citizen, was filling up her visa application and after half an hour of meticulous form filling the system just kick her out. I think the session times out or something. If you haven't created an account or you haven't write down the current application ID everything is lost. In the process she was also directed to a non-.gov website for something during the process, I thought she was getting scammed but no.
It actually makes sense to have a paid service that makes this abomination less painful. Though they work with VFS Global for collecting the applications and relevant documents, the VFS Global itself is an abomination and doesn't help with the handling of the form filling anyway.
Recently EU streamlined the Schengen visa application process for Turkish citizens as those "visa agencies" that are the official agencies and the only way to apply for a visa for many countries don't actually help with anything and are scamming people by selling the "good hours" for the visa appointment on the black market. An agency was dropped for this and the scams by agencies were listed among the reasons to streamline the application process.
Both with US and EU people are losing scholarships etc. due to outrageous wait times that are sometimes are years ahead or there's an issue with the systems handling the applications.
I guess there must be an opportunity there to fix all this together with smaller stuff like handling transliteration and character encodings, I wonder if some of those scam site are not scams and actually help with it. An AI agent can be useful here.
I had to deal with the DS-160 multiple times over the year. I don't think you give justice to how bad this website really is. I have started to notice that these "timeouts" are very random. At the worst times, the session "times out" immediately after login.
These random logouts happens more frequently during certain times of the day and seems to follow a semi-predictable pattern. It is almost certainly tied to system load in some way.
Also, the site's HTML and JavaScript are bloated beyond hope for what should be a fairly simple set of web forms. And itnhas been thisnway since at least 2018 with exactly zero improvements.
One thing a developer sat in DC or SV with a 5G iPhone 16 doesn't realize too, is that if you are visiting these web sites with a phone plan that has a tiny monthly data allowance then this bloat can blow out an entire month in one sitting.
I worked with people on parole that were given free phones to use for job applications, finding their way around etc, and they would only get 3GB data a month. Some of the sites they visited were dropping 250MB of payload on the home page. You'd get some plans that would drop down to 2G, but try using that for Google Maps when you're trying to find a bus to get you across the city.
> You'd get some plans that would drop down to 2G, but try using that for Google Maps when you're trying to find a bus to get you across the city.
Sure, I'll do my best to try it. I'll approximate the throttle by limiting chrome to 128kbps, 500ms delay, and 5% packet loss for fun.
With a fresh incognito session, google responds to "here to 4th street" in 10 seconds, and when I click to open maps it needs just under two minutes to load. Then I can click on the transit option and it needs another 10 seconds to update.
Not too bad for a cold cache. If I do it again with a hot cache it only takes 20 seconds to go through the whole process. And I expect the app to be similar to the hot cache situation. Even with 64kbps I'd expect reasonable results. Do any cell providers throttle worse than that?
I agree with your argument about bloat in general, but google in particular has a lot of good engineering resources and tries to work well on bad connections.
Also I would be in favor of some spectrum licensing rules that say you can't throttle below 1Mbps...
The paper stats on 2G would make it seem like it should work in theory, especially if it's using EDGE or something, but it just consistently fails in the field. You'll get partial renders and then it will jam up. It's super, super frustrating to use. Because it is slow to render people start trying to swipe around the map to make it do something and that just cancels all the async downloads and restarts them.
There are probably a host of other telemetry things going on in your standard $20 Android handset in the background too, eating up all that bandwidth and causing all sorts of bottlenecks.
Agree it would be really nice to have some sane minimum speed.
Well it's not ever going to be actual 2G. It's a throttle. I'm not sure how much worse it could get when you have a reasonably solid signal, but I guess nothing stops network engineers from doing something awful.
> Because it is slow to render people start trying to swipe around the map to make it do something
At a certain point it's the user's fault. And once it gets to the point where you can swipe around, the tile loading should be pretty visible.
And to add more emphasis to the app being usable, I can get driving directions fully offline, then click bus and now it needs one tiny server request to tell me.
Not to defend the US immigration system, but my experience is that this user-hostile behavior (modulo the port scanning lol) is endemic across US government websites - including those that nominally want to serve you, those that are at the state level instead of the federal level (such as the DMV sites), and those that are even internal for use by government employees only.
It's bad enough that in some cases I believe the designers should be threatened with legal penalties.
> user-hostile behavior (modulo the port scanning lol) is endemic across US government websites
I discovered this when it was late at night and I was procrastinating going to bed and I was curious what my estimated Social Security benefit would be at retirement so I tried to log into mySSA and it said the website is closed from like 11 PM to 5 AM or something like that.
I couldn't believe it. I could understand a weekly several-hour maintenance/batch processing window, but DAILY?
That e-filing web site for taxes has never worked for my son because he can’t complete the id.me process, it might be as simple as you are an unperson if you use an android phone or maybe because he’s just started in the workforce he does not have a long history of tax filing and credit history to match up with.
Two years in a row we’ve been able to fill out a 1040 and the NY state equivalent and make a paper submission in less time than it takes to reach an operator on hold.
These identity verification services look like a scam to me. LinkedIn incessantly hassles me to verify with CLEAR and it always fails without a clear error message, either “it just doesn’t work” or my hair has grown too much since I got my driver’s license or it is making me take my glasses off and comparing to a driver’s license photo where I am wearing glasses.
>These identity verification services look like a scam to me.
Even if their intent is to run an 'honest' business, the method of bouncing a user around to god knows how many domains during the process becomes effectively indistinguishable from a compromised service, and the alternative of having each site host their own id verification system screams, HACK US.
I can see users becoming increasingly accustomed to getting out their cards several times during a sign-up and not having the foggiest idea of where their information went to.
It starts to make a lot more sense when you realise there is a huge group in the US actively trying to make the government fail.
It's pretty hard to make a good and user-friendly website when every few years some high-level people try to kneecap you.
These aren't unsolvable problems. The UK, for example, had invested a lot of time and effort into making their websites user-friendly. In most countries filing taxes online is something you can do during your lunch break - without paying the Turbotax maffia. Driver's license? You can order that online, and make an appointment for a 15-minute window to pick it up.
If interacting with the government is painful, it is almost always because someone benefits from it being painful.
Gaming of the procurement system. The websites are all written by big consulting outfits. Not to mention the disaster that is big corporate IT projects combined with government rules.
Obama had the Digital Service (that Trump shut down) which paid higher salaries. Those folks were sharp and everything they touched was actually decent.
As I noted this is not unique to government. Large corporate projects at the Fortune 500 are often the same sort of consultant-driven crap.
It wasn't temporarily retasked, it was reorganized and permanently repurposed and renamed the US DOGE Service, and then within that reorganized service, a subordinate temporary organization was created called the US DOGE Service Temporary Organization that was scheduled to sunset not later than July 4, 2026. (All but 65 of USDS's pre-reorg employees were also fired as part of the reorg, and 21 of those remaining 65 employees did a mass resignation.)
If you visit their website, you will notice that except for historical documents, there is no full name branding at all; mostly only the logo and the occasional "USDS", when prior to the reorg (as can be seen on the Wayback machine) the original full name was prominent.
This. The website for buying treasury products is straight out of the year 2002. The login is so bad I would never consider buying them there - the service fee charged by brokerages is absolutely worth it in this case.
Which brokerages charge fees for purchasing US Treasuries? Schwab definitely doesn't.
Really the only reason you need TreasuryDirect is for buying Series I bonds (and maybe a few other niche Treasury products), which are not available through brokerages.
Back when interest rates peaked around that period I bought a huge number of I bonds which were a great investment —- got fired by my broker because I interrupted a sales presentation with “why don’t I just buy I bonds?”
The web front ends are awful, but the back ends are even worse. The backlogs for some of these applications is insane. I was at a US embassy one time and got talking to a girl who had just had her application approved after an 18 year wait.
I don’t know if you’re US-based or not but in the US, government work has the stigma of attracting the bottom of the barrel. It is nearly impossible to get fired for performance reasons. Combine low pay and high job security, and you’re not going to attract the most innovative, motivated, or competent people.
Early in my career, I was warned that if I took a job with the state of California, I’d be stuck there for my whole career. I’d be unhirable in the private sector.
Not so much after DOGE fired entire departments for dubious reasons.
I don't know why anyone would work for the federal government now - pay still sucks, and job security has been demonstrated to no longer be guaranteed.
Recent events isn't going to change decades of stigma and reputation. People aren't saying, "Oh cool, they purged the low performers. I'll go work for the government!"
From 2014 until it was, in effect, obliterated by DOGE actions this year there was the "United States Digital Service", a crack team of programmers, a sort of skunkworks who worked to improve U.S. government websites of departments that wanted the help. So it seems to be partisan to want good websites, but there are countless people involved in politics with many agendas.
The VISA appointment scheduling site rate limits to a ridiculous degree these days. As in refresh your page within 10seconds and get a 429 error.
That's probably because of the fact that the appointments are near impossible to get, they only allow booking a few months out and it's always completely booked. So everyone was refreshing (or if clever botting) to get an appointment slot.
As I wrote elsewhere; they subcontract the bot protection to F5, an external company that I see for some reason a lot on old/horrible banking websites.
Hey, this is actually something I have a keen interest in as I'm fighting my government (as an MP) to drop those scammers where possible. Do you have any media links to send me about them selling the "good hours" on the black market?
Even if the US has a horrible visa system – as I can attest, despite only having to do it every 5 years – the EU countries could benefit from attracting talent by being more welcoming. So that is part of my mission as an MP and tech-entrepreneur. Any help and pointers is welcome.
Hi, about the Schengen visa situation in Turkey you can find articles like these that describe how the appointments are on the black market(In Turkish but I'm sure AI will do good job translating):
On the social media the anecdotes differ but some say they were able to get the visa appointments bots, others say it was agency personel selling it to them under the table. Maybe its really the agency personel, or maybe it's people running bots to snap appointments and sell those pretending to be from the agency - can't know for sure but there are multiple services where people purchase appointments unofficially.
In general the news situation in Turkey isn't very good as with the law enforcement but as you can see even BBC took notice.
Generally speaking, these visa agencies are very unfriendly and unreachable. They seem to just collect the money, provide no personalized help at all. My GF had some questions about her US visa application, we were not able to reach VFS Global. The phone numbers provided don't work, it's not even like taking long to speak with a human, the phone just gives you calling error.
She previously used the same company for her Schengen visa for a company event in Paris, of course unreachable again and no appointments available. Because she works at a French corporation, she was able to ask a high ranking French person in the company who has a contact with the French embassy and they arranged the appointment shortly.
Whenever I'm filling a long form on an official website, I feel like I'm racing against an invisible clock because of this session time out thing that happened to me countless times.
I had this problem too last year. I found, at the time, it was the website was poorly managing the session in some browsers causing the timeout countdown to not be reset on activity. I had to find a windows computer and use microsoft edge I think (maybe it was chrome). But no browser on my mac would not have that issue.
> According to Ablakwa, a locally recruited staff member and "collaborators" were allegedly involved in a "fraudulent" scheme whereby they extracted money from visa and passport applicants.
> It is alleged that the scheme consisted of creating an unauthorised link on the embassy's website to redirect visa and passport applicants to a private firm where they were "charged extra for multiple services" without the knowledge of the foreign ministry.
> Ablakwa added that the staff member "kept the entire proceeds" in their private account, and that the scheme had been going on for five years.
> Applicants seeking visas were charged unapproved fees ranging from almost $30 (£22) to $60 by the private firm.
The hard truth of it all is that both the US and (partially) the EU don’t want to make this easier because seeing as wanting “outside” people is now a political liability. You may want to adjust your expectations around that.
Turkish tourist are desired, Turks love spending money on restaurants and activities especially since the prices in Turkey have become more expensive than most of the EU. Greeks even introduced special non-Schengen on-arrival visa valid on the Greek islands especially for the Turks. Besides that, EU has "green passport" exception for the Turkish nationals, where they can travel visa-free on this kind of passport that is provided to individuals that meet certain criteria and millions of such passports were issued.
The rejection rates are also not bad and EU has a "return agreement" with Turkey, which is designed to keep the middle eastern refugees in Turkey(essentially, if you come from Turkey EU can send you back to Turkey right away ).
Crime rates for Turks show up among the lowest ones, unlike others from the region. So I don't think that EU is trying to reduce visas for Turks.
I am EU citizen, I happen to know the Turkish perspective only because spent some years in Turkey and in fact it is the Turkish perspective that that EU doesn't want them and intentionally makes things harder but the moment you look at what's actually going on you see that this is not the case, just a Turkish fantasy about the "evil West and snobby Europeans". Considering that last year 50K Turks applied for asylum in EU and another 100K overstayed their visa, IMHO EU can be considered pretty generous actually with only 15% rejection rate since Turkey is the 2nd country with most applications after China.
The "waterfall model" is a toxic way of thinking that pervades corporate management. Simplistic minds can't fathom any states other than "done" or "not done". Corporations are determined to crush the human soul. That is why it's not a progressive series of forms, saving your progress all along.
More-or-less agreed about the waterfall model, but you can't blame horrific US government website performance on "corporations" or "corporate management". This is precisely the sort of thing that would get you fired in any real-world corporation that wants to survive, and it's precisely the fact that you can't get fired by the federal government that allows this sort of thing to continue.
Another data point - 5he Indian visa system is similar. The official website ending in .gov.in, which is hard to find, offers a visa for $10 and minimal hassle. The scam websites, with better SEO sell the same shit for $80. They’re just proxying your application to the real website and pocketing the difference.
It would be good if the Indian government could block the scammers but I guess it’s a lower priority for the moment.
Not sure if this is the case for India, but I've experienced similar situations for other countries, but the 'scam websites' actually provided a real service - if you needed some ultra-urgent processing (like you only realized you needed a visa to this country before boarding a flight, once you were already at the airport check-in...) they were able to provide 30 minute approval, whereas the official site's accelerated processing was 24 hours.
So obviously the only way they could to this is with government contacts meaning the government themselves could already do it, but a lot of immigration stuff everywhere is full of people taking kickbacks.
This was the case with Ghana. The Embassy in the US had an unofficial offical partnership with an expediter scam (charge more for faster shipping, looks very official). They fired the whole visa staff when it finally came to light. Probably because someone forgot to let their manager's manager in on the scam.
I found the real website, but the application never went through, always some issue. My boss told me which service to use and everything just worked. (I could expense that service so cost didn't bother me)
My understanding is that India visa processing improved quite a bit. Back when I was speaking internationally quite a bit, I actually had to cancel trips to India on two separate occasions because of delays in getting visas. (Once was under the old visa system and the other was because of delays in switching to a new system. Both times were through a visa expediting service.)
Just a guess, but maybe a typical bot has a webserver, ssh server, some other servers running on the same machine, whereas a typical Visa applicant doesn't.
Or a browser automation server (Marionette/CDP). I seem to remember watching a presentation where it was mentioned you could detect them this way, <s>but I don't remember where or what it was called.</s> this one: https://www.youtube.com/watch?v=4nZD6ee2Xo8 (WHY2025: Stealth Web Scraping Techniques for OSINT)
Huh, how do you imagine that would work? This "scan" is happening inside client-side javascript, delivering the file through a proxy wouldn't "detect" anything about the proxy.
I imagine it may not be a proxy in the true sense, but a headless browser that's "proxying" the application process rather than the network traffic itself.
it's riddled with scams, and thinking any of this will detect any of the things you mention is very foolish, native and show a total lack of understanding of the scams. of you think using a proxy is essential for visa scam, i would even know where to begin to correct you.
it's one hundred per cent clueless privacy invasion. they are probably also opening ports via other means and using that for side channel ID like Facebook does.
just like any other documentation scam, the only weak point is on the "last mile" that's why you will always have a human interviewer.
the visa process is abusive and unpractical because people will work around any hurdle and their kpi will never be affected no matter how crappy they manage to make to whole process. or how many doge kids implement useless privacy invasion tech just because.
It's coming from a F5 script, which is a company that sells anti-bot protection amid other things. (It's coming from obfuscated script at /TSPD, which is a F5 thing.)
Checking out the initial request on github for this feature I wonder why is this necessary? What access to the local network does the browser provide, or need to provide, and why isn't this something developers are more concerned about? I had a feeling this was possible as I see lots of mdns requests when I connect to certain things running sockets.
There are certainly use cases, but whether they’re warranted is a good question.
One popular router maker offers a ‘magic URL’ (domain name) that scans your network for the gateway management page, and redirects. It’s not necessary, but it certainly helps novice users. Having worked in IT support,
I’ve also purchased hardware devices that have a web management UI; which connects directly instead of proxying through a cloud.
Ultimately this is probably one thing that should be behind a permission request (like webcam access), but it’s not a feature without value.
Internal apps on non-private IP addresses occasionally use this. There is a standard called Private Network Access[1] that requires these requests to have preflights like CORS requests. Only Chrome has implemented it so far.
Why though? What is the use case that demands this? It'd better be a real pressing need because the security risks are immense and obvious. This is a backdoor to every network firewall.
It’s more that it wasn’t prevented back when the web was first coming together, because security wasn’t on almost anyone’s minds at all. There wasn’t a hole added at some point; it’s just that browsers didn’t specifically block domains that resolve to public IPs from accessing domains that resolve to private IPs.
Realistically, it’s a backdoor to every network firewall that has existed for the entire era in which browsers were used in “secured” internal networks also connected to the internet. Everyone has either designed with it in mind, or gotten lucky that nobody tried to use it on them for like 30 years. I think it’s good to put away this footgun, but there’s no useful blame to assign here.
Requests that need a CORS preflight will fail with any browser from the last 20 years, yes. The private IP addresses are not any more vulnerable than `www.google.com` is from `www.notgoogle.com` for cross-origin policy (subdomain-sensitive policies have a small extra vulnerability). But you’re right that doing this kind of thing without nefarious intent is an insane edge case and it should be opt-in. People spray `Access-Control-Allow-Origin: *` like it’s DDT in the 50s and half ass security in general when it’s on an intranet, so an extra guardrail is still worth it.
Home Assistant has a well-known public name that opens your local instance. On first access, you need to give it the name or ip of your instance, which is saved in browser storage. This supports deep links into your config from forum posts.
My mum also had a shitty D-Link wifi mesh device, which was packaged as an appliance. I cannot speak lowly enough about that garbage device, but then, I am not really the target market. iirc it had something similar; a public dns name for local appliance mgmt.
How is that the same thing? That is a DNS entry that resolves to an internal IP. That lets a user explicitly type a domain and get something internal. That wouldn’t allow cnn.com to ports scan my fridge.
… or you can instead phase out those browsers who try to force blocker restrictions i.e. spyware on you (e.g. chrome and such), and use one of the browsers where you can use the full-featured (not "lite") uBlock Origin instead, e.g. Firefox.
I disagree with their politics, I'm concerned by the multiple privacy incidents, and I generally refuse to support them until they refocus on Firefox instead of all the other stuff they're doing.
If they worked only on Firefox, I'd have nothing against them. As it stands, I can't even donate to Firefox if I want to.
then again, if the makers of one big browser (and via there also the derived browsers) start force-shoving spyware upon you (by restricting blockers), it comes down to a decision of how you set your priorities. Personally, It's a clear cut red line, but you do you.
I'm using uMatrix and it blocks by default all connections outside the requested site and parent domains. For example, if I request https://mail.yahoo.com, connections to yimg.com are blocked. I need to manually allow each CDN for each website, so this attack/profiling won't work.
Using uMatrix was very annoying at first, most websites are broken without their CDNs, but after a few months or so, the whitelist grew and it contains 90% of websites I visit.
On my system https://ceac.state.gov/genniv/ tries to connect to captcha.com, google-analytics, googletagmanager, 127.0.0.1 and "burp" (a local hostname that doesn't exist in my network). Interestigly, the browser console doesn't list connection attempts to localhost or burp. If I allow 127.0.0.1 and "tcpdump -i lo", I see connections to port 8888, which isn't open.
How does uMatrix handle the Facebook tracking pixel, or the replacement which is the Conversions API Gateway?
This is a container that FB gives you to host that lives under your domain (it can be your main domain) that slurps up user data and sends it to Facebook from the server side. You embed some JS in your website, and they hoover up the data.
It doesn't handle it. Anyway, there's no way to know what a website does on the server site. Even a completely static website could be sending the server logs somewhere.
There are options to not load JS, images, XMLHttpRequests, frames, cookies, for each site, but it doesn't list individual files.
> On my system https://ceac.state.gov/genniv/ tries to connect to captcha.com, google-analytics, googletagmanager, 127.0.0.1 and "burp" (a local hostname that doesn't exist in my network).
I didn't knew community/public whitelists exist, nor any browser extension that uses whitelists and blocks all other connections by default, like uMatrix does. Do you have any examples?
The requests are not made, because some operating systems prevent this.
If you're on OSX, the permission to "discover on the local network" prevents it from happening ( System Settings -> Privacy & Security -> Local Network -> yourbrowser )
Could also be 'network' permissions on firefox ( Go to Settings > Privacy & Security > Permissions ) which is on a per site level, but iirc that could be set site-wide at some point.
The other browsers likely have similar configs, but this is what I have found.
It seems like they only make the localhost requests on your first visit. If you open devtools in incognito mode (or just clear the cookies) before accessing https://ceac.state.gov/genniv/ you should see those 127.0.0.1 attempts as ERR_CONNECTION_REFUSED in the network tab.
Somewhat more worryingly, Little Snitch doesn't report them at all, though that might just be because they were already blocked at the browser.
(There is some language online suggesting PNA has not actually shipped, but I experienced it myself in stable Chrome several years ago, so I am unsure of the current state).
Firefox doesn't implement either approach -- I assume this is indicative of their lack of development resources.
> Firefox doesn't implement either approach -- I assume this is indicative of their lack of development resources.
Since ublock had this as a feature for a long time, I'm sure they are aware of it. Unlike other non funded oss projects, Firefox can't and shouldn't shield themselves with this lack of development resource excuse. They have millions.
A trillion dollar company (that loves huge vanity projects) gave up on maintaining a browser because it was too much work and just ship a Chrome fork now. I won’t defend Mozilla’s allocation of their resources, but even if they put it all into the “right” Firefox features the web platform is too complex and too much of a moving target for a company with mere centi-million revenues.
To be honest they weren't trying to build a better browser. Atleast not anymore, earlier edge was nice. They just wanted more data for ads / money. Going the chrome way was more profitable for them.
Microsoft? Were they firing as many senior folks as possible in 2018 when they announced they would give up on EdgeHTML and Chakra? Or in early 2020 when it actually came together? That’s not my recollection of the FAANG-ish job market at the time these decisions were made.
If you meant Mozilla, they’re a total indefensible trashfire for sure. But I’m not convinced they could have succeeded with their resources.
That extension has "Access your data for all websites" ... I really don't get how anyone can give that permission to anyone that isn't well known (a company with a lot on the line) or a person famous for their work (the uBO dev) who has stated he will never sell to anyone or do bad things.
"Hacks and Hops" doesn't even have a valid home page. The extension links to https://g666gle.me/ which does not exist. The domain name itself does not want to make me give access to all my data for all websites to them.
As nice as this extension seems, I would ever in a million years install it.
Unfortunately this level of incoherence is almost universal on HN and similar forums. You'd have to be completely out of your mind to install this extension, but people for some reason believe they can install privacy. They got whipped into fearing nebulous online actors so much that they'll download FSB rootkits dressed as VPNs. The minimal set of actions a rational person would take after realizing they've been tricked into installing this extension is setting their entire PC on fire and then running it over with their car, while moving all of their bank accounts to new accounts, in person, and changing all of their passwords using a brand new device.
Many sites do it .Included in many standard device fingerprinting / anti anonymity SAAS. Ebay facebook etc all do this ! But it looks this is first party to prevent the adblocking of them
1MB of obfuscated fingerprinting + portscan + Webgl . But oddity this one is trying to find burp suite specific route's.
The company I work for has a legitimate service that runs on the loopback (it provides our web apps APIs for some device integration) hopefully its just as simple as the user accepting the prompt else we'll be drowning in support.
We had to go the path of the local service because they killed NPAPI. I've been thinking about using web serial as an alternative but Firefox doesn't support it.
That being said, I think this is an overall win, hopefully Firefox implements it in a consistent manner as well.
This how many of them work for transporting vs traditional old way of registering url scheme and requiring user interacts --- Discord, Blizzard net, Riot Client ... all localhost listener's that can interact
It would be the job of the operating system to give or take away the ability of your browser to access your local network. But you can run your browser in a container/vm and disable localhost. (And use a separate browser for localhost only if you need it.)
The "port scan" just seems to be a local connection to 127.0.0.1:8888. I don't know what purpose it serves on this page, but our government websites often use this technique to communicate with native software for digitally signing documents.
Could also be incompetence :D until I fixed it, deploying from my local machine rather than CD resulted in one of the baked in URLs being localhost rather than the public host on the project I'm working on now. Their local development server might just be at port 8888. Wouldn't surprise me.
I looked at the website again and noticed that the request paths looked odd, one of them being `/400_random_url_with_numbers_403`. I googled that and it looks like it's part of a client-side bot detection script that's testing something, the explanation isn't very informative.
> These requests are caused by the bot profile to test the different browser capabilities.
> 'http://127.0.0.1:xxxx' request is a call to the localhost/client machine, which is normal when trying to protect assets like end-server using ant-bot defense. It does not have any impact regarding application page load.
This is a common pattern for connecting to smart cards / hardware security devices. Probably a service or hardware that’s run on official CBP machines that should be disabled for prod, but forgot.
Just a little side note - in this context, it makes sense if the website tries to connect to a local port because you might be running a card reader(ie. terminal). This is how it works with some(all?) EU countries that have a chip in their ID cards, or even vehicle registration cards, which you can use to access sensitive information or perform certain administrative tasks on government websites.
Although, from personal experience, it used to require java and it worked only on internet explorer and since it has been retired and replaced with chromium, i am not sure what is the way to make it work nowadays, as i have not been able to figure out to use it when i needed the last time.
It requires installing a local service that bridges between the browser and the smartcard driver (what Java applets did in earlier years). The web app then communicates with the service via requests on localhost. The card-specific driver and bridge service are often bundled together for installation.
I've had it before where it asked me to use an iPhone/Android app which can read the passport's NFC chip. I guess that's the modern replacement for IE/Java.
Yes. Facebook was using this trick on Android. Meta's android apps would host a server on localhost, and their sites would communicate with this local server to pass tracking information that would otherwise be blocked by all browser protection methods on Android. I guess it is still fingerprinting, but at the most extreme end.
Mostly it's great for tracking although I'm sure it could also be used to exfiltrate data (e.g. if the user is running something sensitive on localhost).
CORS doesn’t prevent requests (i.e. GET requests from IMG tags, or XHR preflight requests), it only prevents web apps from processing the response if the responding server doesn’t agree. And a simple GET or even OPTIONS request can be enough to exploit vulnerabilities in routers and other local devices.
It seems to be part of some "bot defense" product by these F5 people, to "test the different browser capabilities". I doubt it's intended to hit a real endpoint on any system.
Capturing forensic artifacts of the local network allows a building a bridge strategy for identifying fraudulent networks without requiring knowledge of the path taken from destination to recipient. Other local devices do this and send the network map during a phone home, allowing comparison to a source of truth that is tied almost directly to the person, or group of people.
There is also a lot of fingerprintable material within such a port scan from clock skew, TCP ISN, and a few other areas.
You can sieve this quite easily with this available, thanks to Roku's, Phone's, and other things doing this while just sitting locally in a shared collision domain (a digital soldier quartered in every home).
The metadata node graph of devices locally acts as a unique fingerprint once in RFC1918 space, technically not unique but close enough.
As far as I understand it, it is supposed to be a scan done by the browser on the user's computer, not an external scan, which a browser extension wouldn't be able to detect.
I see. So the website would try to access private IP adresses (RFC 1918) by having elements like <iframe src="http://10.0.0.1"> in the web site and then the web site would check if the iframe was loaded successfully?
Judging just from the screenshots, it seems it blocks websites from accessing 127.0.0.1 get requests. Not a port scan to the outside, more of what do you have running on the local machine inside your network.
If would be interesting to see what happens on OpenBSD. With pledge(2) and unveil(2) in Firefox, I wonder what it would see. I expect it would see nothing.
I will give it a try and see what happens and if I see anything I will add it here.
Have you double-checked whether the IP isn't shared among multiple website domains? That's quite a classic with IP based filtering with hosters like GCP...
It looks useful and looks good, there's minimal unneeded whitespace and I'm glad it looks as it does. We'd be better off if the entire web switched to a style like this.
Looking like something from the 90s would be a feature, not a bug.
In the 90s and early 00s, we did tons of user-testing and feedback collection. We threw all that research away to create UX's that are minimal and "sleek". Tons of unnecessary whitespace and the concept of "Discovery" just thrown into the dumpster. Skeuomorphism was one of the greatest features of 90s-00s software, ironically thrown away as computers got faster and were able to handle the graphics better.
Most likely some "antivirus" bs. Probably harmless. Fun fact. Most browsers allow by default GET access to web resources on localhost and LAN. Been used for exploits since last century.
I can one better (worse): A state-run website that my sister frequents for her job requires Internet Explorer. Seriously. I installed a Chrome extension that modifies her user-agent header to IE, and it works fine. Easy work-around, but totally lame.
I might create a login for a porn site so that I can have some favorite videos bookmarked and it can figure out the type of material I like. That doesn't mean I want my history saved locally.
Not contradictory at all. These days private browsing for most people just means (1) don't save the browsing history and (2) log me out of all websites temporarily.
But as the other post notes, it goes further (than, for example, Chrome Incognito) in ways that can break sites. Incognito means exactly what you said, while Safari Private Browsing means more.
Visa application is riddled with scams. From the simple website that charges you twice the price to websites that will tell you that you were rejected and then fake your documents to get in with your name. So they're probably trying to see that you're not one of those web servers, a proxy for them or detect some known C2 channels.
That would be quite clever for an incredibly horrible website. The other day my SO, who is a Turkish citizen, was filling up her visa application and after half an hour of meticulous form filling the system just kick her out. I think the session times out or something. If you haven't created an account or you haven't write down the current application ID everything is lost. In the process she was also directed to a non-.gov website for something during the process, I thought she was getting scammed but no.
It actually makes sense to have a paid service that makes this abomination less painful. Though they work with VFS Global for collecting the applications and relevant documents, the VFS Global itself is an abomination and doesn't help with the handling of the form filling anyway.
Recently EU streamlined the Schengen visa application process for Turkish citizens as those "visa agencies" that are the official agencies and the only way to apply for a visa for many countries don't actually help with anything and are scamming people by selling the "good hours" for the visa appointment on the black market. An agency was dropped for this and the scams by agencies were listed among the reasons to streamline the application process.
Both with US and EU people are losing scholarships etc. due to outrageous wait times that are sometimes are years ahead or there's an issue with the systems handling the applications.
I guess there must be an opportunity there to fix all this together with smaller stuff like handling transliteration and character encodings, I wonder if some of those scam site are not scams and actually help with it. An AI agent can be useful here.
I had to deal with the DS-160 multiple times over the year. I don't think you give justice to how bad this website really is. I have started to notice that these "timeouts" are very random. At the worst times, the session "times out" immediately after login.
These random logouts happens more frequently during certain times of the day and seems to follow a semi-predictable pattern. It is almost certainly tied to system load in some way.
Also, the site's HTML and JavaScript are bloated beyond hope for what should be a fairly simple set of web forms. And itnhas been thisnway since at least 2018 with exactly zero improvements.
One thing a developer sat in DC or SV with a 5G iPhone 16 doesn't realize too, is that if you are visiting these web sites with a phone plan that has a tiny monthly data allowance then this bloat can blow out an entire month in one sitting.
I worked with people on parole that were given free phones to use for job applications, finding their way around etc, and they would only get 3GB data a month. Some of the sites they visited were dropping 250MB of payload on the home page. You'd get some plans that would drop down to 2G, but try using that for Google Maps when you're trying to find a bus to get you across the city.
> You'd get some plans that would drop down to 2G, but try using that for Google Maps when you're trying to find a bus to get you across the city.
Sure, I'll do my best to try it. I'll approximate the throttle by limiting chrome to 128kbps, 500ms delay, and 5% packet loss for fun.
With a fresh incognito session, google responds to "here to 4th street" in 10 seconds, and when I click to open maps it needs just under two minutes to load. Then I can click on the transit option and it needs another 10 seconds to update.
Not too bad for a cold cache. If I do it again with a hot cache it only takes 20 seconds to go through the whole process. And I expect the app to be similar to the hot cache situation. Even with 64kbps I'd expect reasonable results. Do any cell providers throttle worse than that?
I agree with your argument about bloat in general, but google in particular has a lot of good engineering resources and tries to work well on bad connections.
Also I would be in favor of some spectrum licensing rules that say you can't throttle below 1Mbps...
The paper stats on 2G would make it seem like it should work in theory, especially if it's using EDGE or something, but it just consistently fails in the field. You'll get partial renders and then it will jam up. It's super, super frustrating to use. Because it is slow to render people start trying to swipe around the map to make it do something and that just cancels all the async downloads and restarts them.
There are probably a host of other telemetry things going on in your standard $20 Android handset in the background too, eating up all that bandwidth and causing all sorts of bottlenecks.
Agree it would be really nice to have some sane minimum speed.
Well it's not ever going to be actual 2G. It's a throttle. I'm not sure how much worse it could get when you have a reasonably solid signal, but I guess nothing stops network engineers from doing something awful.
> Because it is slow to render people start trying to swipe around the map to make it do something
At a certain point it's the user's fault. And once it gets to the point where you can swipe around, the tile loading should be pretty visible.
And to add more emphasis to the app being usable, I can get driving directions fully offline, then click bus and now it needs one tiny server request to tell me.
You might be making the assumption that the US wants to make the process easier.
Not to defend the US immigration system, but my experience is that this user-hostile behavior (modulo the port scanning lol) is endemic across US government websites - including those that nominally want to serve you, those that are at the state level instead of the federal level (such as the DMV sites), and those that are even internal for use by government employees only.
It's bad enough that in some cases I believe the designers should be threatened with legal penalties.
> user-hostile behavior (modulo the port scanning lol) is endemic across US government websites
I discovered this when it was late at night and I was procrastinating going to bed and I was curious what my estimated Social Security benefit would be at retirement so I tried to log into mySSA and it said the website is closed from like 11 PM to 5 AM or something like that.
I couldn't believe it. I could understand a weekly several-hour maintenance/batch processing window, but DAILY?
That e-filing web site for taxes has never worked for my son because he can’t complete the id.me process, it might be as simple as you are an unperson if you use an android phone or maybe because he’s just started in the workforce he does not have a long history of tax filing and credit history to match up with.
Two years in a row we’ve been able to fill out a 1040 and the NY state equivalent and make a paper submission in less time than it takes to reach an operator on hold.
These identity verification services look like a scam to me. LinkedIn incessantly hassles me to verify with CLEAR and it always fails without a clear error message, either “it just doesn’t work” or my hair has grown too much since I got my driver’s license or it is making me take my glasses off and comparing to a driver’s license photo where I am wearing glasses.
>These identity verification services look like a scam to me.
Even if their intent is to run an 'honest' business, the method of bouncing a user around to god knows how many domains during the process becomes effectively indistinguishable from a compromised service, and the alternative of having each site host their own id verification system screams, HACK US. I can see users becoming increasingly accustomed to getting out their cards several times during a sign-up and not having the foggiest idea of where their information went to.
The id.me process is absolutely horrific.
I'm not sure the word horrific is up to carrying the weight of just how bad id.me is. Still, a great effort.
It starts to make a lot more sense when you realise there is a huge group in the US actively trying to make the government fail. It's pretty hard to make a good and user-friendly website when every few years some high-level people try to kneecap you.
These aren't unsolvable problems. The UK, for example, had invested a lot of time and effort into making their websites user-friendly. In most countries filing taxes online is something you can do during your lunch break - without paying the Turbotax maffia. Driver's license? You can order that online, and make an appointment for a 15-minute window to pick it up.
If interacting with the government is painful, it is almost always because someone benefits from it being painful.
Gaming of the procurement system. The websites are all written by big consulting outfits. Not to mention the disaster that is big corporate IT projects combined with government rules.
Obama had the Digital Service (that Trump shut down) which paid higher salaries. Those folks were sharp and everything they touched was actually decent.
As I noted this is not unique to government. Large corporate projects at the Fortune 500 are often the same sort of consultant-driven crap.
Digital Service didn't shut down, it just temporarily got retasked to DOGE.
It wasn't temporarily retasked, it was reorganized and permanently repurposed and renamed the US DOGE Service, and then within that reorganized service, a subordinate temporary organization was created called the US DOGE Service Temporary Organization that was scheduled to sunset not later than July 4, 2026. (All but 65 of USDS's pre-reorg employees were also fired as part of the reorg, and 21 of those remaining 65 employees did a mass resignation.)
If you visit their website, you will notice that except for historical documents, there is no full name branding at all; mostly only the logo and the occasional "USDS", when prior to the reorg (as can be seen on the Wayback machine) the original full name was prominent.
This. The website for buying treasury products is straight out of the year 2002. The login is so bad I would never consider buying them there - the service fee charged by brokerages is absolutely worth it in this case.
Which brokerages charge fees for purchasing US Treasuries? Schwab definitely doesn't.
Really the only reason you need TreasuryDirect is for buying Series I bonds (and maybe a few other niche Treasury products), which are not available through brokerages.
Schwab folds their fees into their bid/ask spread, they're not doing it for free.
Back when interest rates peaked around that period I bought a huge number of I bonds which were a great investment —- got fired by my broker because I interrupted a sales presentation with “why don’t I just buy I bonds?”
Back then I thought Treasury Direct was great.
Makes it obvious which lobby has a hand in this, doesn't it?
That would be an abysmally poor assumption currently.
The purpose of a system is what it does.
I'd invoke Hanlon's razor, but in this case, it's certainly both malice and stupidity...
They are so frequently intertwined
The web front ends are awful, but the back ends are even worse. The backlogs for some of these applications is insane. I was at a US embassy one time and got talking to a girl who had just had her application approved after an 18 year wait.
18 year wait for approval or 18 year wait for family sponsored immigrant visa? Because from some countries those do have 18 year backlogs.
I believe it was the latter, if memory serves correct.
You use the same system for Business visas. Hard to imagine US wouldn’t want those as easy as possible.
You don't have a good enough imagination for how stupid our current leadership really is.
I guarantee the visa system was created before the current administration.
I don’t know if you’re US-based or not but in the US, government work has the stigma of attracting the bottom of the barrel. It is nearly impossible to get fired for performance reasons. Combine low pay and high job security, and you’re not going to attract the most innovative, motivated, or competent people.
Early in my career, I was warned that if I took a job with the state of California, I’d be stuck there for my whole career. I’d be unhirable in the private sector.
> high job security
Not so much after DOGE fired entire departments for dubious reasons.
I don't know why anyone would work for the federal government now - pay still sucks, and job security has been demonstrated to no longer be guaranteed.
Recent events isn't going to change decades of stigma and reputation. People aren't saying, "Oh cool, they purged the low performers. I'll go work for the government!"
During 8 years of Obama and 4 years of Biden, none of this was different or better. Perhaps this isn't a partisan political issue.
From 2014 until it was, in effect, obliterated by DOGE actions this year there was the "United States Digital Service", a crack team of programmers, a sort of skunkworks who worked to improve U.S. government websites of departments that wanted the help. So it seems to be partisan to want good websites, but there are countless people involved in politics with many agendas.
Hard to imagine that the US wouldn't be as paranoid, self-sabotaging, and bureaucratically inept as possible? </sarcasm>
As a US citizen, I feel it’s opposite. Hard to imagine they’d want anything related to visas to be easy.
If there is any conclusion to be drawn here, it is that the United States doesn't want foreigners in their land (for tourism or otherwise).
I'm not sure I see the upside of moving to a nation knowing that its citizens actively despise my existence.
The VISA appointment scheduling site rate limits to a ridiculous degree these days. As in refresh your page within 10seconds and get a 429 error.
That's probably because of the fact that the appointments are near impossible to get, they only allow booking a few months out and it's always completely booked. So everyone was refreshing (or if clever botting) to get an appointment slot.
As I wrote elsewhere; they subcontract the bot protection to F5, an external company that I see for some reason a lot on old/horrible banking websites.
F5 is huge in enterprise and academia for firewall/VPN/load-balancer services
Hey, this is actually something I have a keen interest in as I'm fighting my government (as an MP) to drop those scammers where possible. Do you have any media links to send me about them selling the "good hours" on the black market?
Even if the US has a horrible visa system – as I can attest, despite only having to do it every 5 years – the EU countries could benefit from attracting talent by being more welcoming. So that is part of my mission as an MP and tech-entrepreneur. Any help and pointers is welcome.
Hi, about the Schengen visa situation in Turkey you can find articles like these that describe how the appointments are on the black market(In Turkish but I'm sure AI will do good job translating):
https://www.bbc.com/turkce/articles/cz5r2l43kn2o
https://medyascope.tv/2024/01/22/vize-sorunu-kontrolden-cikt...
On the social media the anecdotes differ but some say they were able to get the visa appointments bots, others say it was agency personel selling it to them under the table. Maybe its really the agency personel, or maybe it's people running bots to snap appointments and sell those pretending to be from the agency - can't know for sure but there are multiple services where people purchase appointments unofficially.
In general the news situation in Turkey isn't very good as with the law enforcement but as you can see even BBC took notice.
Generally speaking, these visa agencies are very unfriendly and unreachable. They seem to just collect the money, provide no personalized help at all. My GF had some questions about her US visa application, we were not able to reach VFS Global. The phone numbers provided don't work, it's not even like taking long to speak with a human, the phone just gives you calling error.
She previously used the same company for her Schengen visa for a company event in Paris, of course unreachable again and no appointments available. Because she works at a French corporation, she was able to ask a high ranking French person in the company who has a contact with the French embassy and they arranged the appointment shortly.
Whenever I'm filling a long form on an official website, I feel like I'm racing against an invisible clock because of this session time out thing that happened to me countless times.
I had this problem too last year. I found, at the time, it was the website was poorly managing the session in some browsers causing the timeout countdown to not be reset on activity. I had to find a windows computer and use microsoft edge I think (maybe it was chrome). But no browser on my mac would not have that issue.
> In the process she was also directed to a non-.gov website for something during the process, I thought she was getting scammed but no.
No clue if this specific instance if scam but such scams have indeed been done before
https://www.bbc.com/news/articles/cdr56vl410go
> According to Ablakwa, a locally recruited staff member and "collaborators" were allegedly involved in a "fraudulent" scheme whereby they extracted money from visa and passport applicants.
> It is alleged that the scheme consisted of creating an unauthorised link on the embassy's website to redirect visa and passport applicants to a private firm where they were "charged extra for multiple services" without the knowledge of the foreign ministry.
> Ablakwa added that the staff member "kept the entire proceeds" in their private account, and that the scheme had been going on for five years.
> Applicants seeking visas were charged unapproved fees ranging from almost $30 (£22) to $60 by the private firm.
The hard truth of it all is that both the US and (partially) the EU don’t want to make this easier because seeing as wanting “outside” people is now a political liability. You may want to adjust your expectations around that.
Turkish tourist are desired, Turks love spending money on restaurants and activities especially since the prices in Turkey have become more expensive than most of the EU. Greeks even introduced special non-Schengen on-arrival visa valid on the Greek islands especially for the Turks. Besides that, EU has "green passport" exception for the Turkish nationals, where they can travel visa-free on this kind of passport that is provided to individuals that meet certain criteria and millions of such passports were issued.
The rejection rates are also not bad and EU has a "return agreement" with Turkey, which is designed to keep the middle eastern refugees in Turkey(essentially, if you come from Turkey EU can send you back to Turkey right away ).
Crime rates for Turks show up among the lowest ones, unlike others from the region. So I don't think that EU is trying to reduce visas for Turks.
You are looking at it from Turkish perspective unfortunately.
I am EU citizen, I happen to know the Turkish perspective only because spent some years in Turkey and in fact it is the Turkish perspective that that EU doesn't want them and intentionally makes things harder but the moment you look at what's actually going on you see that this is not the case, just a Turkish fantasy about the "evil West and snobby Europeans". Considering that last year 50K Turks applied for asylum in EU and another 100K overstayed their visa, IMHO EU can be considered pretty generous actually with only 15% rejection rate since Turkey is the 2nd country with most applications after China.
https://home-affairs.ec.europa.eu/news/visa-applications-rea...
https://ec.europa.eu/eurostat/statistics-explained/index.php...
B-visa rejection rate for Turkey in FY24, as per the US State Department, was 19.78%, btw. https://travel.state.gov/content/dam/visas/Statistics/Non-Im...
The US gov’t has been actively targeting CANADA, one of the countries historically closest trading partners and allies.
Maybe in the EU it’s all good, but expect a lot of turbulence in the US.
That doesn't explain the same poor operational quality before it became a liability
>> the system just kick her out
The "waterfall model" is a toxic way of thinking that pervades corporate management. Simplistic minds can't fathom any states other than "done" or "not done". Corporations are determined to crush the human soul. That is why it's not a progressive series of forms, saving your progress all along.
More-or-less agreed about the waterfall model, but you can't blame horrific US government website performance on "corporations" or "corporate management". This is precisely the sort of thing that would get you fired in any real-world corporation that wants to survive, and it's precisely the fact that you can't get fired by the federal government that allows this sort of thing to continue.
Another data point - 5he Indian visa system is similar. The official website ending in .gov.in, which is hard to find, offers a visa for $10 and minimal hassle. The scam websites, with better SEO sell the same shit for $80. They’re just proxying your application to the real website and pocketing the difference.
It would be good if the Indian government could block the scammers but I guess it’s a lower priority for the moment.
Not sure if this is the case for India, but I've experienced similar situations for other countries, but the 'scam websites' actually provided a real service - if you needed some ultra-urgent processing (like you only realized you needed a visa to this country before boarding a flight, once you were already at the airport check-in...) they were able to provide 30 minute approval, whereas the official site's accelerated processing was 24 hours.
So obviously the only way they could to this is with government contacts meaning the government themselves could already do it, but a lot of immigration stuff everywhere is full of people taking kickbacks.
No the scammers were slower than the official Indian website.
The scam websites are probably owned by someone who works in the Indian govt.
This was the case with Ghana. The Embassy in the US had an unofficial offical partnership with an expediter scam (charge more for faster shipping, looks very official). They fired the whole visa staff when it finally came to light. Probably because someone forgot to let their manager's manager in on the scam.
Almost certainly, entire industries have been given over to indian scammers and their government allies.
Modhi, for one
damn bro, how bad situation on there????
I know that Indian scam stereotype is racist and bad but how much it is "that bad"
I found the real website, but the application never went through, always some issue. My boss told me which service to use and everything just worked. (I could expense that service so cost didn't bother me)
My understanding is that India visa processing improved quite a bit. Back when I was speaking internationally quite a bit, I actually had to cancel trips to India on two separate occasions because of delays in getting visas. (Once was under the old visa system and the other was because of delays in switching to a new system. Both times were through a visa expediting service.)
I'm not too familiar with network side stuff. What would a port scan be able to detect that would indicate that you're a scammer?
Just a guess, but maybe a typical bot has a webserver, ssh server, some other servers running on the same machine, whereas a typical Visa applicant doesn't.
Or a browser automation server (Marionette/CDP). I seem to remember watching a presentation where it was mentioned you could detect them this way, <s>but I don't remember where or what it was called.</s> this one: https://www.youtube.com/watch?v=4nZD6ee2Xo8 (WHY2025: Stealth Web Scraping Techniques for OSINT)
Huh, how do you imagine that would work? This "scan" is happening inside client-side javascript, delivering the file through a proxy wouldn't "detect" anything about the proxy.
I imagine it may not be a proxy in the true sense, but a headless browser that's "proxying" the application process rather than the network traffic itself.
Proxy is being used in the traditional sense here. It’s common for a business (scam or legit) to handle visa applications on behalf of customers.
This is a very clever answer.
If the proxy scams are just a little clever, they'll run the proxy on an another IP.
it's riddled with scams, and thinking any of this will detect any of the things you mention is very foolish, native and show a total lack of understanding of the scams. of you think using a proxy is essential for visa scam, i would even know where to begin to correct you.
it's one hundred per cent clueless privacy invasion. they are probably also opening ports via other means and using that for side channel ID like Facebook does.
just like any other documentation scam, the only weak point is on the "last mile" that's why you will always have a human interviewer.
the visa process is abusive and unpractical because people will work around any hurdle and their kpi will never be affected no matter how crappy they manage to make to whole process. or how many doge kids implement useless privacy invasion tech just because.
It's coming from a F5 script, which is a company that sells anti-bot protection amid other things. (It's coming from obfuscated script at /TSPD, which is a F5 thing.)
https://www.f5.com/
TS seems to be short for TrafficShield (a product of some company F5 acquired in early 2000s) and PD seems to be Proactive Defense (?)
Isn't F5 the company that makes nginx?
I didn't know that! But apparently yes
That bought nginx, but yes!
"Since looking into it, I noticed that uBlock Origin already has the default list "Block Outsider Intrusion into LAN" but it wasn't enabled."
Never knew that this existed. Thank you!
Checking out the initial request on github for this feature I wonder why is this necessary? What access to the local network does the browser provide, or need to provide, and why isn't this something developers are more concerned about? I had a feeling this was possible as I see lots of mdns requests when I connect to certain things running sockets.
https://github.com/uBlockOrigin/uAssets/issues/4318
There are certainly use cases, but whether they’re warranted is a good question.
One popular router maker offers a ‘magic URL’ (domain name) that scans your network for the gateway management page, and redirects. It’s not necessary, but it certainly helps novice users. Having worked in IT support,
I’ve also purchased hardware devices that have a web management UI; which connects directly instead of proxying through a cloud.
Ultimately this is probably one thing that should be behind a permission request (like webcam access), but it’s not a feature without value.
It's being looked at.
https://bugzilla.mozilla.org/show_bug.cgi?id=1481298
7 year old ticket updated and prioritized because of https://localmess.github.io/
This is how it always is with Firefox, you hit some bug and then find that it was entered YEARS ago, while they burn focus on things like Pocket.
I’m flabbergasted that this is even allowed. Who thought it was a good idea to allow any web page you visit to access your local network?
Internal apps on non-private IP addresses occasionally use this. There is a standard called Private Network Access[1] that requires these requests to have preflights like CORS requests. Only Chrome has implemented it so far.
[1]: https://wicg.github.io/private-network-access/
Why though? What is the use case that demands this? It'd better be a real pressing need because the security risks are immense and obvious. This is a backdoor to every network firewall.
It’s more that it wasn’t prevented back when the web was first coming together, because security wasn’t on almost anyone’s minds at all. There wasn’t a hole added at some point; it’s just that browsers didn’t specifically block domains that resolve to public IPs from accessing domains that resolve to private IPs.
Realistically, it’s a backdoor to every network firewall that has existed for the entire era in which browsers were used in “secured” internal networks also connected to the internet. Everyone has either designed with it in mind, or gotten lucky that nobody tried to use it on them for like 30 years. I think it’s good to put away this footgun, but there’s no useful blame to assign here.
i thought it was prevented by standard browser cross-domain security checks. Thats why I'm so surprised.
Requests that need a CORS preflight will fail with any browser from the last 20 years, yes. The private IP addresses are not any more vulnerable than `www.google.com` is from `www.notgoogle.com` for cross-origin policy (subdomain-sensitive policies have a small extra vulnerability). But you’re right that doing this kind of thing without nefarious intent is an insane edge case and it should be opt-in. People spray `Access-Control-Allow-Origin: *` like it’s DDT in the 50s and half ass security in general when it’s on an intranet, so an extra guardrail is still worth it.
I'm hazy on the details, but:
Home Assistant has a well-known public name that opens your local instance. On first access, you need to give it the name or ip of your instance, which is saved in browser storage. This supports deep links into your config from forum posts.
My mum also had a shitty D-Link wifi mesh device, which was packaged as an appliance. I cannot speak lowly enough about that garbage device, but then, I am not really the target market. iirc it had something similar; a public dns name for local appliance mgmt.
How is that the same thing? That is a DNS entry that resolves to an internal IP. That lets a user explicitly type a domain and get something internal. That wouldn’t allow cnn.com to ports scan my fridge.
No, it is not, or it would not work outside 192.168.
I remember Fritzbox devices doing the same. Wasn’t a real problem until someone actually hijacked the fritz.box domain.
Massively improved my security posture with this. Thanks all!
Likewise I didn't know it existed, but it was enabled on my laptop and mobile browsers.
Is that available in lite version too? Now that the origin js being phased out
… or you can instead phase out those browsers who try to force blocker restrictions i.e. spyware on you (e.g. chrome and such), and use one of the browsers where you can use the full-featured (not "lite") uBlock Origin instead, e.g. Firefox.
Firefox might be an okay browser, but that would imply supporting Mozilla.
I've been meaning to switch to Vivaldi. Just as soon as the onboarding dialog stops crashing.
I wonder how bad does Mozilla have to be that you have to continue using Chrome without ublock?
Worse than it is now, I suppose.
I'm curious: What's your reasons for not wanting to support Mozilla?
I disagree with their politics, I'm concerned by the multiple privacy incidents, and I generally refuse to support them until they refocus on Firefox instead of all the other stuff they're doing.
If they worked only on Firefox, I'd have nothing against them. As it stands, I can't even donate to Firefox if I want to.
It’s only being phased out on Chrome, by Google.
Yes, to make us safer, now you enable developer mode and disable signature checking to install it locally, thanks Google
Soon, you won't be able to install it locally because the API it relies on will no longer be available. Use Firefox.
Or Microsoft Edge
Microsoft will eventually (TBD) remove Manifest v2 support from Edge, too[1].
> Manifest V2 extensions will no longer function in Microsoft Edge, even with the use of enterprise policies.
[1] https://learn.microsoft.com/en-us/microsoft-edge/extensions/...
Isn't that because Edge has been a wrapper around Chromium for a while now? Presumably support will follow upstream.
I thought Edge also did not support true ad blocking?
You can't change browser? Or is there something bigger happening?
Not everyone wants to change browsers.
then again, if the makers of one big browser (and via there also the derived browsers) start force-shoving spyware upon you (by restricting blockers), it comes down to a decision of how you set your priorities. Personally, It's a clear cut red line, but you do you.
Just checked, and it seems like it is. Not enabled by default for some reason.
It is not being phased out for Firefox.
I'm using uMatrix and it blocks by default all connections outside the requested site and parent domains. For example, if I request https://mail.yahoo.com, connections to yimg.com are blocked. I need to manually allow each CDN for each website, so this attack/profiling won't work.
Using uMatrix was very annoying at first, most websites are broken without their CDNs, but after a few months or so, the whitelist grew and it contains 90% of websites I visit.
On my system https://ceac.state.gov/genniv/ tries to connect to captcha.com, google-analytics, googletagmanager, 127.0.0.1 and "burp" (a local hostname that doesn't exist in my network). Interestigly, the browser console doesn't list connection attempts to localhost or burp. If I allow 127.0.0.1 and "tcpdump -i lo", I see connections to port 8888, which isn't open.
How does uMatrix handle the Facebook tracking pixel, or the replacement which is the Conversions API Gateway?
This is a container that FB gives you to host that lives under your domain (it can be your main domain) that slurps up user data and sends it to Facebook from the server side. You embed some JS in your website, and they hoover up the data.
It doesn't handle it. Anyway, there's no way to know what a website does on the server site. Even a completely static website could be sending the server logs somewhere.
There are options to not load JS, images, XMLHttpRequests, frames, cookies, for each site, but it doesn't list individual files.
Then why use it? They're number one.
No other extension is giving me control like uMatrix does, even considering it's limits.
> On my system https://ceac.state.gov/genniv/ tries to connect to captcha.com, google-analytics, googletagmanager, 127.0.0.1 and "burp" (a local hostname that doesn't exist in my network).
That will be this burp: https://portswigger.net/burp/documentation/desktop/tools/pro...
Sounds like they don't want you to analyze their site.
uMatrix is archived and I think uBlockOrigin is now advised to use(which incorporate uMatrix by enabling advanced settings)
For those who want to try blocking more stuff you can enable hard mode and bind relax blocking mode keyboard shortcut
I'd recommend also enabling filter lists(I advice yokoffing/filterlists and your region/language)
https://github.com/gorhill/uBlock/wiki/Blocking-mode:-hard-m...
But uBlockOrigin UI is so much worse...
Besides, uMatrix works fine. It's that kind of program that doesn't need any updates.
I would really like an intuitive UI for people who don't want to do 'a project' to get their config tight.
But it is intuitive... I don't know what you mean.
You can't manage a whitelist with a single big red on/off button, if that's what you want.
You technically can, that is what community website rule Recipe are for.
I didn't knew community/public whitelists exist, nor any browser extension that uses whitelists and blocks all other connections by default, like uMatrix does. Do you have any examples?
I meant that uMatrix has that community rules recipe feature you can apply with few clicks.
Until uBO has an even remotely usable interface for this use case people (including myself) will continue to use uMaxtrix or forks of it instead.
Amen. I would (and did!) switch browsers to continue using uMatrix rather than go without (and uBO is not a replacement)
I reluctantly switched to only uBo because of uM bugs. But the UI/UX is just a huge step backwards to enable mobile usability.
uBO advanced settings still isn't as flexible as uMatrix was though, fwiw. (I did give in and switch in the end though.)
With uBO I can't block cookies by domain.
It seems to try to check if you are using the Burp Suite on their web application.
How does it manage to hide the requests to 127.0.0.1 from the network tab?
The requests are not made, because some operating systems prevent this.
If you're on OSX, the permission to "discover on the local network" prevents it from happening ( System Settings -> Privacy & Security -> Local Network -> yourbrowser )
Could also be 'network' permissions on firefox ( Go to Settings > Privacy & Security > Permissions ) which is on a per site level, but iirc that could be set site-wide at some point.
The other browsers likely have similar configs, but this is what I have found.
Looks like this is new to MacOS 15 Sequoia, as I don’t see a Local Network option in Sonoma.
I have no ideea. Possibly that's a limitation of Chrome+Firefox developer tools (I get the feeling it's the same code)?
But I found what "burp" is: https://portswigger.net/burp/communitydownload
It seems like they only make the localhost requests on your first visit. If you open devtools in incognito mode (or just clear the cookies) before accessing https://ceac.state.gov/genniv/ you should see those 127.0.0.1 attempts as ERR_CONNECTION_REFUSED in the network tab.
Somewhat more worryingly, Little Snitch doesn't report them at all, though that might just be because they were already blocked at the browser.
This is what I see.
https://i.imgur.com/lvjg2YQ.png
> 400_random_url_with_numbers_403
That looks so much like test code that was shipped to prod.
Searches for that string on GH does return results.
Whitelisting seems to be the way to go. With IPv6 and OS generated IPs (up to what the ISP domestic router allows) could be very efficient.
How and why do browsers allow this? Why wouldn't the browser ask for permission in the same way that it does for Microphone access?
It's insane to allow any random website to port scan my LAN. If this wasn't a "feature", I would have considered this a high severity vulnerability
Chrome doesn't allow it - local network services have to opt-in to being fetchable from public sites (https://github.com/WICG/private-network-access), although they're replacing it with a user-permission-based approach (https://github.com/WICG/local-network-access).
(There is some language online suggesting PNA has not actually shipped, but I experienced it myself in stable Chrome several years ago, so I am unsure of the current state).
Firefox doesn't implement either approach -- I assume this is indicative of their lack of development resources.
> Firefox doesn't implement either approach -- I assume this is indicative of their lack of development resources.
Since ublock had this as a feature for a long time, I'm sure they are aware of it. Unlike other non funded oss projects, Firefox can't and shouldn't shield themselves with this lack of development resource excuse. They have millions.
A trillion dollar company (that loves huge vanity projects) gave up on maintaining a browser because it was too much work and just ship a Chrome fork now. I won’t defend Mozilla’s allocation of their resources, but even if they put it all into the “right” Firefox features the web platform is too complex and too much of a moving target for a company with mere centi-million revenues.
To be honest they weren't trying to build a better browser. Atleast not anymore, earlier edge was nice. They just wanted more data for ads / money. Going the chrome way was more profitable for them.
I thought Mozilla was different.
They are also firing as many senior folks as possible. You should revisit what ever argument you are trying to make.
Microsoft? Were they firing as many senior folks as possible in 2018 when they announced they would give up on EdgeHTML and Chakra? Or in early 2020 when it actually came together? That’s not my recollection of the FAANG-ish job market at the time these decisions were made.
If you meant Mozilla, they’re a total indefensible trashfire for sure. But I’m not convinced they could have succeeded with their resources.
That extension has "Access your data for all websites" ... I really don't get how anyone can give that permission to anyone that isn't well known (a company with a lot on the line) or a person famous for their work (the uBO dev) who has stated he will never sell to anyone or do bad things.
"Hacks and Hops" doesn't even have a valid home page. The extension links to https://g666gle.me/ which does not exist. The domain name itself does not want to make me give access to all my data for all websites to them.
As nice as this extension seems, I would ever in a million years install it.
Unfortunately this level of incoherence is almost universal on HN and similar forums. You'd have to be completely out of your mind to install this extension, but people for some reason believe they can install privacy. They got whipped into fearing nebulous online actors so much that they'll download FSB rootkits dressed as VPNs. The minimal set of actions a rational person would take after realizing they've been tricked into installing this extension is setting their entire PC on fire and then running it over with their car, while moving all of their bank accounts to new accounts, in person, and changing all of their passwords using a brand new device.
Many sites do it .Included in many standard device fingerprinting / anti anonymity SAAS. Ebay facebook etc all do this ! But it looks this is first party to prevent the adblocking of them
1MB of obfuscated fingerprinting + portscan + Webgl . But oddity this one is trying to find burp suite specific route's.
Madness! How do I harden my network against that?
Chrome is already in the process of killing it https://developer.chrome.com/blog/local-network-access
The company I work for has a legitimate service that runs on the loopback (it provides our web apps APIs for some device integration) hopefully its just as simple as the user accepting the prompt else we'll be drowning in support. We had to go the path of the local service because they killed NPAPI. I've been thinking about using web serial as an alternative but Firefox doesn't support it.
That being said, I think this is an overall win, hopefully Firefox implements it in a consistent manner as well.
How is your company's service started on the loopback interface? You bundle a web server that is installed alongside a native app?
This how many of them work for transporting vs traditional old way of registering url scheme and requiring user interacts --- Discord, Blizzard net, Riot Client ... all localhost listener's that can interact
Roughly, yes. Customers (or more often, their IT department) runs our installer which installs the server as a windows service.
Enable "Block Outsider Intrusion into LAN" filter list in uBlock Origin.
Thank you!
You should actually harden your browser or PC... to block any unwanted requests. Apparently some browser extensions can do that.
It would be the job of the operating system to give or take away the ability of your browser to access your local network. But you can run your browser in a container/vm and disable localhost. (And use a separate browser for localhost only if you need it.)
my bank did this on the site they sent me to in order to activate my new card.
The "port scan" just seems to be a local connection to 127.0.0.1:8888. I don't know what purpose it serves on this page, but our government websites often use this technique to communicate with native software for digitally signing documents.
Are you seeing connection attempts to other IPs?
Might also be card readers, debug servers, etc.
Could also be incompetence :D until I fixed it, deploying from my local machine rather than CD resulted in one of the baked in URLs being localhost rather than the public host on the project I'm working on now. Their local development server might just be at port 8888. Wouldn't surprise me.
I looked at the website again and noticed that the request paths looked odd, one of them being `/400_random_url_with_numbers_403`. I googled that and it looks like it's part of a client-side bot detection script that's testing something, the explanation isn't very informative.
https://my.f5.com/manage/s/article/K000138794
> These requests are caused by the bot profile to test the different browser capabilities.
> 'http://127.0.0.1:xxxx' request is a call to the localhost/client machine, which is normal when trying to protect assets like end-server using ant-bot defense. It does not have any impact regarding application page load.
This is most likely an attempt to connect to a webserver on your own device to collect data and/or do tracking.
Remember back in June when Facebook/meta got caught tracking users trough a webserver on Android phone thought Messenger and Instagram? Same thing.
See: https://news.ycombinator.com/item?id=44169115 and https://news.ycombinator.com/item?id=44175940
Why do you say that’s most likely?
This is a common pattern for connecting to smart cards / hardware security devices. Probably a service or hardware that’s run on official CBP machines that should be disabled for prod, but forgot.
This is by far the most likely reason.
I personally use pages that authenticate via a smartcard using this exact scheme.
There is a Java "plugin" that is nothing but a mini webserver that listens on a specific port and performs authentication.
How are you so sure?
Just a little side note - in this context, it makes sense if the website tries to connect to a local port because you might be running a card reader(ie. terminal). This is how it works with some(all?) EU countries that have a chip in their ID cards, or even vehicle registration cards, which you can use to access sensitive information or perform certain administrative tasks on government websites.
Although, from personal experience, it used to require java and it worked only on internet explorer and since it has been retired and replaced with chromium, i am not sure what is the way to make it work nowadays, as i have not been able to figure out to use it when i needed the last time.
It requires installing a local service that bridges between the browser and the smartcard driver (what Java applets did in earlier years). The web app then communicates with the service via requests on localhost. The card-specific driver and bridge service are often bundled together for installation.
I've had it before where it asked me to use an iPhone/Android app which can read the passport's NFC chip. I guess that's the modern replacement for IE/Java.
Embarrassed to say that I wasn't aware of this practice. Are there malicious uses for this beyond fingerprinting?
Yes. Facebook was using this trick on Android. Meta's android apps would host a server on localhost, and their sites would communicate with this local server to pass tracking information that would otherwise be blocked by all browser protection methods on Android. I guess it is still fingerprinting, but at the most extreme end.
https://news.ycombinator.com/item?id=44169115
Mostly it's great for tracking although I'm sure it could also be used to exfiltrate data (e.g. if the user is running something sensitive on localhost).
https://www.digitalsamba.com/blog/metas-localhost-spyware-ho...
Routers with vulnerable URLs. You can search for: "router" "authentication bypass".
Isn't CORS supposed to prevent this?
CORS doesn’t prevent requests (i.e. GET requests from IMG tags, or XHR preflight requests), it only prevents web apps from processing the response if the responding server doesn’t agree. And a simple GET or even OPTIONS request can be enough to exploit vulnerabilities in routers and other local devices.
https://files.catbox.moe/g1bejn.png
When I visit the site from Safari on macOS I see this in the console. Are there any particular services that use port 8888 for the website to do this?
https://my.f5.com/manage/s/article/K000138794
It seems to be part of some "bot defense" product by these F5 people, to "test the different browser capabilities". I doubt it's intended to hit a real endpoint on any system.
Capturing forensic artifacts of the local network allows a building a bridge strategy for identifying fraudulent networks without requiring knowledge of the path taken from destination to recipient. Other local devices do this and send the network map during a phone home, allowing comparison to a source of truth that is tied almost directly to the person, or group of people.
There is also a lot of fingerprintable material within such a port scan from clock skew, TCP ISN, and a few other areas.
You can sieve this quite easily with this available, thanks to Roku's, Phone's, and other things doing this while just sitting locally in a shared collision domain (a digital soldier quartered in every home).
The metadata node graph of devices locally acts as a unique fingerprint once in RFC1918 space, technically not unique but close enough.
> Blocks malicious websites from port-scanning your computer/network
How does that work? A browser extension can't influence how your router and other machines in your network react to incoming requests.
As far as I understand it, it is supposed to be a scan done by the browser on the user's computer, not an external scan, which a browser extension wouldn't be able to detect.
Hopefully should soon be a thing of the past with https://developer.chrome.com/blog/local-network-access
I see. So the website would try to access private IP adresses (RFC 1918) by having elements like <iframe src="http://10.0.0.1"> in the web site and then the web site would check if the iframe was loaded successfully?
It could also just try making the request with javascript. Or try a websocket connection.
Judging just from the screenshots, it seems it blocks websites from accessing 127.0.0.1 get requests. Not a port scan to the outside, more of what do you have running on the local machine inside your network.
but it can hook javascript methods before that scan can happen.
Very interesting. Having looked at NoScript it seems like you can disable LAN as a default value under the allow tab.
Looking further
* uBlock Origin and Lite have it as an option under Filter List > Privacy > Block Outsider Intrusion into LAN
* Brave prevents it, tested with Aggressively block Trackers and Ads.
Why do you need a heavyweight extension to block sites from scanning your local network? Ridiculous.
Also I wonder if this protection is available only with old extension manifest version or new network request hooks API also supports it.
Data my friend, data. Ports scanning? Well, tell us about the hosts and the port numbers. Add some logs if you got.
If you did not go into the details, chances are that when you will, this will turned out to be a false positive case.
If you did, where are the evidence?
Perhaps to avoid people using misconfigured open proxies https://en.wikipedia.org/wiki/Open_proxy
Like a less sophisticated Tor/VPN that is easily detected by port scans
If would be interesting to see what happens on OpenBSD. With pledge(2) and unveil(2) in Firefox, I wonder what it would see. I expect it would see nothing.
I will give it a try and see what happens and if I see anything I will add it here.
I saw nothing of note on OpenBSD. I added the plugin and it prompted me an attempt was made to scan the network, it said it blocked the scan
SO, I guess that is going to be used on all my firefox runs.
Pushing the burden of network permissions management outside the browser, to the OS? Heresy!
To be serious, this has introduced me to sandboxing on BSD via pledge [0] and comparisons against Linux seccomp [1] - thank you!
[0] https://news.ycombinator.com/item?id=17289654
[1] https://kristaps.bsd.lv/devsecflops/ (submission by same poster at https://news.ycombinator.com/item?id=44264021)
Have you double-checked whether the IP isn't shared among multiple website domains? That's quite a classic with IP based filtering with hosters like GCP...
They’d likely block you if they detected something like RDP open, cause that would likely indicate you’re hiding your real IP address.
Checking if you are sharing torrents, run a tor node, mine coins?
It's most likely smartcard authentication code.
My biggest grief with that site is that it's like something from the 90s.
Yeah it should have a fixed header and footer along with a pop-up consent drawer so you can only see 10% of the actual site content.
So much better.
Modern web design is a joke.
t-shirt worthy quote - "modern web design is a joke" ;)
>like something from the 90s
It looks useful and looks good, there's minimal unneeded whitespace and I'm glad it looks as it does. We'd be better off if the entire web switched to a style like this.
As something from the 90s myself, I find this rude.
It's also inaccurate, as this style of page (relating to layout and specific graphic style) didn't appear until 2006ish.
The 1990s web was actually good
I think you are confusing something from the 90 with something from the gov
Looking like something from the 90s would be a feature, not a bug.
In the 90s and early 00s, we did tons of user-testing and feedback collection. We threw all that research away to create UX's that are minimal and "sleek". Tons of unnecessary whitespace and the concept of "Discovery" just thrown into the dumpster. Skeuomorphism was one of the greatest features of 90s-00s software, ironically thrown away as computers got faster and were able to handle the graphics better.
These guys need to look at Gov.uk, this site is a total horror show.
I wish gov.uk was even a smidgen as "outdated" looking as that page.
Most likely some "antivirus" bs. Probably harmless. Fun fact. Most browsers allow by default GET access to web resources on localhost and LAN. Been used for exploits since last century.
For another example, studentaid.gov doesn’t work in private browsing.
I can one better (worse): A state-run website that my sister frequents for her job requires Internet Explorer. Seriously. I installed a Chrome extension that modifies her user-agent header to IE, and it works fine. Easy work-around, but totally lame.
I just tried opening it in a private window and the page loaded and rendered. What part doesn't work?
Isn't it sort of contradictory to try to use private browsing with a service that requires your identity?
Not necessarily.
I might create a login for a porn site so that I can have some favorite videos bookmarked and it can figure out the type of material I like. That doesn't mean I want my history saved locally.
Not contradictory at all. These days private browsing for most people just means (1) don't save the browsing history and (2) log me out of all websites temporarily.
But as the other post notes, it goes further (than, for example, Chrome Incognito) in ways that can break sites. Incognito means exactly what you said, while Safari Private Browsing means more.
this is awesome
Be careful your security tool isn't producing false positives.
I remember years back when people would run these firewalls and we'd get complaints from home users about normal traffic.
Thinks like complaints our mail servers was scanning them on port 25 when they sent email.