Just a small note on the code in the linked script:
API_KEY = os.environ.get("YOUTUBE_API_KEY")
CHANNEL_ID = os.environ.get("YOUTUBE_CHANNEL_ID")
if not API_KEY or not CHANNEL_ID:
print("Missing YOUTUBE_API_KEY or YOUTUBE_CHANNEL_ID.")
exit(1)
Presenting the user with "Missing X OR Y" when there's no reason that OR has to be there massively frustrates the user for the near zero benefit of having one fewer if statement.
if not API_KEY:
print("Missing YOUTUBE_API_KEY.")
exit(1)
if not CHANNEL_ID:
print("Missing YOUTUBE_CHANNEL_ID.")
exit(1)
Way better user experience, 0.00001% slower dev time.
I write a decent amount of Python, but find the walrus operator unintuitive. It's a little funky that API_KEY is available outside of the `if`, perhaps because I had first seen the walrus operator in golang, which restricts the scope to the block.
This isn't really unique to the walrus operator, it's just a general python quirk (albeit one I find incredibly annoying). `for i in range(5): ...` will leave `i` bound to 4 after the loop.
Also true of JavaScript pre-ES5, another language that on first glance seems to only have function scope: it actually does have block scope, but only for variables introduced in `catch` blocks. AFAIU that was the standard way for a dumb transpiler to emulate `let`.
I wonder if that was ever popular, considering the deoptimization effects of try/catch, and given that block scope can also be managed by renaming variables.
Exception blocks don't create a different scope. Instead, the name is explicitly (well, implicitly but deliberately) deleted from the scope after the try/except block runs. This happens because it would otherwise produce a reference cycle and delay garbage collection.
This is the type of things that make me roll my eyes at all the wtf JavaScript posts[0], yes there are a lot of random things that happen with type conversions and quite a few idiosyncrasies (my favourite is that document.all is a non empty collection that is != from false but convert to false in an if)
But the language makes sense at a lower level, scopes, values, bindings have their mostly reasonable rules that are not hard to follow.
In comparison python seems like an infinite tower of ad-hoc exceptions over ad-hoc rules, sure it looks simpler but anywhere you look you discover an infinite depth of complexity [1]
[0] and how half of the complaints are a conjugation of "I don't like that NaNs exist
[1] my favourite example is how dunder methods are a "synchronized view" of the actual object behaviour, that is in a + b a.__add__ is never inspected, instead at creation time a's add behaviour is defined as its __add__ method but the association is purely a convention, eg any c extension type need to reimplement all these syncs to expose the correct behaviour and could for funzies decide that a type will use __add__ for repr and __repr__ for add
> yes there are a lot of random things that happen with type conversions and quite a few idiosyncrasies... the language makes sense at a lower level, scopes, values, bindings have their mostly reasonable rules
The "random things" make it practically impossible to figure out what will happen without learning a whole bunch of seemingly arbitrary, corner-case-specific rules (consider the jsdate.wtf test currently making the rounds). And no, nobody is IMX actually simply complaining about NaNs existing (although the lack of a separate integer type does complicate things).
Notice that tests showcasing JavaScript WTFery can work just by passing user data to a builtin type constructor. Tests of Python WTFery generally rely on much more advanced functionality (see e.g. https://discuss.python.org/t/quiz-how-well-do-you-know-pytho...). The only builtin type constructor in Python that I'd consider even slightly surprising is the one for `bytes`/`bytearray`.
Python's scoping is simple and makes perfect sense, it just isn't what you're used to. (It also, unlike JavaScript, limits scope by default, so your code isn't littered with `var` for hygiene.) Variables are names for objects with reference semantics, which are passed by value - exactly like `class` types in C# (except you don't have to worry about `ref`/`in`/`out` keywords) or non-primitives in Java (notwithstanding the weird hybrid behaviour of arrays). Bindings are late in most places, except notably default arguments to functions.
I have no idea what point you're trying to make about __add__; in particular I can't guess what you think it should mean to "inspect" the method. Of course things work differently when you use the C API than when you actually write Python code; you're interacting with C data structures that aren't directly visible from Python.
When you work at the Python level, __add__/__iadd__/__radd__ implement addition, following a well-defined protocol. Nothing happens "at creation time"; methods are just attributes that are looked up at runtime. It is true that the implementation of addition will overlook any `__add__` attribute attached directly to the object, and directly check the class (unlike code that explicitly looks for an attribute). But there's no reason to do that anyway. And on the flip side, you can replace the `__add__` attribute of the class and have it used automatically; it was not set in stone when the class was created.
I'll grant you that the `match` construct is definitely not my favourite piece of language design.
> methods are just attributes that are looked up at runtime
At runtime when evaluating a + b no dunder method is looked up and there is no guarantee that a + b === a.__anydunder__(b) https://youtu.be/qCGofLIzX6g
What i mean with weird scoping is
def foo():
e = 'defined'
try:
raise ValueError
except Exception as e:
print(e)
print(e) # this will error out
foo()
I also dislike how local/global scopes work in python but that is more of a personal preference.
I agree that that Javascripts standard library is horrible the jsdate.wtf is an extreme but apt example, IMO most of these are solved with some "defensive programming" but I respect other opinions here.
> And no, nobody is IMX actually simply complaining about NaNs existing
I watched many Javascript WTF! videos on youtube and NaNs and [2] == "2" were usually 90% of the content.
It's not that odd, since it's the only situation where you cannot keep it bounded, unless you enjoy having variables that may or may not be defined (Heisenberg variable?), depending on whether the exception has been raised or not?
Compare with the if statement, where the variable in the expression being tested will necessarily be defined.
> Compare with the if statement, where the variable in the expression being tested will necessarily be defined.
if False:
x = 7
print(x)
print(x)
^
NameError: name 'x' is not defined
Ruby does this sort of stuff, where a variable is defined more or less lexically (nil by default). Python doesn't do this. You can have local variables that only maybe exist in Python.
Well, after writing my comment, I realized that a python interpreter could define the variable and set it to None between the guarded block and the except block, and implicitly assign it to the raised exception right before evaluating the except block, when the exception as been raised. So technically, it would be possible to define the variable e in GP example and have it scoped to "whatever is after the guarded block", just like what is done with for blocks.
Is there any chance this would cause trouble though? Furthermore, what would be the need of having this variable accessible after the except block? In the case of a for block, it could be interesting to know at which point the for block was "passed".
The answer is: it is unbound. Intellisense will most likely tell you it is `Unbound | <type>` when you try to use the value from a for loop. Would it be possible that it could be default initialized to `None`? Sure, but `None` is a destinctivly different value than a still unbound variable and may result in different handling.
I stand corrected, the exception case is definitely an oddity, both as being an outlier and as a strange behaviour wrt Python's semantics. Or is it a strange behaviour?
In the case of an if like in your example, no provision is made about the existence of x. It could have been defined earlier, and this line would simply update its value.
Your example:
if True:
x = 5
print(x) # 5
Same with x defined prior:
x = 1
if False:
x = 5
print(x) # 1
What about this one?
if False:
x = 5
print(x) # ???
On the other hand, the notation "<exception value> as <name>" looks like it introduces a new name; what if that name already existed before? Should it just replace the content of the variable? Why the "as" keyword then? Why not something like "except <name> = <exception value>" or the walrus operator?
While investigating this question, I tried the following:
x = 3
try:
raise Exception()
except Exception as x:
pass
print(x) # <- what should that print?
> `for i in range(5): ...` will leave `i` bound to 4 after the loop.
reply
This "feature" was responsible for one of the worst security issues I've seen in my career. I love Python, but the scope leakage is a mess. (And yes, I know it's common in other languages, but that shouldn't excuse it.)
I don't remember the exact details, but it basically involved something along the lines of:
1) Loop through a list of permissions in a for list
2) After the loop block, check if the user had a certain permission. The line of code performing the check was improperly indented and should have failed, but instead succeeded because the last permission from the previous loop was still in scope.
Fortunately there was no real impact because it only affected users within the same company, but it was still pretty bad.
Oof that's a near miss. That's the sort of hard-to-find issue that keeps me up at night. Although maybe these days some ai tool would be able to pick them up
Python 2 actually did let comprehension variables leak out into the surrounding scope. They changed it for Python 3, presumably because it was too surprising to overwrite an existing variable with a comprehension variable.
I cannot tell you how many times I've hit issues debugging and it was something like this. "You should know better" -- I know, I know, but I still snag on this occasionally.
It would be utterly nuts otherwise. For loops over all elements in a sequence. If the sequence is a list of str, as an example, what would the «item after the last item» be?
the issue isn't the value of i, the issue is that i is still available after the loop ends. in most other languages, if it was instantiated by the for-each loop, it'd die with the for-each loop
There's no block scope in Python. The smallest scope is function. Comprehension variables don't leak out, though, which causes some weird situations:
>>> s = "abc"
>>> [x:=y for y in s]
['a', 'b', 'c']
>>> x
'c'
>>> y
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'y' is not defined
Comprehensions have their own local scope for their local variables, but the walrus operator reaches up to the innermost "assignable" scope.
It's only existed for 6 of those years so perhaps you can be forgiven :)
The last time I wrote Python in a job interview, one of the interviewers said "wait, I don't know Python very well but isn't this kinda an old style?" Yes, guilty. My Python dates me.
It seems that way! I briefly experimented with it when it first came out, but never used it in any production code. I've never seen it used by anyone else, either.
I find it interesting that none of the voluminous python code I've had AI tools generate has ever had a walrus operator in it. Reflects the code they're trained on, I guess.
Neophytes take notice. Attention to details like this is what separates truly great programmers from merely good ones. That said, for scripts reusable by others you should use command line arguments . Environment variables in lieu of command line arguments is a huge code smell.
The threat model is that history is persistent while the environment isn't. That said, whenever possible you should handle secrets using file descriptors as opposed to environment variables.
Using read or an equivalent, presumably. Just because you don't know why a practice is recommended against doesn't mean that there isn't a good alternative.
Typer has a great feature that lets you optionally accept argument and flag values from environment variables by providing the environment variable name:
No, that's an anti-feature. :) Sibling comments here claim that command line arguments "leak" whereas environment variables does not. It's plain wrong. An attacker with access to arbitrary processes' cmdline surely also has access to their environ. Store secrets in files, not in the environment. Now you can easily change secret by pointing the --secret-file parameter to a different file. The only reason people use BLABLA_API_KEY variables is because Heroku or something did it back in the day and everyone cargo-culted this terrible pattern.
One could write a huge treatise on everything that is wrong with environment variables. Avoid them like the plague. They are a huge usability PITA.
This is bad advice. Please don't make claims about security if you're making it up as you go.
Environment variables are substantially more secure than plain text files because they are not persistent. There are utilities for entering secrets into them without leaking them into your shell history.
That said, you generally should not use an environment variable either. You should use a secure temporary file created by your shell and pass the associated file descriptor. Most shells make such functionality available but the details differ (ie there is no fully portable approach AFAIK).
The other situation that sometimes comes up is that you are okay having the secret on disk in plain text but you don't want it inadvertently commited to a repository. In those cases it makes sense to either do as you suggested and have a dedicated file, or alternatively to set an environment variable from ~/.bashrc or similar.
Thank you. I see something like this all the time on one of the sites I use for work. If you fail the 2-factor, it'll tell you your password was wrong and reset the whole thing instead of telling you the 2-factor code was wrong or expired.
if not API_KEY and not CHANNEL_ID:
print("Missing both YOUTUBE_API_KEY and YOUTUBE_CHANNEL_ID.")
exit(1)
if not API_KEY:
print("Missing YOUTUBE_API_KEY.")
exit(1)
if not CHANNEL_ID:
print("Missing YOUTUBE_CHANNEL_ID.")
exit(1)
That way you don't end up fixing one just come back and be told you're also missing another requirement
Even better would be to only check each once and buffer the decision:
valid = True
if not API_KEY:
print("Missing YOUTUBE_API_KEY.")
valid = False
if not CHANNEL_ID:
print("Missing YOUTUBE_CHANNEL_ID.")
valid = False
if not valid:
exit(1)
This way you only check each value once (because your logic might be more complicated than just checking it's not set, maybe it can be wrongly formatted) and you still get to do whatever logic you want. It also removed the combinatorial problems.
This is a pretty general principle of separating decision from action.
I feel like use case and audience matters when making these decisions. In this case, the user is probably someone interacting with a python script they're running in a console (I assume by print), then I really don't think it matters - the user will check that both things are set. Should you also give them some documentation about setting env vars? Should you customize that documentation to the OS they're running? etc.
If the user is a typical consumer using a typical consumer interface, then yes you want to handhold them a bit more.
$ python3 -c "print('clear messaging'); exit(1)"
clear messaging
$ python3 -c "raise ValueError('text that matters')"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ValueError: text that matters
and that story gets _a lot_ worse when some programs raise from within a "helper" module and you end up with 8 lines of Python junk to the one line of actual signal
It doesn't copy template repos, but rather creates a list of imperative steps to perform. Steps can be both manual (obtain an API key and store it here) and automatic (run 'uv init'). Markdown syntax, ruby string interpolation and bash.
I made a small wrapper for cookiecutter, to handle some minimal Git integration (making a repo and doing the first commit) and to match my workflow better (write some initial code first, then transform it into a "project folder" in place): https://github.com/zahlman/cookiebaker
I'm not a huge fan of cookiecutter on aesthetic principles, though. I think it's chosen some needlessly heavyweight dependencies for such a simple task. PyYAML comes with a big chunk of compiled C, while performance is really not going to matter and I'd rather use TOML anyway. I've yet to see a project depending on Rich that uses more than a tiny portion of it; this is no exception. More to the point, Rich brings in the much larger Pygments (for syntax highlighting; most of the bulk is actual syntax definition rules for a variety of programming languages) and you can't disable that. And honestly I'm not a fan of the whole "download other people's templates using an integrated client" model anyway; Requests and its dependencies are pretty bulky, too.
If you work at an agency or as a freelancer and you build various similar apps with similar tooling and base setup, being able to scaffold and have them all setup quickly, not having to do it manually and waste hours id important. Similarly, if you work on various small open source packages, you want the tooling to be the same, READMEs look the same, etc, a script or tool to “spit out” the basic structure can be nice.
On the other hand, if you set up the app or larger open source package and you’ll work only on that project for potentially years, setting up a project individually, organically makes a lot of sense.
I’m legitimately curious about what you enjoy and why. Rewriting boiler plate code and trying to ensure that I get everything right with a process I don’t often execute is the very definition of toil to me.
> Not only because the syntax is more human-friendly, but also because the Python interpreter is natively integrated in all Unix distros
That's kind of very optimistic evaluation - literally anything beyond "import json" will likely lead you into the abyss of virtual envs. Running something created with say Python 3.13.x on Ubuntu 22.04 or even 24.04 (LTSs) / Rocky 9 and the whole can of worms opened.
things like virtual envs + containers (docker like)/version managers become a must quickly.
“import json” is the kind of thing which requires picking and installing libraries in batteries-not-included languages, and it’s just one of many modules which are in the standard library. That’s not a compelling basis for large projects but over the years I’ve shipped a ton of useful production code which never needed more than the stdlib and thus spent no time at all thinking about deployment or security patching.
Also, it’s not the 2000s any more. Using venv to isolate application installs is not very hard anymore and there have been decent package managers for a long time.
“broken” is hyperbole. It works fine for millions of people every day. If you have some specific scenarios where you want it be better, it’s better to say those rather than just complain about an open source project.
> Millions of people put it into Docker, or they just deal with it and you see the results with tons of Stackoverflow questions
Arrogantly wrong.
I've coded in Python for almost 20 years. Many of those years I've had it as my primary language at work.
2024 was the first year I actually needed a virtualenv. Before that, I'd happily use pip to install whatever I want, and never had a version conflict that caused problems.
I often encounter junior folks who default to using a virtualenv, not because they need to, but because they've been brainwashed into believing that "you're doing it wrong if you don't use one".
>2024 was the first year I actually needed a virtualenv. Before that, I'd happily use pip to install whatever I want, and never had a version conflict that caused problems.
Okay I'll bite: how did you deal with situations where you needed to work on two different projects that required different versions of the same dependency?
I was talking about pip, not venv. I don't use venv either, not because I think it's a bad idea but because I can't be bothered. Stuff does end up conflicting unless I use Docker (lol) or uv.
Because I went to the official pip "getting started" docs and did exactly what it says? It's bad even in venv though, like not managing your requirements.txt or installing conflicting stuff. The web is full of questions about this, with answers that involve installing some other thing.
but yes, pip predates the paradigm of package managers managing your dependency file instead of you managing your dependency file and then invoking the package manager
The rest of the thread makes it clear that you expect a "package manager" to do more things than Pip does.
Pip is focused on installing the packages, not really "managing" them. The problem is that there are quite a few different takes on what "management" should entail. That's where all the alternatives are coming from.
Whichever you call it, at the end of the day it's more common to encounter dep problems in Python than with NodeJS, Rust, etc.
But also, ignoring things pip isn't meant to do like version management and just focusing on installing, pip's default of installing to some unclear system location was always confusing, particularly when Py2 vs 3 was a thing.
I would like to try uv, but I don't find Poetry that bad for dependency resolution times. If I have to wait 2 minutes once a week it's not the end of the world. Maybe if you're constantly installing new things that are tough to resolve it's an issue.
It's not the end of the world, but it's annoying. I used to work at a place that had a huge monorepo with tons of dependencies. Poetry would sit there grinding away for 4 or 5 minutes. With smaller projects, I agree it's not much of an issue. But, uv has other cool features and it's super snappy!
I'm not arguing on venv to isolate installs, I'm saying that relying on Python coming from UNIX-like OSes is close to impossible for, at least web related projects. I.e. Ansible does a lot [in terms of what code it generates] to keep itself compatible with whatever Python3 versions it may find on systems ( remote hosts )
I have a silly theory that I only half joke about that docker/containers wouldn't've ever taken off as fast as it did if it didn't solve the horrible python dependency hell so well. You know something is bad when fancy chrooting is the only ergonomic way of shipping something that works.
My first taste of Python was as a sysadmin, back in 2012 or so, installing a service written in Python on a server. The dependency hell, the stupid venv commands, all this absolute pain just to get a goddamn webserver running, good lord. It turned me off of Python for over a decade. Almost any time I saw it I just turned and walked away, not interested, no thanks. The times I didn't, I walked right back into that pile of bullshit and remembered why I normally avoided it. The way `brew` handles it on macOS is also immensely frustrating, breaking basic pip install commands, installing libraries as commands but in ways that make them not available to other python scripts, what a goddamn disaster.
And no, I really have no clue what I'm talking about, because as someone starting out this has been so utterly stupid and bewildering that I just move on to more productive, pleasant work with a mental note of "maybe when Python gets their shit together I'll revisit it".
However, uv has, at least for my beginner and cynical eyes, swept away most of the bullshit for me. At least superficially, in the little toy projects I am starting to do in Python (precisely because its such a nicer experience), it sweeps away most of the horrid bullshit. `uv init`, `uv add`, `uv run`. And it just works*.
> I have a silly theory that I only half joke about that docker/containers wouldn't've ever taken off as fast as it did if it didn't solve the horrible python dependency hell so well.
I don't think this is a silly theory at all. The only possibly silly part is that containers specifically helped solve this problem just for python. Lots of other software systems built with other languages have "dependency hell."
Back in the early days of Redhat, rpm's didn't really have good dependency management. Yes there were rpms, yes you could download them, but getting the full dep tree was a PITA. Most people installed the full Linux distro rather than a lightweight version because of this.
Debian's apt-get was very "apt" at the time when it came out. It solved the entire issue for Debian. There was a point at which there was an apt-rpm for redhat. Yum tried to solve it for redhat, but didn't really work that well -- particularly if you needed to pin packages to certain versions.
I won't touch Python either, but because I've been burned debugging large Python programs. Something that would have taken a minute in a statically typed language took hours of tracing data through the program to understand what was supposed to be in a dict. There are alternative languages that are pithy, statically typed, can write programs quickly, and can grow into large code bases that are maintainable; so there is never a reason to start a new project with Python today.
I've seen the same thing in .NET and Java where there's 800 layers of interface and impl and it's an adventure trying to find the actual business logic in all the layers of indirection
>so there is never a reason to start a new project with Python today
Nothing else has an ML/data ecosystem that compares. Perl/Go are maybe a distant 2nd
I deal with that all the time cause an adjacent team uses Java with tons of boilerplate and frameworks. At that point, your static typing isn't so static. Takes them forever to make changes, to the point where I started taking over responsibilities using Python code.
Most python written at a large scale uses types (TypedDict) and/or Pydantic for safety and never plain dict objects. That's a code smell in any language, we can stuff data into `map[string]interface{}` all day long and cause problems downstream.
> However, uv has, at least for my beginner and cynical eyes, swept away most of the bullshit for me.
uv is _much_ better than what came before. As someone who has only had only glancing contact with Python throughout my career (terrible experiences at jobs with conda and pip), uv feels like Python trying to actually join the 21st century of package management. It's telling that it's in Rust and clearly takes inspiration from Cargo, which itself took inspiration from Ruby and Bundler.
These are all features of the [pipx backend] for mise, and the docs discuss what to do in the case of things like python updates, etc. The advantage of doing it this way, particularly for a global mise config, is that you treat these python tools as basically any other mise tool, so their versioning is easy to control.
I know mise isn't really a package manager. But with its support for things like this, be it for python, ruby, npm, or cargo, as well as more universal support from things like Ubi and the upcoming github backends, its rapidly becoming my favorite package manager. I've a number of projects that use particularly useful node based tools, like markdown-lint or prettier, that aren't JS based in any way, shape, or form. Having to carry around a package.json felt weird, and with the way mise handles all of it, now I don't have to
If you're dealing with a managed system, chances are, compilers are banned. You'll have to be unsafe and work outside the constrained environment - potentially violating policies, contracts, regulations and laws.
I don't have to deal with this in JS or I think in other stuff like Golang. I give someone a package.json with versions of everything. npm install always sets deps up locally, but doesn't need to copy the entire NodeJS runtime.
The difference is it's default, it always works the same everywhere, it actually writes down the deps, and I don't have to manually switch between them (or set up fancy bashrc triggers) like venvs.
Hope they make uv default then. It's nice, but I have to separately create a project with it, and regular python commands don't work with it, both of which go back to it not being default. But even that won't fix all the old projects.
Basically is by now. This is more of a recent thing for frontend JS, but NodeJS (which is more directly comparable to Python) had npm I think from the start.
Those browser JS libs installed via <script> tags though, honestly were pretty convenient in a way.
Yes, please use virtual envs or containers. I know it seems overly heavy and hard to manage, but you don't want to end up in a situation where you're afraid to update the system because the library upgrades might break some of your Python code.
+1 - I've been shocked at how many little portability issues that I've had shipping software, even within the relatively constrained environment of fellow employees.
Minor differences between distro versions can make a big difference, and not everyone that uses a Python script knows how to use something like pyenv to manage different versions.
I consider my point as still valid with UV, what you wanted to express?
On UV specifically - say 'asdf' compiles python right on your system from official sources - means using your ssl libs for example. UV brings Python binary - I feel worried on this.
uv works just as well with whatever Python you want to bring -- you're not required to use the Pythons that uv is capable of installing on your machine.
I agree - Python without uv is masochistic. But that really negates the "Python is already available" advantage. If you have to install uv you can just as easily install Rust or Go or Deno.
I agree that the built-in Python is typically not suitable for development, especially if you're planning to distribute your code and/or care about the versions of its dependencies. (And especially on Debian and its derivatives, in my experience; they even remove parts of the standard library, like Tkinter [0].)
I disagree that virtual environments represent an "abyss". It takes very little effort to learn how they work [1], plus there a variety of tools that will wrap the process in various opinionated ways [2]. The environment itself is a very simple concept and requires very few moving parts; the default implementation includes some conveniences that are simply not necessary.
In particular, you don't actually need to "activate" a virtual environment; in 99% of cases you can just run Python by specifying the path to the environment's Python explicitly, and in the exceptional cases where the code is depending on environment variables being set (e.g. because it does something like `subprocess.call(['python', 'foo.py'])` to run more code in a new process, instead of checking `sys.executable` like it's supposed to, or because it explicitly checks `VIRTUAL_ENV` because it has a reason to care about activation) then you can set those environment variables yourself.
Creating a virtual environment is actually very fast. The built-in `venv` standard library module actually does it faster in my testing than the equivalent `uv` command. The slow part is bootstrapping Pip from its own wheel - but you don't need to do this [2]. You just have to tell `venv` not to, using `--without-pip`, and then you can use a separate Pip (for recent versions — almost the last 3 years now) copy cross-environment using `--python` (it's a hack, but it works if you don't have to maintain EOL versions of anything). If you need heavy-duty support, there's also the third-party `virtualenv` [3].
Much of the same tooling that manages virtual environments for you — in particular, pipx and uv, and in the hopefully near future, PAPER [4] — also does one-off script runs in a temporary virtual environment, installing dependencies described in the script itself following a new ecosystem standard [5]. Uv's caching system (and of course I am following suit) makes it very fast to re-create virtual environments with common dependencies: it has caches of unpacked wheel contents, so almost all of the work is just hard-linking file trees into the new environment.
3. run ./configure --enable-optimizations --with-lto
4. run make -s -j [num cores]
5. sudo make altinstall
This will install that specific version without overwriting default system python.
You can then bash alias pip to python3.xx -m pip to make sure it runs the correct one.
All the libraries and any pip install executable will be installed locally to ~/.local folder under the specific python version.
Alternatively, if you work with other tools like node and want to manage different versions, you can use asdf, as it gives you per folder version selection.
virtual environments are really only useful for production code, where you want to test with specific versions and lock those down.
Virtual environments are useful for isolating dependencies for different projects, not just isolating the interpreter.
(I mean, except on Windows, your venvs default to symlinking the interpreter and other shared bits, so you aren't really isolating the interpreter at all, just the dependencies.)
AFAIK, it's the same even on POSIX, when you create a venv, "./bin/python" are symlinks and the pyvenv.cfg has a hardcoded absolute path of the current python interpreter the venv module was using at the time of creation. It really doesn't isolate the interpreter.
(also one of the reasons why, if you're invoking venv manually, you absolutely need to invoke it from the correct python as a module (`python3.13 -m venv`) to make sure you're actually picking the "correct python" for the venv)
I meant that the symlink behavior is default except on Windows (though it may just be on POSIX platforms and I think of it as “except Windows” because I don't personally encounter Python on non-Windows non-POSIX platforms.)
Looking at just the first link, looks way more complicated than venv. And I'm a C++ developer, imagine someone who less experienced, or even who just isn't familiar with C toolchains.
It’s really not hard; the person you’re replying to put a lot of details in. I’ve lost track of how many times I’ve built a Python interpreter. Virtual envs used to work badly with some tools and dependencies, so I had to do it a lot. It’s gotten better and now I only compile Python to get a version that’s not provided by my Linux distribution.
Even what you just pasted is wrong. And it would give a confusing error if you used it. Try "sudo apt install vim \ make" to see.
It's the kinda thing an experienced engineer wouldn't have that much trouble with, but you should be able to recognize how much experiential knowledge is required to compile a complex C code base and what kinda dumb stuff can go wrong.
You probably don't need to do much of the stuff on that page to build, but "What is dnf?", "Is the pre-commit hook important?", "Do I need cpython?", "What's an ABI dump?" are questions many people will the wrestling with while reading.
> The first link is a sudo apt install command that you copy paste into terminal. in what world is that more complicated than venv?
venvs also aren't complicated.
I have built Python from source before, many times. I do it to test Python version compatibility for my code, investigate performance characteristics etc.
Re-building the same version of Python, simply in order to support a separate project using the same version with different dependencies, is a huge waste of time and disk space (hundreds of MB per installation, plus the mostly-shared dev dependencies). Just make the virtual environment. They are not hard to understand. People who want a tool to do that understanding for them are welcome to waste a smaller amount of disk space (~35MB) for uv. A full installation of pip weighs 10-15MB and may take a few seconds; you normally only need one copy of it, but it does take some work to avoid making extra copies.
Maybe I'm the only one that finds Python simultaneously verbose and lacking? Either you need 500 dependencies to do something in a simple way, or you need dozens (if not hundreds) of lines to do trivial things. I avoid writing Python because there's so much bullshit to add. Much prefer Perl, I can actually get things done quickly. Python feels like programming for the sake of programming.
I have a bunch of zero dependency projects. You can get a lot done with the standard library and single file projects which you can run on most systems without needing to do anything except curl it down and run it since Python is available.
About 25% of the code is using argparse to parse commands and flags. With that said, I tend to prefer code that yields more lines for the sake of clarity. For example this could technically be 1 or 2 lines but I like each parameter being on its own line.
parser_edit.add_argument(
"-s",
"--sort",
default=False,
action="store_true",
help="Sort your profile and show a diff if anything changed",
)
Argparse is part of my problem with Python! I can never remember the weird ways it wants to collect all these attributes and features, so I'm always leafing through manual pages or googling for examples. Same for if I'm modifying existing Argparse code, I have no idea how to make it do what I want. It can do a lot for you, but it's a pain in the ass to use.
It's so much easier to write a simple usage() function that has all help text in one big block, and Getopt with a case block. Anyone can just look at it and edit it without ever looking up documentation.
One nice thing about argparse is it comes with a helper function for mutually exclusive groups that can be optional or required. For example you can have --hello OR --world but not both together.
> Much prefer Perl, I can actually get things done quickly
Python lets you just nest data structures without having to twist your brain. You want a tuple in a list in a dictionary value: you just write it down and can access it with a unified notation. Bam. No reference madness and thinking about contexts. It's a big part of what I typically need to get things done quickly and understand how I did it 5 years later. That has to count for something. Python is boring in that sense, Perl is fun. But that's exactly my problem with it. It's too clever for it's own good and writing it does things to your brain (well at least mine).
That's what got me off Perl. I loved it and was skeptical about Python, but one time after trying it for a week or so, I wondered what it would look like to pass a dict of lists of dicts into a function, then reference items inside it. My first try worked: I made a dict, with a list inside it, and a dict inside that list, and passed it as an argument without any sigils or reference decorators or anything.
That was about the time I stopped using Perl for any new projects. I never wanted to go back.
Perl also lets you nest data structures, no brain twisting involved. Perl doesn't have Tuples natively but you can add them with a module. You can also nest objects in complex data structures.
perl <<'EOF'
my $object = bless sub { my ($s) = @_; if ($s eq "thingy") { return("hello world!") } else { return("goodbye world!") } };
my %hash1 = ( baz => "here we go", foo => $object );
my @array1 = ( undef, undef, undef, \%hash1 );
my %hash2 = ( bar => \@array1 );
my $reference = \%hash2;
print( $reference->{bar}->[3]->{baz}, "\n" );
print( $reference->{bar}->[3]->{foo}->('thingy') , "\n" );
print( $reference->{bar}->[3]->{foo}->('something') , "\n" );
EOF
here we go
hello world!
goodbye world!
Perl being Perl, TMTOWTDI, but you can enforce your own style using linters and formatters.
Major downside to dictionaries being so useful in python... Everyone uses dictionaries everywhere. I'm so happy for dataclasses and attr, because having to check that str keys are right without ugly ass constants is miserable.
YMMV, but in my experience, Python's top use cases tend to require fewer dependencies, not more. Many trivial tasks are already built into the language. Not saying Python is perfect, but it's definitely well-oriented in that regard. I'm curious in what kinds of use cases have you found the opposite to be true?
I think the classic Python vs. Perl question ultimately comes down to using what you feel most comfortable with. People are different, and that's totally fine.
Using it as a replacement for Bash. No environment required, Perl is already installed on every computer on earth. Perl is generally quicker to write for string-oriented workflows - which is what Bash is. Regex is quick and easy.
Perl falls apart when you need structure and even small projects. For short scripts meant to do what you would use Bash for, it's a godsend. Safer, more portable, less friction.
Perl does not fall apart when you need structure, it's OO. In fact it's better structured than Python (hello, PyPI? It's CPAN calling, we wanted to let you know hierarchical packages are a thing). If you mean "forcing people to program in an opinionated manner", it doesn't do that by default, but that's what linters and formatters are for (like the ones Python programmers use)
I'm glad someone else discovered they can like python.
I got forced to learn it for a project where I was proposing Ruby and the customer insisted on Python. This was years ago when Ruby was much slower. I was annoyed but I got used to it and here I am enjoying it many years later.
I take issue with the description and use of make though! :-D What is the point of it if you're not going to use dependencies? One might as well just write a script with a case statement..... I'm adding smileys because I don't mean to be all that serious but I really do think that the failure of the youth of today to get to grips with Make is a sad reflection on our culture....... and get off my lawn, ok? :-)
I wish python had support for optional braces. Maybe time I copy some snippet or integrate some other project into mine and have to worry if someone using 2 spaces, 4 spaces or tabs.
> One might as well just write a script with a case statement.....
I started with this and evolved into simple flat Makefiles, because they're basically the same but Make feels more standard (there's a Makefile here vs. there's a random bash script here).
For me, python is the closest thing to writing pseudocode that functions. Every time I have the instinct to gloss over a thing when writing it down (because it feels obvious in my head), it turns out that python has an intuitive abstraction for it.
Coming from a mathy background I found it incredibly satisfying, although I’ve come around to other languages since.
As a mathy person myself, I find the OOP leaning difficult to think about. Better equational reasoning, better lambdas, fewer side effects to worry about, avoiding mutation so that I can define something and still know what it is later during runtime, those are what help me think clearly. To me OOP is about the furthest paradigm from math that I’ve used.
You’re right, it’s not good for math, but unfortunately we have the legacy of AI / ML being entrenched into that ecosystem, with no clear leaders as alternatives.
From what I was told, Python was originally seen as a Swiss Army knife for sysadmins. It started gaining more traction when Canonical adopted it as the main language for Ubuntu 4.10 in 2004.
Then, in 2005, Guido van Rossum was hired by Google to work on Google Cloud. That opened the door for wider adoption in academia, since Python had strong math libraries and integrated well with tools researchers were already using, like Hadoop, right around the time big data and ML were starting to take off.
Also, between 2005 and 2006, two important things happened: Ruby on Rails came out and inspired Django, which was starting to gain popularity, and web developers were getting tired of Perl's messy syntax. That's how Python quickly became a solid choice not just for server-side scripts, but for building proper web apps. In the meantime, another language that could be embedded directly into HTML was storming the web: PHP. Its syntax was similar to JavaScript, it was easy to pick up, lowered the barrier to entry for software development, worked straight out of the box, and didn't require thousands of print statements to get things done.
The 3 Ps made history. According to programmers from 20 years ago, they were like religions. Each had its own philosophy and a loyal group of followers crusading online, getting into heated debates, all trying to win over more adopters. The new generation of devs is more pragmatic. These days it's less about language wars and more about picking the right tool for the job.
> According to programmers from 20 years ago, they were like religions. Each had its own philosophy and a loyal group of followers crusading online, getting into heated debates, all trying to win over more adopters.
It's very weird reading something you lived through described in these terms, as though it were being described by an anthropologist.
Can't help but wonder what the future will have to say about today.
"In 2025, programmers used 'frameworks' to hang 'apps' from some kind of 'web'. They believed these frameworks gave them magic powers. Many even fought in 'flame wars' on this topic, which experts believe involved the use of fire to destroy webs woven by competing programmers."
The key factor imo was Travis Oliphant merging the competing numeric and numarray libraries into numpy in 2005. That quickly became the foundation of Python as the key environment for open source numeric processing.
It brought across a ton of users from R and Matlab.
Pandas, Matplotlib and ScikitLearn then consolidated Python's place as the platform of choice for both academic and commercial ML.
Python's success is entirely due to entry-level programming courses. They all switched to Python, because you have to explain less. I don't think I heard about web servers in Python before 2012. I suppose a 2005 computer wouldn't be able to serve a Python backend smoothly.
PHP's popularity isn't really from 2005-2006. It was popular at the end of the 90s, and it looks like JS as much as it looks like a potato.
This is entirely backward. They all started switching when python was already immensely popular after the popularity boosts given by sysadmin, django and numerical/datascience users.
This is what I have seen as well. When Python became ubiquitous in the industry, they started teaching it in the colleges.
I come from a core science background. I studied Physics. And it was like this in my institute: FORTRAN -> A lot of C, small amount of FORTRAN -> C -> Python. I was taught C, but from the exact next year, it was switched to the Python ecosystem.
It was done much later when Python became the standard in research universities, the language of recent research papers, etc.
A generation of Scienctists learned C/FORTRAN/MATLAB in college/grad school as it was taught, but they switched to Python early/mid career. Teaching Python in undergrad followed.
I also taught a mid-career Economics professor Python. She used SPSS before for her papers. Humanities professors doing data crunching are now switching to Python, too. There is a clear trend.
You could also accidentally summon an Ancient One while trying to write a regex :D
Perl is a fine language, but it's like using a wood chipper with no safeties. It takes extreme care and know-how to use it without splattering ASCII everywhere and making an unmaintainable write-only mess.
For every beautiful and maintainable perl program (mostly irssi scripts) I've seen 99 abominations that are quicker to rewrite completely than decode wtf they are doing.
> Python's success is entirely due to entry-level programming courses.
No, entry-level courses were in a mix of Scheme, C, and other languages until Java’s industrial ubiquity ended up in it becoming a popular choice, but not completely displacing the others. Then as Python (along with the broader class of dynamic OO scripting languages) became quite popular, Python got added to the mix, but, unlike Java, it fairly quickly displaced not only Java but a lot of what had been around longer in introductory courses.
Python’s industrial success drove its initial use in introductory courses, but doesn't fully explain it, as it doing what Java never did indicates.
I think the people teaching introductory courses find it less of a compromise to industrial popularity than they did with Java.
Python’s success is multifaceted, some of it is right-place right-time, a lot of it is the ecosystem it built because of that, but a lot of it is, I think, that it turns out to be a language that has been designed (both initially and in how it is been managed over time) to be a very good language for real people solving real problems, despite not adhering to any of what various typing and performance and paradigm purists like to posit as the essential features of a good language.
1. Python was pretty popular well before entry-level programming courses adopted it. I think they adopted Python because it was a good general purpose scripting language, multiplatform, easy to install and get going, that taught programming concepts in a more approachable way for beginners.
2. Python on web servers was a thing long before 2012. You had Zope in 1998 or so, and it was pretty popular for a while, and hugely influential to subsequent web frameworks. Django came out in about 2005. TurboGears, Pylons in about 2005 or so. Flask in 2010... and these are just the more popular frameworks.
3. I think the author meant that PHP was also vaguely C-like in syntax, like JS. Keyword: vaguely. You had a common way of declaring stuff with curly braces and semi-colons.
> I suppose a 2005 computer wouldn't be able to serve a Python backend smoothly.
Python had web servers from 2000, including Jim Fulton's Zope (really a full framework for a content management system) and in 2002 Remi Delon's CherryPy.
Both were useful for their day, well supported by web hosting companies, and certainly very lightweight compared to commercial Java systems that typically needed beefy Sun Solaris servers.
I'd forgotten about CherryPy until Turbogears was mentioned the other day in the Django birthday thread.
But yeah Python was on an upswing for webdev and sysadmin (early DevOps?) tooling, but took quite a hit with Ruby eg Rails, Puppet, Vagrant and Chef etc.
But Python hung on and had a comeback due to data science tooling, and Ruby losing it's hype to node for webdev and golang for the devops stuff.
> They all switched to Python, because you have to explain less
I think this could be generalized to ergonomics. Java 1.6 is an absolute nightmare for a newb compared to Python. No top-level statements, explicit typing, boxing, verbose declaration syntax, long import statements, curly braces everywhere... and, most importantly, no out-of-the-box REPL. Java has since made strides and borrowed from Kotlin and Lombok but my understanding is that it was too little too late.
Depending upon preference and application one might consider ~half of these things anti-features at the expense of stability but it speaks volumes to why people join and stay in a software ecosystem. If I had to work on a Python project again I'd use mise with uv and viciously lint the codebase down to a reasonable fully-typed subset of the language devoid of custom decorator magic and dunder abuse.
> but my understanding is that it was too little too late.
Too little too late to be the #1 language of choice for serious server-side software that it is today?
The weird thing about Java is that people naturally compare its popularity today to its dominance in the early '00s, which was an aberration. The ecosystem has long since returned to its fragmented self, and while Java is not nearly as dominant as it was during that very short period, no other language is, either.
> Too little too late to be the #1 language of choice for serious server-side software that it is today?
I was replying to
> Python's success is entirely due to entry-level programming courses. They all switched to Python,
not to mention that there are an awful lot of qualifiers in your statement. There are certainly plenty of Java jobs to be had but all the usual suspects like PYPL, TIOBE, SO (disregarding the old adage about damn lies and statistics) put Python squarely above Java in terms of popularity.
This is all to say that if I got conked on the head and lost all of my programming knowledge and had to make a living I'd restart with Python. This isn't a value judgment - the JVM and Python ecosystem are on roughly equal footing in my mind. It's just how things are.
Python may be somewhat more popular [1] in terms of number of people using it (partly because a lot of Python programming isn't about shipping software and many Python users aren't professional programmers), but JS, Python, and Java, the three most popular programming languages today, are each dominant in specific domains, and none of them has a huge market share in the areas where the others dominate. There isn't a lot (relatively) of client GUI software written in Python or Java, there isn't a lot (relatively) of data analysis in Java or JS, and there isn't a lot (relatively) of serious servers in Python or JS.
I argue that Python's success was almost entirely due to it not adopting any best practices on dependency management and being installed by default on systems.
Being default installed was almost certainly the larger factor. As evidenced by how much pain it caused people when they started using dependencies that were not stable. They had to add specific support for it to not let you pip install things to the system install.
> The 3 Ps made history. According to programmers from 20 years ago, they were like religions. Each had its own philosophy and a loyal group of followers crusading online, getting into heated debates, all trying to win over more adopters. The new generation of devs is more pragmatic. These days it's less about language wars and more about picking the right tool for the job.
I feel like the religious wars aspects of this is completely overblown. The conversations around languages really hasn't changed all that much in the last 20 years. Look at the way you see things happening in HN, heck even in the comment thread right here. It's the exact same kinds of discussions as happened back 20 years ago, with exactly the same kind of pragmatism.
I think the gulf comes about from the fact that the primary sources are conversations occurring between people that are largely anonymous behind an online handle, interacting with people that aren't face to face. There's always been an element of exaggeration in the interactions. What might be a "Umm, no" with a maybe a shake of the head, becomes "Your mother was a hamster and your father smells of elderberries". Almost every party involved comes to recognise the particular cultural quirks (which varied from forum to forum) and how to translate what was said, to what was actually meant.
>From what I was told, Python was originally seen as a Swiss Army knife for sysadmins.
Yea, I was a sysadmin around 2000 (before that too) and I knew it as such.
>between 2005 and 2006, two important things happened:
Somewhat - I used it in 2001 for Plone which is based on Zope, which was somewhat popular around that time. Writing all the web stuff with Python made sense, since Plone provided a CMS and could include a wiki. Adding on some sql calls to it in python just made sense. The competition was between PHP and Python, though there were some other less popular choices. Ruby on Rails definitely was getting a lot more popular around those times. PHP didn't start getting popular around 2005, if anything, people started using Python more, and started criticizing the crappy code that was circulating in the PHP community.
In any case, it was a fun time, but what's the point of looking back like that?
This aligns with what I remember and Guido going to Google explains things. Mid 2000's was all about PHP (Cake/Symphony/CodeIgniter/etc) vs Ruby (Rails) in the open source world. I remember hearing about Python only in sysadmin work and Plone. Then, suddenly were suddenly a lot of articles from Google about just about everything, all in Python.
I remember saying to a coworker, "Google is single-handedly keeping Python alive."
Then bam. It was everywhere. Mid 2010's, I took a cybersec job and they told me to learn Python in the two weeks between accepting and starting. "Python is all over cybersec," I was told. It was then I realized Python took over academia, which positioned it perfectly for ML. It's features made it easy to start, but it also benefited from right place, right time.
One of my colleagues, a retired programmer who spent 20 years at Microsoft, told my students that it used to be Java vs Visual Basic, until Microsoft brought in the big guns, like Anders Hejlsberg. That's when it turned into Java vs .NET, and things really started to heat up :)
M$ was too late to open source it and too late to embrace Linux. On desktop side .NET was only for windows back then and xamarin immature. On server side in java land you had framework war so you had many choices and libraries to choose from - companies didn't want to fully embrace M$ ecosystem (windows sql server, windows deployment etc). Then Android came and adopted Java and M$ lost mobile ecosystem war.
I can see similar way why Swift eventually can loose market share - too late to open source and too late to embrace cross-platform.
I'd say the switch was learning that Google Search was written in Python (remember when error pages URL were in .py ?), and Youtube rewritten in Python. After that, reddit and dropbox followed (and later Instagram).
Apple also added Python in, I think, Mac OS X Jaguar, so it blipped on the radar.
When you have big players using it, the community automatically grows.
Django was the continuity of that and definitely contributed to the growing popularity of Python, but I think the hype started way before (and in fact we had to suffer Zope/Plone/Twisted for it :)).
Another decisive date was circa 2010 when the Belgian book "learn programming with python" came out. Granted, Django was already 5 years old at the time, but it brought many beginners who knew nothing about programming at the time.
I make projects following almost identical patterns. It's a little uncanny. Maybe the people in the python developer ecosystem are converging on a pretty uniform way to do most things? I though some of my choices were maybe "my own", it seeing such consistency makes me question my own free will.
It's like when people pick a "unique" name for their baby along with almost everyone else. What you thought was a unique name is the #2 most popular name.
This sort of architecture has been in favor with python for at least 10 years or so, but I think you're right — the structure just makes sense, so many reasonable engineers converge on using it.
Python has done an impressive job over the years of making steady robust improvements. The typing and tooling has just gotten better and better. There are still plenty of problems though, imho async is still a much bigger pain than it should be (compared to other runtimes with a very nice experience like go or elixir, even dotnet has been less pain in my experience). Overall I like python, but it mainly boils down to the robust libraries for things I do (ML, Data munching/analysis)
I don't know what I am doing wrong but nothing written in Python has ever worked for me.
I download the .py repo from from github or wherever and try to run it - errors.
I try to install missing libraries pip this and that - errors.
I battle fixing endless error with dependencies and when the .py finally runs - errors - wrong version of whatever library or wrong patch of this or that or the "production ready" .py does not work correctly on Windows or single file script uses excel library that has changed in incompatible ways 3 times in 2 years.
I Download all the files from the flashy looking web site and follow all the instructions to the letter - errors.
Python is anything but "robust".
It is the most fragile environment in the universe, worse than c, c++ and javascript put togeter, at least this is my experience with it.
That’s wild. The last time I had that experience with Python must have been more than 10 years ago. I’ll admit a decade ago you definitely did need to know way to many details about way too many tools from virtualenvs to setuotools to wheels, etc. to get things working from somebody else’s project, but imho, poetry and uv have really changed all that.
This is exactly my experience too. I avoid python software like the plague, because even when it does work, I can not rely on it continuing to work when python gets updated on my system.
You cant really make a judgement on python from this, as your story just screams user error. I have never encountered anything of what you say, and I have been programming in Python from before I had any clue what I did.
My own experience could hardly be more different. For that matter, I have never had to even look at a "flashy looking web site and follow all the instructions to the letter" in order to install something.
For example, to install yt-dlp, I followed these steps:
sudo apt install pipx
pipx install yt-dlp
Actually, only the second one, because I already had pipx (https://pipx.pypa.io/ — a wrapper for pip that does basic virtual environment management) installed.
Can you name some specific things in Python you have tried to use, and give more concrete descriptions of how you tried to set them up?
It's quite crazy, as Python first version was in 1991, 4 years before Java. It's a language with a ton of legacy baggage, the 2/3 transition, the BDLF being replaced by a completely different project leadership system, and it still keeps going strong.
This interview from Brett Cannon (old core dev who worked on packaging, imports, the vscode python extension...) is eye opening:
The guy cares SO MUCH and they have so many things not to break you can feel the sense of passion and the weight of responsibility in everything he says.
> Python has done a good job of hiding its legacy ugliness (such as __init__, __new__, and similar aberrations), swettening its syntax to accomodate developers with good taste.
What exactly is the problem with __init__ or __new__? @dataclass is very nice syntactic sugar, but are we arguing here that having access to initializer/allocator/constructor dunder methods is "legacy ugliness"? This is the core of pythonic built-in aware python. Bizarre.
I like that magic method names generally all follow the same form so it's obvious that they are magic methods and not intended to be part of the public API. Whether that form uses double underscores or something else doesn't really matter to me as they are not being called directly.
"Magic methods" is just the name people use for these special methods (another is dunder methods). They're "magic" because you do not call them by name, rather they are invoked by the interpreter in specific situations.
Yes, it is magic the same way other magic methods are, that is, because it is syntactically special and callable (and primarily called by) a syntax other than <instance>.<method>(...) or <class>.<method>(instance, ...)
Specifically, in the case of constructors, via <class>(...).
> it's obvious that they are magic methods and not intended to be part of the public API
Is there an alternative API? No. This is public API regardless of anyone's intentions. Though "it's weird" is really not a very strong argument against it.
Public API refers to the user of the class, not the implementer. You never call __init__ directly. __new__ allows modifying object creation itself for the implementer, but the user of the class will never call it.
There are private and public methods. Private methods are only supposed to be called within other methods, as in privately. Public methods are the ones that are normally called through the code, repl, by the user or whatever. You are not supposed to write `myclass.__add__(x)` anywhere except where you define the class itself and its methods.
Internal is more convention as the language doesn't really do anything with it, but it does with munged, and magic methods are specifically for things implemented in the language.
Internal and munged don't exactly map to private and protected, but are kinda similar ish.
Oh right. I am not really into python, had read some doc about naming conventions some poitns.
In any case I actually like how one can use underscores to point on how exposed some method is supposed to be. Makes it simpler to actually know what to skip and what not.
There are several alternative API's. @dataclass is one, Pydantic offers another, there's also attrs, plus less general things like namedtuple.
Admittedly it's obnoxious when you've got habits for one and you're on a team that uses another--totally flies in the face of the zen re: "there should be only one obvious way to do things".
...but that was always a rather ambitious goal anyway. I'm ok navigating the forest of alternative API's if it means not being locked into something that I can only change by choosing an entirely different language. I'm happy that it's very easy to tell when somebody is mucking about with python internals vs when they're mucking about with some library or other.
That's kind of the point. These methods are never meant to be called directly. They're used to desugar.
I think it's fairly short sighted to criticize these. FWIW, I also did that the first time I wrote Python. Other languages that do similar things provide a useful transparency.
Anyone using a monospacdd font, a font where underscores are separated, or a run-together proportional font that they are even a little bit familiar with, because of the relative width of underscores and other characters.
Which, between them, covers approximately everyone.
I mean, maybe you should use a font that doesn't make it hard to figure that out. Next you're gonna say we should ban 0s and Os, Is and ls, because your font doesn't allow you to figure out which one it is.
...by being as shockingly contrasting with their environment as possible. Maybe your aesthetics differ, but I find it quite ugly. If it wasn't, it wouldn't grab my attention so well. Wearing something shockingly ugly is a great way to not be mistaken for a deer.
By contrast, I find a well camouflaged deer quite beautiful--once I notice it at all. The beauty comes from the way that it is so clearly a thing of its surroundings. Nothing at all like a bright orange hat.
> Wearing something shockingly ugly is a great way to not be mistaken for a deer.
Sure... yes the bright orange is ugly, but it's not the ugliness that prevents you from getting shot, it's the bright unnatural color. Other hunters aren't thinking "oh that's really ugly, it must not be something I can shoot" they're thinking "bright orange means person, I should not fire my rifle in that direction".
> If it wasn't, it wouldn't grab my attention so well.
Are you saying that if you thought the bright orange was pretty it wouldn't occur to you not to fire your gun in its direction?
If I thought that bright orange was a part of a healthy forest ecosystem, I would likely see beauty in it. And if my intent was to shoot denizens of such an ecosystem, then yeah bright orange would make a poor indicator for "don't shoot here". You'd be better off with something ugly, something which clearly doesn't belong.
Amazing. If you need to ask this question I am guessing you have spent too long with Python . No other language uses this underscore madness for special methods or otherwise. Because it’s just , I don’t know a more appropriate word for it , stupid.
What's actually amazing is how you're unable to answer a straight question.
If you call something "stupid" it doesn't really convey anything meaningful, especially not in the way you're using there, it comes across as "I don't actually have a reason I don't like it, I just don't".
The programming languages world is broad/varied enough that any statement like "no other language does this!" is almost certainly wrong (outside of esoteric languages, which python and php most certainly are not)
You don't call __init__ directly so the ugliness is limited to the definition which seems fairly minimal to me. It seems to be a naming convention for methods that are not invoked by name anywhere and that's at least informative when you're reading the class definition.
IMO this is less horrendous than e.g. go's insistence that exported functions are indicated by a capital letter - that really affects code using the module not just the definition.
I have found to believe that ugly and beautiful are irrelevant. As long as it's doing the job and not cause major headaches. Life is too short to worry about syntax.
I had honestly not considered that the author might have been talking about the visual aesthetic of the dunder methods themselves. Honestly that has me a bit stumped; all I can say is that if you don't like underscores, don't pick python.
I would think the dumber thing is having to create an empty file called __init__.py for a package to register as a package. And relative imports being completely fubar.
> The __init__.py files are required to make Python treat directories containing the file as packages (unless using a namespace package, a relatively advanced feature).
I wrote it in about an hour and a half. It seems to work in Python 3.6 and 3.12. Now you never need to write another double-underscore magic method again.
As if other language don't have their warts.. Ruby and Javascript for example have hideous syntax, comparing this with some special naming is quite bizarre.
> What exactly is the problem with __init__ or __new__? @dataclass is very nice syntactic sugar, but are we arguing here that having access to initializer/allocator/constructor dunder methods is "legacy ugliness"?
My read was that the "ugliness" was in the method naming, and specifically the double underscores, not the availability of the methods themselves.
Probably the system of giving special meaning to what are otherwise ordinary identifiers. __ isn't a reserved keyword in Python or anything. But there's a set of conventions you're supposed to follow when naming methods and attributes, and they're visibly artificial. It would be cleaner to make the features that are a special part of the language definition also a special part of the language syntax instead of running them in indistinguishably with other nonsimilar things.
In C++, if you want to define the binary + operator on a class, you give the class a method with the special name `operator+`. In Python, to do the same thing, you give the class a method with the pseudo-special name `__add__`. You don't think the Python way is worse?
Have you considered how much familiarity might shape your reaction to the two? Both are specific patterns with arbitrary restrictions which new learners have to internalize, and that’s a fairly advanced task most people won’t hit until they are familiar with the language syntax.
Here’s a counter argument: the Python methods are normal method declarations whereas C++ developers have to learn that “operator=“ means something different than what “operator =“ (or any other assignment statement) would mean, and that this is the only context where you can use those reserved symbols in method names.
To be clear, I don’t think either of these is a big deal - the concepts and usage are harder to learn than the declaration syntax – but I think it’s incredibly easy to conflate familiarity with ease of acquisition for things like this.
> Have you considered how much familiarity might shape your reaction to the two?
It doesn't.
> Both are specific patterns with arbitrary restrictions which new learners have to internalize, and that’s a fairly advanced task most people won’t hit until they are familiar with the language syntax.
No, the Python methods are just ordinary methods with valid names. What 'arbitrary restrictions' are you referring to?
The arbitrary restriction is that you have to just learn a specific pattern and follow it. I can’t have my Python class use “addition” if I hate underscores or my C++ code use “mathematical_division” if I’m horribly pedantic. In both cases, it’s just a particular design decision by the language developers I have to learn, and in both cases I won’t think much of it after the initial acclimation.
Funnily enough, just starting your class method/variable name with __ does do magic things as a pseudo-keyword. Specifically, it mangles the name for callers outside of that class -- `self.__foo` would need to be accessed as `obj._ClassName__foo`. It's python's approach to having private methods, while remaining somewhat ideologically opposed to them existing.
This doesn't apply to the dunder methods, though. They're magically exempt from this magical mangling, so you could call them directly if you wanted. ¯\_(ツ)_/¯
> You don't think the Python way is worse?
They seem about equivalent? I don't see any real reason to pick one or the other, beyond personal preferences.
It prevents accidentally overriding the member in subclasses which can prevent private implementation details of parent classes from being accidentally interfered with by child classes.
As composition over inheritance becomes more idiomatic, there are probably less places where this matters, but as long as inheritance is used at all it can be useful.
It makes it nearly impossible to reference that attribute by accident. Basically, if class Foo defines "self.__bar", the Python interpreter sees the leading "__" and mangles it. It's the lightweight equivalent of a private attribute in other languages, with the difference that Python will let you attempt to access it anyway. But if you do, and it makes your code utterly brittle and prone to explosion, no one will give you sympathy or help you fix it. They're much more likely to tell you not to do that.
It's as close as they'll get to private methods while still having the "we're all consenting adults" philosophy that doesn't prevent you doing something dumb. Or if you use this and it breaks, you can't say you weren't warned.
What problem is that addressing? It doesn't solve the problem that `__add__` is a valid identifier -- so is `add` -- and it also doesn't solve the problem that there is no connection between the symbol `+` and the name `add`.
I've been doing Python professionally for a while (~10-ish years in backend web dev and file processing), and while I don't like how exactly the author described dunder methods as "legacy", it's still not a language facility I reach out to often because, well, it's sort of odd coming from a C# background. The author has blog posts about Java, which isn't that different from C#, so maybe that's the reason.
Maybe as I grow to think of the "big picture" architecture-wise with my code, I will start incorporating dunders, but until then...
Out of curiosity, why would I use Dataclass vs a Pydantic Basemodel. If we did not have a PyDantic dependency I could imagine wanting to use Dataclass. But if I have it, why not use everywhere?
This and not just for the performance reasons. A Pydantic model represents an I/O boundary of a program. This conveys a lot of information to a programmer reading the code. It would be quite misleading to find it's actually only passed around internally and never used for I/O. A bit like using an int type to store a Boolean value.
On the other hand if I see a dataclass I can tell what it's purpose is by whether it's frozen or not etc.
> the Python interpreter is natively integrated in all Unix distros
It's included in the default install of most desktop/server Linux distros (with plenty of exceptions), but I don't believe any of the BSDs ship it in their base system.
IIRC macOS used to have python 2 in its default install, but I vaguely recall that being deprecated and removed at some point. My only Mac is on the other side of the country at the moment, so I can't check myself.
Python 2 was removed in Monterey 12.3, which was incredibly stupid and disruptive as it caught everyone by surprise. We all knew Apple said they would remove it, but everyone was expecting them to be sensible and do it on a new major OS release, like they did with PHP, not mid-cycle.
I wonder if that kerfuffle is why they ended up not removing Ruby and Perl yet, despite the same promise. macOS’ Ruby is around 2.6. Perl I somehow doubt they’ll get to, as it’s such an important part of Unix admin I bet they themselves use it somewhere.
There is still a /usr/bin/python3 which is a shim. When you call it, if you don’t have the Xcode Developer Tools you’ll be asked to install them (it’s a non-scary GUI dialog which takes two clicks) and then you’re set. That is also a few versions behind the cutting edge, but it does get updated sometimes.
> [Software that was officially EOL and no longer supported by the developer as of January 1, 2020 — even though they went back on that with an emergency patch in April] was removed in [an OS released on October 25, 2021], which was incredibly stupid and disruptive as it caught everyone by surprise [despite the fact that the intent to EOL that software was declared many years ahead of time, and that sunset date already represented an extension of multiple years to the usual release schedule].
I will never understand this.
But then, I've been using Python 3 since 3.2 and my first reaction to that was a sigh of relief, and by the time I updated to 3.4 I was already wondering why everyone else was lagging behind on the switch.
Perhaps because your interpretation of my comment is wrong.
> was removed in [an OS released on October 25, 2021]
No, no it was not! That would have been fine. Heck, it would even have been fine if they had removed it the year before. Two years. Three. I don’t care. The problem is that it was removed on a point release (not October) without warning, after setting the precedent of removing another language on a major release.
> But then, I've been using Python 3 since 3.2 and my first reaction to that was a sigh of relief
And I don’t even care about Python. But I still had to deal with the fallout from that from things I didn’t write.
> The problem is that it was removed on a point release (not October) without warning, after setting the precedent of removing another language on a major release.
My perspective is that the problem is that people were trying to use Python 2 after January 1, 2020. I left it behind years before that.
I think that by now we shouldn’t take a .0 major release as a sign of what is really going to be present in macOS and what’s not. The OS stabilizes around .4 or so, and in the meantime some deprecated things may get removed, too.
They also have a habit of sometimes add or change some APIs mid-cycle, so you may see requirements for the latest Xcode being a mid-cycle version of the OS. Or how was it with Mac App Store, that not only itself, but the relevant APIs for programs distributed in it appeared in 10.6.4?
I dont think python is pre-installed on macOS now. (uv has replaced it for me)
Edit: Unlike older versions of macOS that came with Python 2.7 pre-installed, newer macOS versions (Catalina and later) no longer include Python pre-installed.
If you have access to their MacBook and can open a terminal, even `rsync` (which comes preinstalled) would do the job.
It seems much more likely to me they were just tired of having to manage the languages (and being constantly criticised they were behind) and simply chose to remove them.
Why? Installing python and php can be done in 2 seconds with brew, and you have control over what you install instead of using whatever is in the system, with deprecated versions, etc. It is actually much better now. System tool should be left to the system.
I switched from Python to JS for backend stuff a while back, thoroughly enjoying it. I agree that "Python installation and package management is broken," but the async stuff was the biggest improvement to productivity. Yes I know Python got asyncio, but there's a big difference between having one well-accepted way of doing things vs multiple competing, incompatible ways, where the good one has the least momentum.
The rest is small stuff that adds up like Py whitespace scoping, or Py imports somehow not taking relative paths, or JS object syntax is nicer: https://news.ycombinator.com/item?id=44544029
uv + ruff (for formatting AND linting) is a killer combo, though.
And the more you use uv, the most you discover incredible stuff you can do with it that kills so many python gotchas like using it in the shebang with inline deps or the wonders of "--with":
I've avoided Python for a long time, but I'm getting roped in myself, mainly because certain tasks seem to require a lot less code than Java or Perl.
That said, call me old-fashioned, but I really take issue with "curl $URL | bash" as an installation method. If you're going to use an install script, inspect it first.
If your going to execute the code anyway, you either have to inspect everything or trust whoever is providing it. There is nothing special about bash that makes it more dangerous to execute than python.
I like Python for the language, and for a lot of jobs the threading model limitations do not matter. Its a great language to get stuff done. I find the package management story challenging but I will try uv next time!
I resisted pipx, poetry, anaconda and probably some other things, as I didn't see the point over venv+pip. But uv is just so damn good, there's no going back.
Good write up, and solid choices. As someone primarily working in python in the last few years, I have a very similar stack.
Two additional suggestions:
* mise to manage system dependencies, including uv version and python itself
* just instead of make; makefile syntax is just too annoying.
Mise actually has a command runner as well which I haven't tried yet, and might work better for running commands in the context of the current environment. It's pretty nice when your GitHub actions workflow is just:
Otherwise, the existence of a file or folder with the same name as your task ("test", for example) will stop that task from being run, which might be very annoying if you're using the Makefile as part of a script or CI or something where you won't notice the "Nothing to be done for..." message.
Funny enough I feel the other way about JetBrains IDEs. They * seem * super powerful, but there is always a lot of config that needs to go into them if I'm doing app, infra, and pipeline work. (Edit, not saying they aren't powerful, just more that as coming into them I'm never sure how to wield it the best out of the box)
In my experience (not saying this is universal), the folks that like JetBrains IDEs came from java/intellij backgrounds, where I hear it really shines.
This all might be a skill issue, as almost all my professional projects have been VSCode based, but since I've only worked at smaller places I definitely can't rule out this was because it was easier to set things up than to fight for Fin to get us all licences.
In your opinion, what makes PyCharm (or CLion if you want to add that in) 'just work'? Do you think it is because you've used it for so long and just know the ins-and-outs? Or is there something you see that they have and VSCode doesn't?
I've always been curious about this as someone who hasn't had a lot of professional exposure to the JetBrains world.
The things that just work that I look for are mouseover to get info on an identifier (local or external), right click to go to the definition, and running code in the debugger with a click. Maybe I just don’t know how to set that up in vscode; I’m a company of one and don’t have a coworker to ask how to do it. My Python work was just making requests, analyzing the data and outputting results to csv’s. This was replaced by Rust and then C. And the C is 60k loc as it does much more. In Clion when you open a cmake project it automatically understands your project structure from the cmake files and provides those need-to-have features I listed.
"Just work" is certainly subjective, but the introspection in all the JetBrains products are what make them ferocious. If one has never heard of all the 18 quadrillion lint tools before, you can immediately get value from opening a file in PyCharm and having it point out the ways it is going to fail on you, including one of my favorite tricks that no VSCode+lint horseshit has ever thought about trying:
import re
import sys
if re.match(r"[A-Za-z}", sys.stdin):
print("ok")
PyCharm will spot that error immediately, no insaneo configuration required
PyCharm Professional also gets into the SQL side of things:
with connect(":memory:") as conn:
c = conn.cursor()
c.execute("SELECT * FROM oops WHERE NAME IN ('alpha, 'beta')")
instantly spotted, no configuration required
I was going to be cute and use json.loads as an example, but it seems somewhere along the way they botched the fact that the first argument to json.loads is a fucking JSON string. But, it does allow showcasing that you can have PyCharm syntax check the "inner" language of any string literal you'd like via their language injection annotations:
import json
# language=json
bogus = """{"this is subtly wrong"; true}"""
json.loads(bogus)
# and the same with tomllib:
import tomllib
# language=toml
bogus = """
[alpha]
beta = []onoz
"""
tomllib.loads(bogus)
# or, if you have some tricky internal stuff
def do_the_thing(
# language=sql
s: str
):
with connect(something) as conn:
c = conn.cursor()
c.execute(s)
do_the_thing("SELECT * FROM oops WHERE NAME IN ('alpha, 'beta')")
# ^^^ also gets syntax checked because it knows the first arg is SQL
From what? It would be useful to mention towards the front of the post so we know what is the context in which you approach python.
I've switched to python primarily (from perl) in early 2010s (I think my first "seriously" used version was 2.6. This is mostly for system management, monitoring, and data transformation /visualisation. Nothing fancy like AI back then in a work setting.
I found the biggest impact was not so much on writing code but on it remaining readable for a very long time, even if it was created hastily "just get this working now" style. Especially in a team.
Python is still one of my favourites and the first tool I reach if bash is not enough for what I'm trying to do.
First, I grok make. I'm saying this from a position of familiarity, not of ignorance and fear.
Make is great at compiling code in languages that don't have bespoke build systems. If you want to compile a bunch of C, awesome. For building a Rust or JavaScript project, no way. Those have better tooling of their own.
So for the last 15 years or so, I've used make as a task runner (like "make test" shelling out to "cargo test", or "make build" calling "cargo build", etc.). As a task runner... it kinda sucks. Of course it's perfectly capable of running anything a shell script can run, but it was designed for compiling large software projects and has a lot of implicit structure around doing that.
Just doesn't try to be a build system. It's optimized for running tasks. Here, that means it provides a whole lot of convenient functions for path manipulation and other common scripty things. It also adds dozens of quality-of-life features that devs might not even realize they wanted.
For example, consider this trivial justfile:
# Delete old docs
clean:
rm -rf public
# This takes arguments
hello name:
@echo Hello, {{name}}
If you're in a directory with it and run `just --list`, it'll show you the list of targets in that file:
$ just --list
Available recipes:
clean # Delete old docs
hello name # This takes arguments
That second recipe takes a required command line argument:
$ just hello
error: Recipe `hello` got 0 arguments but takes 1
usage:
just hello name
$ just hello underdeserver
Hello, underdeserver
You can do these things in make! I've seen it! Just doesn't add things that were impossible before. But I guarantee you it's a lot harder to implement them in make than it is in just, where they're happy native features.
There are a zillion little niceties like this. Just doesn't try to do everything make does. It just concentrates on the smaller subset of things you'd put in .PHONY targets, and makes them really, really ergonomic to use.
You wouldn't use just to replace make in a large, complicated build. I would unhesitatingly recommend it for wrapping common targets in repos of newer languages, so that `just clean build test` does the same things whether you're in Python or TS or Rust or whatever, and you don't want to hack around all of make's quirks just to build a few simple entry points.
I don't really like Make as a build system, especially since today every language I've worked with has a much better one - cargo, uv, cmake, bazel, others.
My thing is that it's ubiquitous and if you stay simple, the syntax is perfectly readable; if you're doing anything complicated, I'll argue you don't want your logic in a Makefile anyway, you want it in a shell script or a Python script.
I get that if you want to do really complicated things, Just can be more ergonomic. But I haven't seen any real argument that it's more ergonomic, or understandable, or worth the learning curve, when your Makefiles are simple or have actual scripts doing the complicated stuff.
`just --list` is indeed missing (though I hear make is adding a --print-targets flag), but I usually need to run the same commands - make run, make test, make typecheck, make lint, make format. Note how none of these take an argument - that's also something I've never found myself needing.
That’s the thing: just is a lot simpler and cleaner for the simple things, too. Like not requiring .PHONY, for instance. And if you already have to install cargo/uv/cmake, add just to that list of dependencies.
The argument stuff is nice when you realize you’re no longer constrained by not having easy access to it. I used just to build my blog, and added a target to create a template entry whenever I updated to a new release of certain software. I could write “just newversion 1.2.3” to publish an announcement using the same mechanisms that did everything else. Without that feature, I could have scripted something up to do the same. With it, I didn’t have to. I wouldn’t have tried that with make.
If you're an expert in make and already have a Makefile, I would not recommend switching to just for that project.
The benefit of just is that it's designed to be a command runner, whereas make is designed to be a build tool. justfile syntax is much simpler and more ergonomic. It also has nice features: Private recipes, submodules, recipes that you can specify to run only in a particular OS (we use the same justfile for both Windows and Linux), writing your recipes in a language other than your shell language, and many many other niceties.
A new user can start doing "advanced" stuff in just in a couple of hours. They'll take a lot longer if trying to do it via make.
It's funny, I'm the opposite: LLMs have made it easy to draft something in Python, then translate to a more appropriate language for the target problem-domain, for example Go.
Agree . We had training to learn how LLM applications can be developed and it was obviously in Python. Never liked Python but between dependencies issues (the demo project used uv but somehow the author forgot to declare some dependencies which she probably had installed globally) which cost hours of everyone’s time, and the persistent magic going on (pipe operators that do nothing similar to unix pipes, wtf!, puydantic shit just to declare a data type with metadata and lots of things like this) nearly everyone immediately switched to a typed language like Kotlin (they have a nice AI framework) as soon as we got the basics of how things work.
Is go the language that doesn't include a module to do mmap and doesn't include a module to place binary data into a struct?
If you want to read binary files, you can use C or python, but not go (unless you use segfault prone 3rd party libraries of very dubious quality, of course).
"And guess what's the de facto programming language for AI? Yep, that sneaky one."
Is this referring at all to to PyTorch. If not, any guesses what the author has in mind
"Not only because the syntax is more human-friendly, but also because the Python interpreter is natively integrated in all Unix distros."
Is this referring to GNU/Linux.
UNIX (UNIX-like) includes more than Linux; some UNIX distributions do not include Python in the base system
Where it is left as choice to the user whether to install it
I know this because I use such distributions and, unless some software needs it, I do not install Python
In such case, when I am done using that software I uninstall it^1
For example, he mentions retrieving YouTube channel metadata
I do not use Python for this; I use a 19-line shell script (ash not bash), its startup time is faster
Unlike Python, it is included in the base system of the UNIX distributions (both Linux and BSD) that I use
But if I need to test something using yt-dlp, then I might temporarily install Python
1. I compile Python from source and one annoying aspect of the project , in addition to the slow startup time, is their failure to include an uninstall target in their Makefile
"And guess what's the de facto programming language for AI? Yep, that sneaky one."
He's referring to Python in general
"Not only because the syntax is more human-friendly, but also because the Python interpreter is natively integrated in all Unix distros."
I think he means that its available or readily available in many major linux distributions like Ubuntu, Fedora, NixOS, etc. I don't think native is the right word.
I use bash too but Python is amazing. You're right, there are problems related to packaging and speed, but it is still very often the right tool for the job. It's powerful, easy to use, and open source.
For this script I have "implemented HTTP" using printf, an ash builtin.
The TCP networking is done with orginal netcat reading the HTTP from a pipe.
The TLS is handled by a TLS forward proxy listening on the loopback.
The orginal netcat and other TCP clients I use, like tcpclient from djb's ucspi or tcploop from haproxy, are not part of the NetBSD base system but are easily added when I compile the OS. For Linux I use a custom distribution I make myself, without LFS. Busybox has ash and nc together in the same binary.
These TCP client programs are stationary targets, they will work reliably year after year, and small enough that I can store and compile them quickly even on computers with modest resources. Python is constantly evolving, a moving target, and much larger.
I wrote a utility in C89 that "implements HTTP" called yy025. This is produced using GCC, specifically flex, which is intsalled on all systems I use. flex is part of the NetBSD toolchain. It's a requirement for compiling the OS. It's a requirement for building many GNU userland utilities. It's even a listed requirement when building the Linux kernel.
yy025 is what I normally use in shell scripts when I need to generate HTTP. It reads URLs on stdin and outputs customised HTTP to stdout. There is no "third party dependency" for HTTP. This is a "first party" program. I wrote it.
But this script to fetch YouTube metadata doesn't use yy025. It's just some printf statements.
In more recent times, NetBSD has added OpenBSD's version of nc to the base system.
One could thus argue that there is a TCP client "installed on every system".
For me, what is more important is an installed copy of GCC. I have yet to use a UNIX-like system that did not have the needed networking functions to create a basic TCP client.
Worked at a company where that approach lead to huge unwieldy structure that no one dared to touch to not break anything other teams are working on. The problem was not so much the repo, but dependencies structure (like single requirements.txt for the whole repo) and build scripts.
In theory it should've worked great -- you only need to update a dependency once and be sure all the code has the most recent security patches. In reality everyone was afraid to touch it, because it will break someone's else code. Monorepos work great only if you have serious NIH syndrome (Google).
I actually started appreciating damned micro-services there, as long as each service just reflects organization team structure.
Sounds more like a monolith. You can do microservices with separate requirements in a monorepo. We have one setup at work using pantsbuild (eh the tool is a bit quirky) but it's much nicer than constantly spinning up new git repos and CI pipelines for small services/utilities
That said, I think service-per-team is a good pattern
When did Python go out of fashion? This is the second article I've seen talking about it as if it's some kind abomination.
I get that it's not the shiny new thing, but I don't understand people hating on it. Is this just junior devs who never learned it, or is there some new language out that I missed? (And please don't tell me Javascript....)
I have had a somewhat unearned distaste for Python for the last decade or so.
I mostly just don’t like some of the design decisions it made. I don’t like that lambdas can’t span multiple lines, I don’t like how slow loops are, I don’t like some functions seem to mutate lists and others don’t, and I am sure that there are other things I missed.
But it really comes down to the fact that my career hasn’t used it much. I occasionally have had to jump into it because of scripting stuff, and I even did teach it for a programming 101 course at a university, but I haven’t had a lot of exposure otherwise.
For scripting stuff for myself, I usually end up using Clojure with GraalVM (yes I know about babashka), or nowadays even just a static linked compiled language like Rust.
I don’t really understand why people think that compiled languages can’t be used for scripting (or at least task automation), honestly. Yes you have to add a compilation step. This involves maybe one extra step during development, but realistically not even that. With Rust I just do cargo run while developing, I don’t see how that’s harder than typing Python.
> The most insane python feature is that for loops keep their intermediate variable.
"Insane feature" is a generous way of describing this behavior. I would say it is just a stupid bug that has managed to persist. Probably it is impossible to fix now because of https://xkcd.com/1172/
How typecheckers and linters should deal with this is a tricky question. There is how the language ought to work, and then there is how the language actually does in fact work, and unfortunately they are not the same.
Comments like this basically should say "I want as much handholding as possible"
Lucky for you, LLMs are pretty good at that these days.
>And the types I can never trust. I've used all the tooling and you still get type errors in runtime. It's also ridiculously slow.
IDE integrated mypy checking does this in the background as you type. As for errors, it all has to do with how much typing you actually use. You can set the IDE to throw warning based around any types or lack of type annotation.
Because sometimes I want to have logic that spans multiple lines and I don't want to assign it a name. An easy example might be something with a `map` or a filter. For example, in JavaScript
[1,2,3,4].filter(x => {
let z = x * 2;
let y = x * 3;
let a = x / 2;
return (z + x * a) % 27 == 2;
});
Obviously I know I could name this function and feed it in, but for one-off logic like this I feel a lambda is descriptive enough and I like that it can be done in-place.
You're free to disagree, but I think there's a reason that most languages do allow multi-line lambdas now, even Java.
> Obviously I know I could name this function and feed it in, but for one-off logic like this I feel a lambda is descriptive enough and I like that it can be done in-place.
FWIW, you'd also have the benefit of being able to unit test your logic.
I mean, maybe, that's why your lambdas shouldn't be too long.
I have done a lot of Haskell and F#, and I'm very familiar with the concept of "lifting", and yeah being able to individually test the components is nice, but even within Haskell it's not too uncommon to use a lambda if the logic doesn't really need to be reused or is only a couple lines.
If you have a huge function, or you think there's any chance of the logic being reused, of course don't use a lambda, use a named function. I'm just saying that sometimes stuff that has 2-4 lines is still not worthy of having a name.
Honestly, I don't really see the appeal of unnamed functions in general. I so rarely use lambdas that I wouldn't really miss them if they were gone. Just occasionally as a sort key, or in a comprehension.
I have seen people do this in JavaScript quite often, but I always assumed there was some kind of underlying performance benefit that I didn't know about.
As I think about it I guess it makes sense if you're passing a function to a function and you just want it to be concise. I could imagine using something like that off the top of my head, but then pulling it apart and giving it a name the moment I had to troubleshoot it. Which is how I currently use nested comprehensions, just blurt them out in the moment but refactor at the first sign of trouble.
I think maybe I just have trouble seeing some of the braces and stuff, and it's easier for me to reason about if it's named. I guess that's why we have 32 flavors.
Thanks for answering me honestly I really do appreciate it, even if my tone came off as dismissive. Sometimes I don't realize how I sound until after I read it back.
Obviously it's totally fine to have a difference of opinion for something like this.
> I have seen people do this in JavaScript quite often, but I always assumed there was some kind of underlying performance benefit that I didn't know about.
I don't think so, at least I haven't heard of it if there is.
I tend to have a rule of thumb of "if it's more than 6-7 lines, give it a name". That's not a strict rule, but it's something I try and force myself to do.
Like in Python, most lambdas can be done in one line, but that also kind of gets into a separate bit of gross logic, because you might try and cram as much into an expression as possible.
Like, in my example, it could be written like this:
But now I have one giant-ass expression because I put it all into one line. Now where previously I had two extra names for the variables, I have the ad-hoc logic shoved in there because I wanted to squeeze it into a lambda.
And I think that's where the reasoning behind only allowing one line lambdas came from. I believe I read a thread a long time ago where GVR didn't even want to include lambdas at all, if I have time I might look for it and edit in a link.
At it's core, I think it's fair to say Python is about forcing the user into formatting their code in a readable way. It's gotten away from it over the years for practicality reasons, and to increase adoption by people who disagree on which ways are more readable.
Sometimes I wish they would take nested comprehensions away from me, I am too lazy to avoid them in the heat of the moment, and I get a thrill out of making it work, even though I know they're disgusting.
If your style is doing a lot of functional programming, multi-line lambda is a very natural thing to do. Other times you want to use a variable or several variables without actually passing it around as an argument. It makes sense especially if you are already used to it in Java/C++/JavaScript/Go.
Is it "better" than a named function? No, of course, they work mostly the same. But we are not talking about better or not. We are talking about syntax just for the sake for syntax, because some people prefer to write code in a way you don't necessarily care about.
This makes a lot of sense, I got to a similar conclusion in my other reply.
I always thought the appeal of functional programming was more about test-ability and concurrency, it never occurred to me that people actually preferred the syntax.
Can't speak for anyone else, obviously, but part of the reason that I got into functional programming is specifically because I found the syntax very expressive. I felt like I was able to directly express my intent instead of describing a sequence of steps and hope that my intent comes to fruition.
Different strokes and whatnot, not everyone likes functional programming and of course there are valid enough criticisms against it, but I've always appreciated how terse and simple it feels compared to imperative stuff.
Even with regards to testability, if your function is pure and not mucking with IO or something, then even using a multi line lambda shouldn't affect that much. You would test function calling it.
Keep in mind, Haskell doesn't really have "loops" like you'd get in Python; in Python you might not necessarily need the lambda because you might do your one-off logic inside a for loop or something. In Haskell you have map and filter and reduce and recursion, that's basically it.
weeell, you can still do stuff like this in Haskell:
import Data.Vector.Mutable
...
let n = length vec
forM_ [0 .. n-1] $ \i -> do
next <- if i < n-1 then read vec (i+1) else pure 0
modify vec (+ next) i -- increase this value by next value
it's just in many cases the functional machinery is so accessible that you don't need to reach for for-loops.
Apparently the author used to be a Java person. I have this feeling - or prejudice - that people still in the Java World are a bit out of tune with the tech industry, working on huge legacy projects on big retail or US banking. These people tend to be conservative and resistant to change, so maybe that's where these types of articles are coming from.
Java shops are certainly where I've witnessed the most disdain for Python. IME the strongest feelings tend to come from people who didn't actually have any significant experience with Python, and perhaps don't even have much practical experience with any language that isn't Java. So they tended to consider it to be objectively inferior purely because it's interpreted and dynamically typed.
At a previous job I did manage to put a chip in that when I demonstrated replacing one of our Java services with a Python implementation. It was a fraction of the code, and achieved much better latency and throughput. Obviously not every Python program is going to do that. But my point isn't that Python is better, it's that these kinds of things are never so cut-and-dried. Many non-trivial Python programs are just thin shells around a large core of very well-optimized and battle-tested C/C++/Rust code. And Java, for its part, can also accumulate a lot of dynamic language-style performance losses to pointer chasing and run-time type lookups (and GC churn) if you're not careful about how you use generics. As always, the devil's in the details. It's also less able to pull the "actually I'm a C++" trick because using a compacting garbage collector makes it difficult to interop with native code without paying a hefty marshaling tax.
I have way more experience with Python than Java. In at least 4 companies (including Google that I wouldn't call a javashop) we used mainly Python.
Still I believe Java is a better application language. Python is a better scripting language (replacement for Bash). Small apps tend to be easier on Python, but large apps are way easier on Java, both for syntax (types) and ecosystem (libs).
Mostly agreed, though I would add that I'm generally happier with Python as a default for reasonably sized services that don't have a lot of (non-numpy-friendly) compute load and therefore don't have a pressing need for multithreading. Which is a lot of what happens now that we're all trapped in the cloud. Like you say, small apps tend to be easier in Python.
That is general take as well. A lot of small apps/simulators are in python. Ops scripts tend to be python. Java for the core/data. Refactoring/tooling is easier in Java when you are dealing with a 100k codebase imo. Typescript always.
Seen plenty of coding horrors in both ecosystems...
Algorithms are a lot easier to understand when they are written in Python.
I'm actually right now documenting medium size Java codebase by writing pseudocode, which looks like Python.
Just by doing that, I've already discovered multiple bugs, which I didn't catch by looking at the Java code.
That might be the big thing that I think gets glossed over in a lot of these discussions. I agree that I wouldn't want to maintain 100kloc of Python. But, I don't really view that as a realistic hypothetical for a business application. Idiomatic Python tends to require a fraction as much code as idiomatic Java to accomplish the same task. The only time it even gets close is when you have code written by people who go out of their way to make things look like old-school enterprisey Java. So it ends up accumulating a bunch of stuff like
@dataclass
class IDontWantToBeABean:
field1: int
field2: str
field3: str
The worse case for Python is when you get people doing the oldschool Python thing of acting like dynamic and duck typing means it's OK to be a type anarchist. Scikit-learn's a good one to put on blast here, with the way that the type and structure of various functions' return values, or even the type and structure of data they can handle, can vary quite a bit depending on the function's arguments. And often in ways that are not clearly documented. Sometimes the rules even change without fanfare on minor version upgrades.
The reason why large Python codebases are particularly scary isn't necessarily the size itself. It's that for a codebase to even get that large in the first place it's very likely to have been around so long that the probability of it having had at least one major contributor who likes to do cute tricks like this is close to 1. And I'd take overly verbose like the Java example above over that kind of thing any day.
I wouldn't call it a new language, for me it's just a syntactic sugar over Java, but for any problem you would google "how to do X in Java", not "how to do X in Kotlin".
But there you can do way simpler syntax, like:
0..100 meters with -45..45 deg within 3 seconds
Because "0..100 meters ..." is equivalent to "(0..100).meters(...)"
(0..100) is a built-in IntRange type, that you can extend:
data class MyDistanceRange(val meters: ClosedRange<Double>)
val IntRange.meters: MyDistanceRange
get() = MyDistanceRange(first.toDouble()..last.toDouble())
IMO Kotlin has some features that take it way beyond just being syntactic sugar over Java. Like, I know you could probably use some eye-watering gang-of-four pattern to achieve the same effect as extension methods. But it's going to be such a PITA to maintain that calling the Kotlin feature syntactic sugar for the same thing is about as useful as saying that function definitions are just syntactic sugar over assembly language calling conventions.
I've played with it for a bit but it doesn't excite me enough to spend time learning it properly. I also don't care about Python beyond using it for learning algorithms and maybe some smaller Pyside UIs.
However, together with Mojo it might become more interesting again.
> Java, for its part, can also accumulate a lot of dynamic language-style performance losses to pointer chasing and run-time type lookups (and GC churn) if you're not careful about how you use generics.
The worst thing about Java is the average quality of Java programmer.The same could probably be said about Python. However I think that there are fewer Python programmers trying to write AbstractFactoryFactory than in Java. Java has a terrible culture of overly verbose, deep inheritance trees that make debugging and development worse, with worse performance.
I certainly think so but wasn’t sure if it was just my unfamiliarity or lack of advanced programming knowledge but I attempted what seemed like a very simple patch to a Java project (guacamole) and it was insane how I ended up having to add the new function to like a base and abstract class and interface etc. it was crazy. All the same boilerplate too.
I have a related feeling or prejudice that people not in the Java World view their corner as "the tech industry" without realizing that they are living in a village compared to the giant metropolis that is the Java World.
Java programmers may not blog as much, and Java doesn't show up on Hacker News as much, but not being Extremely Online does not mean that it isn't extremely widely used by real people whose experiences are just as valid.
you could say the same thing about a dozen other languages, but there is definitely a stereotypical cranky java dev that is frustrated writing corporate crud apps that should in theory be blazingly fast, but never quite are for reasons. They revel in the verbosity, and look down on anyone they feel is taking the easy path. If you've never met one you're lucky, but they've been at every company I ever worked for.
> you could say the same thing about a dozen other languages
No, you could not. You could say it about maybe four others: PHP, C, C++, and C#.
No other languages have anywhere near the userbase size while being fairly quiet when it comes to online tech discussion.
I agree there are crusty old Java devs (as well as crusty old C, C++, PHP, etc. devs). In a decade or two, there will be crusty old TypeScript devs. It's just the nature of technology lifecycles and career paths. Some people get relatively tired of learning when they get older and just want to capitalize on what they already know for their remaining earning years.
I think I made a conscious effort in my 30s not to become a crusty old Python dev. But I predict that in another decade or two, I (now in my 40s) will be a crusty old Rust dev.
ok I may have exaggerated, but there are quite a bit of people silently writing and maintaining mission critical code in Fortran, Lisp, Cobal, Perl, Pascal, R, Assembly, the different BASICs, Lua, TCL, Erlang, and the list goes on. Plus those are just the ones off the top of my head where I have personally met people doing it in the last decade. I am always shocked by how much
Most of those rarely make to the top of HN, and other generalized forums. If anything, Java and Python are together at the popular kids table and we forget about the silent heros keeping the ship afloat.
I read it less as "Java is tiny" and more as "Java developers can be peculiarly insular and conservative, by the standards of other communities."
Considering that as recently as 4 years ago I was working on a project where we still had a hard requirement to support running in Java 7, and this kind of thing was not considered unusual, I can't really disagree too strongly with that. Yes, that was still inside of Java 7's extended support period, so there was really nothing unusual or surprising about this, from a Java developer perspective. But that's kind of the point.
It's also not really a bad thing, considering what kinds of things run on Java. Mainframe developers have a similar thing going on, for a similar and similarly good reason.
I'm not sure if they refer to how large the language's reach is, but more about how advanced Java is once you stop looking at syntax bloat. The JVM can do stuff that's not easy to do in other environments.
I recall things like updating packages/code on the fly, recompiling fast paths on the fly. Maybe that's not necessary in a borg/kubernetes world where you can just restart things and have your load balancer infra take care, or just run slower code because compute isn't that expensive once you accelerate your CPU-heavy libraries, but cool anyways.
This is absolutely correct as someone who has worked in a few Java shops. Although, the same thing is Java's failing, as it is well nigh impossible for people external to the Java ecosystem to learn what's inside it without having work-related exposure.
You may know all this and are just singling out the Hacker News crowd. But I read your comment and thought "surely he doesn’t think Java is much bigger than Python?" I’m not even sort of sure Python is smaller.
The reason that people still code in Java (or derivative) is because legacy code that they are working on is in Java, and nobody has either the skill or time to go through and translate it. Which means that the jobs where its used are basically just big enterprise, low tech software that just been around for a while.
The Log4shell incident is the perfect demonstrator of what kind of people are in Java world.
> nobody has either the skill or time to go through and translate it.
To what? Java is still a very efficient, productive language. Updating a legacy codebase to use newer Java features would probably be good, but migrating to another language is unlikely to significantly move the needle in terms of runtime performance or developer velocity.
Most of the use cases for java are in places where network latency dominates. If Python is fast enough for Uber and Youtube, its fast enough for your service.
When you work with a codebase that doesn't need compilation, the development velocity is quite fast. You don't need to compile, you can prototype features on the fly, you can even write real time updates while the web server is running.
And standard compilation in Java is done in a VERY inefficient manner with groovy - you have groovy that gets compiled to bytecode, which then gets JIT compiled to native, which then actually runs the compilation of the source code to bytecode, and then you have a first startup latency as it gets JIT compiled to native.
All you really need to write any modern app is Python + C. Anything that needs to go fast, just encapsulate in a small C program, and launch that from Python.
I feel like the opposite is also true. People view Java as enterprise Java from 2008; a clunky outdated language for bad, verbose spaghetti code.
In fact, a lot of the most interesting plt and compiler r&d going into real world applications is on the jvm (project loom, graal etc), and the features of modern Java (pattern matching, records, etc) make it a great choice for lots of projects that aren’t legacy enterprise apps.
I gave a talk recently at a conference about how modern Java doesn’t suck very much, and that the worst part of Java is Java developers, who seem to be completely intellectually unambitious.
I don’t think Java makes anyone unambitious, I think it’s that Java is taught in schools and unambitious people don’t feel the need to learn anything else, and they get jobs at unambitious corporations. It selection-biases towards unambitious people who don’t want to learn more than they have to.
Compare this to something like Clojure or Haskell or something, which isn’t routinely taught at schools and is not very employable. People who learn these languages generally seek out these things because they’re interested in it. This selection-biases towards intellectually ambitious people.
As a result, Java people can be insufferable for people like me.
The people who make Java have actually made the platform and language pretty ok in the last two decades, but I had to fight at a previous job to use NIO, which I think was introduced in Java 4, but none of my coworkers had really heard of it or used it because the regular Java blocking IO has been “good enough”.
Given fact that Lombok is still pretty much widely used, with its under the hood functionality of essentially hacking the AST, or the fact that annotation processors write out Java code to files, or the fact that you could be using a standard library like Log4j and have a massive vulnerability on your system because someone decided that it would be a good idea if log statements could execute code and nobody said anything otherwise, or the fact that Kotlin and Groovy were literally made to address inefficiencies in Java, amongst other things....
Yeah not really sure how you came to that conclusion.
That gets to my point though. For example, Lombok isn't really necessary for a lot of stuff now, because Records give you a lot of what you would use with Lombok.
Kotlin and Groovy did come and address problems with Java, you should use use them if your employer allows it. I'm just saying that Java 21 is actually kind of fun to write.
Yes, some of the libraries have been unsafe, but that's one example of the 30 years of Java.
I just feel like Java has improved in the last twenty years. It's the engineers that haven't.
I'm afraid that my humor isn't really reflected in the slides, but imagine everything here is said kind of snarkily.
Java can be mostly as nice as Go, the BlockingQueues and Virtual Threads can get you pretty far, though they're not quite as nice as Go channels because there's no real way to select across multiple BlockingQueues like you can with Go channels.
Overall though, I think Java 21 is actually not too bad. Shockingly, I even sometimes have fun writing it.
> I have this feeling - or prejudice - that people still in the Java World are a bit out of tune with the tech industry
Other than being ageist, it’s wrong; or misattributed to Java. I work with Python every day, and what’s missing is static typing and IDEs that make use of it to greatly reduce the amount of code and context I have to store in my head. Python (a dynamically typed language obviously) is exhausting to maintain. But easy to write. Java/C#/whatever statically typed language with great IDE is easy to write and maintain by comparison.
Of course there are IDE for Python and dynamically typed languages, but everyone I’ve tried has fallen short compared to the best Java/c# IDEs.
Static vs dynamic used to be a huge flame war on the internet, but over the past few years I’ve encountered people who’ve never heard of it. This is it.
I personally dislike Python, but it does surprise me that anyone is acting like it's generally disliked because that has definitely not been my impression since time immemorial. I doubt that it's gone out of fashion. More likely, acting like something actually doesn't suck after all has become the clickbait framework du jour.
>A decade ago, Python was pretty rough around the edges in production.
Not really.
Environment/dependency management is/was never an actual problem. People act like you update a version and stuff breaks. Even then, venv existed for this reason, and you could specify version in the setup.py or requirements.txt.
Type safety should in theory be covered by unit tests. If you assign a variable to another one in a dynamic setting accidentally, like a string to a dict, then your functionality breaks. In reality though, its really not that hard to use different variable names. In my experience, the only time things start getting confusing is if you start using Java concepts with things like Factories and a lot of OOP and inheritance in Python. None of that stuff is really necessary, you can get by with dicts for like 90% of the data transfer. And in very type safe languages, you spend a lot of time designing the actual type system instead of just writing code that does stuff.
Performance is still slow, but language performance doesn't really matter - you can launch natively compiled code easily from Python. Numpy was built around this concept and is also 10 years old. You will never beat C (with explicit processor instructions for things like vector math) or CUDA code, but most of your code doesn't require this level of performance.
I haven't tried uv but my biggest pain point with Python has been the way env works. I always find it painful and odd.
Backward compatibility, which I suppose is closely related to needing to use env, is also a pain. In my experience you can't go forward or backward in many cases. It's especially painful on projects that don't change very often. I'm sure that an active codebase can probably be kept updated from version to version, but if you've waited a bunch of versions, it seems painful.
But, I'm not sure I've given it a fair shake because I haven't needed to. It's use in AI does make it an attractive option, now more than ever.
- statically typed languages got better, reducing the relative benefits of dynamic typing. people realized they didn't really hate static types, they hated the way [insert 90s-00s enterprise language here] did static types
- the GIL became more and more painful as computers got more cores
- it used to be a language for passionate hackers, like a latter-day lisp (cf that old pg essay). "written in python" used to be a signal of quality and craftsmanship. now it's the most commonly taught beginner language; millions of bootcamp devs put it in their CV. average skill level plunged.
- PSF was asleep at the wheel on the packaging / tooling problems for years. pip/venv are dinosaurs compared to cargo, nix, or even npm.
I don’t know anything about go. But Rust is more of a competitor to C and C++, right? It is sort of bizarre if these languages are butting heads with a scripting language like Python.
Python compares fairly well to Bash or JavaScript or whatever, right? (Maybe JavaScript is better, I don’t know anything about it).
I think for a lot of us, it was very frustrating when it was being pushed with no real analog for the practices we had in other languages. Poetry used to be pushed as how to do dependency management, but it was not obvious that it was not necessarily your build manager, as well. Even today, I'm not entirely clear what the standard approach is for how to manage a project. :(
- Performance: uv is so much faster that some operations become transparent. poetry's dependency resolver is notoriously slow. uv being a native binary also means it has a faster startup time.
- Interpreter version management: uv can handle separate python versions per project automatically. No need for pyenv/asdf/mise.
- Bootstrapping: you only need the uv binary, and it can handle any python installation, so you don't need to install python to install a management program.
- Environment management: uv will transparently create and update the environment. You don't need to source the venv either, you use `uv run...` instead of python directly.
Overall, it makes common Python project management tasks simple and transparent. It also follows standards (like project definition metadata and lock files), which poetry often doesn't.
I'll pay attention to poetry soon. As is, I don't recall my builds ever going slow because of poetry. I don't think I've noticed it have any impact on speed, at all.
I did just update the dependencies of some of my projects. I could see how that could be faster. I don't do that often enough for me to care about it, though. `poetry run pytest` is the slowest thing I have, and I'm confident most of that slowness is in my direct control already.
I'm intrigued on the lock file point. I thought that was literally one of the main reasons to use something like poetry, in the first place? Does uv have a better lock file mechanism?
uv supports the standardised lock file format. This means that you're not tied to uv for anything, as most project settings it uses (eg metadata, dependencies, extras, etc.) are based on PEPs. poetry has its own format for many things.
Thanks! I'm not entirely sure I see a reason to change on there, oddly.
I'm very fortunate that my python projects are all relatively small, so maybe that colors things a bit. Certainly looks like something that would have swayed me to uv at the start, but as things are, I think I mainly just wish there was a more standard/accepted work flow for build and release.
Makes sense. I've stopped using python for development but I run into python utilities I want to use all the time. uv/uvx are perfect for someone like me.
I coded happily in python for many years and fell out of love with it.
It doesn’t ship with a first party package manager so you got the community trying to fill this gap. Use any other language with good tooling like golang or rust and it is a breath of fresh air.
Python used as an actual PL is a footgun because it’s dynamic scripted. (Don’t tell me about using tools X, Y, Z, mypy, …) You essentially become the compiler checking types, solving basic syntax errors when uncommon paths are executed.
A programming language that’s only good for < 100 line scripts is not a good choice.
I honestly wish python were in a better state. I switched to rust.
> Don’t tell me about using tools X, Y, Z, mypy, …
What's wrong with using tools that improve on common issues? I don't think I'd use Python without them, but ruff and pyright make Python a very productive and reliable language if you're willing to fully buy into the static analysis.
> A programming language that’s only good for < 100 line scripts is not a good choice.
What a bunch of crap. It's so trivial to show very popular and useful programs written in Python that far exceed this number I'm not even going to do the work.
Hi rsyring, I made this comment out of experience.
As python projects grow and grow, you need to do lots of support work for testing and even syntactic correctness. This is automatic in compiled languages where a class of issues is caught early as compile errors, not runtime errors.
Personally I prefer to move more errors to compile time as much as possible. Dynamic languages are really powerful in what you can do at runtime, but that runtime flexibility trades off with compile time verification.
Of course, every project can be written in any language, with enough effort. The existence of large and successful python projects says nothing about the developer experience, developer efficiency, or fragility of the code.
All perfectly valid perspectives and I agree with most of what you wrote. But the comment above is pretty different from the tone/effort behind the comment I took issue with. :)
In hindsight, I should have just left it alone and not replied which is what I usually do. But Python's popularity isn't an aberration. It's tradeoffs make sense for a lot of people and projects. The low effort bad faith swipes at it from subsections of the HN community got me a bit riled today and I felt I had to say something. My apologies for a less than constructive critique of your comment.
> Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.
> Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
> When disagreeing, please reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3."
> Please don't fulminate. Please don't sneer, including at the rest of the community.
> Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
There are massive numbers of devs who would never even think of trying to code in a dynamically-typed language, even though the big players all have gradual typing. They don't know what they're missing.
I have no problem with dynamically-typed languages, my main problem with Python is the significant whitespace. I really do not like it. I can deal with everything else in Python, or any other programming language, but significant whitespace is what kills it for me.
Even after writing tons of python the whitespace is still the reason why I never reach for it by default.
Today the IDEs got much better, but I still can’t see the significant whitespace as anything than downside. With brackets I can indent, but also have automatic indenting succeed every single time.
Having gotten back into Python for a project after some time away, I found the solution (for me) is to automatically fix the whitespace every time I run the code.
I just have a line in my Justfile that does this. Probably would be better to format on save but I use different editors and haven’t gotten around to it.
Still doesn’t fix the creeping doubts about everything in a language conceived by people who made that whitespace call, but it removes the specific pain point.
do you mean requirement of whitespace or how much space that takes up? Because you can change how many spaces python needs for its indentation to get some reduction in whitespace
I've been writing code for 40+ years. I know how to set up my editor, thanks.
If you copy and paste some Python code and it isn't indented properly, it breaks the python program. It's the stupidest thing I've seen in any language, including javascript (which isn't as stupid as many claim it is).
Yes, I'd say Python's crown as the go-to lightweight scripting and web development language was mostly ceded to JS.
It's still reigning champion of data science, and of course it has a huge number of uses and users still around, but it's not really cool or in vogue outside of those circles.
Python has never been THE web development language, that's always been JavaScript. Python is one of the more popular server-side languages, and JavaScript has move from frontend to backend in the last decade too. Not sure whether this has really taken the crown of any backend-language.
And what is lightweight scripting? Isn't scripting by definition lightweight?
But Python overall is still very popular outside the Data Science-circles. Not sure where this claim is coming from.
> It's still reigning champion of data science, and of course it has a huge number of uses and users still around, but it's not really cool or in vogue outside of those circles.
This is a wild take. You're never going to get a fully accurate measurement but every source I've seen[0][1][2] puts Python as the most common programming language by a country mile.
If it doesn't seem "cool" or "in vogue", that's because everyone is already using it.
I mean if its really light weight, and I want to avoid bash, then JS is not too bad. The downside being that the built-ins are not really there compared to Python.
After using Rust's tooling, Python's tooling is obscenely painful. Like intolerable. Which makes some of us feel overwhelmingly frustrated with the language itself. The amount of time I've squandered on Python because of its tooling... I'm never getting that back.
But then uv came along. It's not just another poetry or pipenv or whatever. It works well and it has uvx and `uv tool install` and other nice things.
Previously when I saw that something was written in Python I'd curse under my breath because I don't have the mental energy to make a virtual environment and source its shell script and remember where I put it just to try something out. On arch Linux I can't just pip install something without going through some crazy setup that I used to have patience for but as I get older it's been better to just curse that snake language altogether and spend time doing something more chill like writing rust.
But now I can just type "uvx" and it works. I'm probably not going to start writing python again any time soon, but at least now I have less reason to be disappointed with people who themselves choose to write Python.
Can we please stop reiterating the same old stories? That was years ago. Most Python devs these days never wrote a single line of Python 2. I learned it 16 years ago and, while frameworks like Django were still in the middle of the transition back then, I pretty much started learning & writing Python 3 right away.
Hah that brings back memories of me smugly insisting to my freshman roommate that using Python 3 tutorials to learn programming was a complete waste of his time and he should be using 2.7 for life like the rest of us l33t hax0rs.
Tbf at that point Django was still pretty shaky with 3 and basically none of the 3rd party Django libraries supported it at all, plus I was using Google AppEngine which at the time was tightly coupled to the 2.7 runtime. But really I was just parroting the Slashdot hivemind which was 100% convinced the transition was the new Perl 6 and would kill Python, and that Python.org was dishonestly teaching newbies who didn't know better a dead language and worthless skill when they changed the default.
Fortunately for him he ignored me and most of the big Django libraries were ported like a year later at which point I had to switch anyway to get updates. Fully agreed that in 2025 it's pretty much irrelevant, and honestly despite some legitimate pain the transition was much more successful than the cynics assumed it would be at the time.
Python is probably too awesome and versatile to go out of fashion, but as a long time user.. the ecosystem is frustrating. Dependencies and packaging have been, and still are a nightmare after thousands of years. Problems are fixable sure but the thing is that they never end. After you get plenty of practice fixing all the related problems, you'll have to keep fixing them for yourself, your less savvy teammates, and in third-party code pretty much forever. This is worth it in exchange for "import antigravity" for the first 100 years, but it's frustrating eventually.
Everyone will mention uv/pyenv/poetry/conda/virtualenvs, so fine, let's pretend it's not a problem that you tried each of those in desperation and they are all household names. Suppose the packaging wars actually ended and everyone uses what you use without you needing to tell them, and suppose further that every pypa problem isn't blaming debian maintainers for obvious regressions. Pypi will still yank[0] packages at the source to perhaps randomly break deterministic behaviour, smashing anything in the blast radius rather than only clients using a --strict flag or something. You can pin your own dependencies but who knows what they will pin (nothing, or the wrong stuff, or the wrong way probably!) or what they will yank. Now for repeatability you need to host a mirror for everything you use- which is fine for corporate but a nonstarter for the novice and annoying for FOSS projects.
If you have zero dependencies or a slowly changing environment that is receiving constant care and feeding, you'll probably never notice how bad things are. If you put down most projects for a month though and pick them back up, perhaps with a different environment, machine, or slightly different python version it's broken, bitrotted, and needs serious attention to be rehabilitated.
People might argue.. that's just software dev. Nope. I say it with lots of love and gratitude for the efforts of the community.. but most langs/ecosystems would never tolerate this level of instability. One has to eventually just admit that working, portable, and actually repeatable environments with python basically just require docker. Can we talk about how "--break-system-packages" is hilarious? After you retreat to docker you can type this a few times a day, push the container up, pull it down months/years later, and then realize that literally the only way to get a stable working environment is to request a broken one. QED
I think during the python 2-> 3 transition period (which was almost a decade long) a lot of people started looking into alternatives, because frankly that mess was the last straw. In the scientific domain this is when Julia started growing.
Of course then crypto bros happened and the rest is history.
I love writing Python. However, there are caveats:
1: I don't like dealing with language crossing boundaries with it - it's painful, especially if to/from a compiled language - there's just too much friction. It's easy to write but hard to maintain.
2: Debugging python can be...painful. If you have a pure perfect environment dedicated to pure python and all the tooling set up, it can be breezy. But if you have something messier like C++ that calls Python or vice-versa and are using an editor not made for Python like QTCreator then suddenly it becomes hell.
Who owns this data!? Where was this initialized!? What calls this function!? What type is this data!?!?!?!? These questions are so effortless in C++ for me and so very painful in Python. It slows efforts to a crawl.
It feels exhausting to have to go back to print statements, grep, and hand-drawing a graph that shows what functions call what just to figure out WTF went wrong. It's painful to a degree that I avoid it as much as possible.
...and don't even get me started on dealing with dependency problems in deployments...
I've long lusted after a smooth debugging workflow writing Cython that would let you seamlessly debug between Python and C/C++. I think there are some hacks to do it, but nothing really well integrated.
Python debugging is great when you're strictly debugging Python code. Placing a `breakpoint` or spinning up an IPython instance to investigate the issue via `import IPython; IPython.embed()` makes life breezy. For neovim, you can send code to the ipython REPL easily via vim-slime.
I've never found myself debugging the underlying C++ code unless developing a C++ extension. But is it really that hard? Just point lldb to the python process and run your script in lldb.
If the C++ is not yours & assuming it's a mature lib (e.g. PyTorch): it's probably an error caused by you in Python land.
Both the C++ and Python are mine. I have cases of both C++ calling Python, and Python calling C++. Problems could be on either side.
The problem is how to step into one from the other, you really can't as far as I know.
The python parts aren't standalone enough that it could just be run on its own, there's so much setup involved prior to that point that it can really only be run from the top-level user controls.
I really don't find the Python language elegant at all. I prefer the Ruby syntax.
But Python's tooling, particularly with what Astral is doing (uv, ruff, ty), is so good, I'm always using Python and virtually never using Ruby. And yeah, the rich libraries, too.
> And guess what’s the de facto programming language for AI?
I thought nodejs/typescript seemed to be the default that most LLMs choose? Or is that just v0/lovable/replit? (although replit seems better about going non-js sometimes)
For the last 20 years that has been the mantra. Some X "solves" all the problems.
Except it doesn't. It just creates another X that is popular for a while, and doesn't somehow retroactively "fix" all the chaotic projects that are a nightmare to install and upgrade. Yes, I understand people like Python. Yes, I understand the LLM bros love it. But in a real production environment, for real applications, you still want to avoid it because it isn't particularly easy to create robust systems for industrial use. You may survive if you can contain the madness in a datacenter somewhere and have people babysit it.
Solving it at least involves the Python maintainers making a choice, integrating it into the Python binary and sticking to it. At least. But that requires possibly annoying some people until a) whatever solution becomes mature and b) people get over it.
I've ignored the trends and just used the bog standard requirements.txt with pip and a virtualenv and have had no problems in the past 10+ years. Either way, you always want to catch production deploys if something breaks.
That is only true if you never reexamine the universality of your statement. I promise that it is possible to "solve" the mess that was Python's ecosystem, that uv has largely done so, and that your preconceptions are holding you back from taking advantage of it.
Here's the thing: there has never been a lack of people who have declared this problem as solved in the 20 or so years since Python started Poking its way into my professional life. (And for about 12-13 years before that I could gladly ignore it since nobody did much of anything in it). People have said this since the days of the Blackberry.
Multiple times people have explained why they think whatever they are madly in love with now is the definitive solution. And none of those times, over those couple of decades did it turn out to be true.
I understand that you are enthusiastic about things. I get it. But perhaps you might understand that some people actually need to see things stick before they declare a winner? I'm not big on wishful thinking.
I would take a look at uv adoption. That's what makes it different. It nails everything that all the other tools have done and it does it fast. So it's been what people have been using for a while now. Even poetry never seemed to get this ubiquitous of support.
Every single language enthusiast says that some of the biggest codebases in the world are whatever their favorite major language is. And here's the thing: it is completely irrelevant whether the codebase is small or large. What counts is what it is like to use and maintain programs.
Python isn't the only language that has poor tooling. C/C++ is even bigger than Python in terms of established code base, and its tooling is nothing short of atrocious.
What helps is people realizing where tooling and production readiness should be. They can learn a lot from Rust and Go.
The it's big so therefore it must be right argument is nonsense. Worse yet: it is nonsense that excuses lack of real improvement.
> But in a real production environment, for real applications, you still want to avoid it because it isn't particularly easy to create robust systems for industrial use.
This is silly and seems to discount the massive Python codebases found in "real production environment"s throughout the tech industry and beyond, some of which are singlehandedly the codebases behind $1B+ ventures and, I'd wager, many of which are "robust" and fit for "industrial use" without babysitting just because they're Python.
(I get not liking a given language or its ecosystem, but I suspect I could rewrite the same reply for just about any of the top 10-ish most commonly used languages today.)
I can get that Pythons not for everyone, it certainly has its flaws and maybe uv is just another transient solution which will come and go and others have. I might disagree, but I can accept that. What I can't accept is the idea that it should be avoided for real production environments, which is frankly a bit ridiculous considering all the real applications and real production environments running on Python.
There’s still 20 years of projects using everything that became before uv. They didn’t upgrade the moment uv came into existence. Data science-land still uses other rubbish too.
> They didn’t upgrade the moment uv came into existence.
There's also projects that can't use `uv` because it doesn't like their current `requirements.txt`[0] and I have no bandwidth to try and figure out how to work around it.
[0] We have an install from `git+https` in there and it objects strongly for some reason. Internet searches have not revealed anything helpful.
Unrelated to uv but the problem with having a git ref in requirements.txt is that pip will treat it as a fixed version so it will strictly narrow the other dependencies it can resolve, which gets exceptionally difficult to reason about once that packages also loads a package from another git ref. Throwing everything into codeartifact (or equivalent on other clouds) is better longterm.
Yes, I've never really understood the complaint about python packaging - building native code is not something that is ever easy to guarantee across multiple distributions and operating systems.
Those native packages can be in any language and require any odd combination of tools to build. Who has truly solved that problem?
If you got object files you can compile whatever if you have the compiler toolset.
I have made a go program that links ffmpeg and is built from linux to Windows and Macos. It was not super easy but it's doable.
flatpak, docker ? i.e. include an almost complete distribution just to make one package work? but what if you want that package to work in the distribution you have now? What if you need 2 different packages that are in different flatpacks or different docker images?
Well, if you're using GRPC anyway, and have the protocol buffers already... It's hard to resist the temptation after a few hours of installing broken package versions.
Incidentally, I suspect this is the spiritual origin of microservoces...
obviously don't try those first I'm just saying there's a stack of options to work with for any level of isolation you desire.
start with wheels if you just want pre-built binaries,
move up to conda if you need arch and OS-specific C libs that depend on system packages,
flatpack or docker if you dont want those to mess up your host,
or unikernel/firecracker/VMs if you need kernel modules or hardware virtualization.
I'll throw in BLAS and LAPACK. Fuck, what a nightmare it always has been to get scipy running. I've always ended up with a wild mix of conda and pip installed shit that just wouldn't work.
I expect that for every two python users who know what `pip` is, there are three who are discouraged from thinking about the environment at all. Given that, focusing on the language and not the packaging is the right choice. Packaging is most often somebody else's problem.
It's just HN users are more likely to be that somebody else. Probably we have to deal with non-python dependencies anyway so we're reaching for bigger hammers like docker or nix. It would be nice if there wasn't a mess of python package managers, but whichever one I use ends up being a somewhat unimportant middle-layer anyway.
Going to be honest: With uv it really isn't that bad anymore. Sure, the whole packaging ecosystem is still really bad, but uv abstracts over most of it. Unless you're doing some really esoteric stuff you'll be just fine.
Using a nix devshell has worked out well for scripts for me so far. I haven't figured out what that workflow is like for larger projects though. I'm not interested in learning Uv
As a nix user, you hardly need to know uv, but you might later need to know how to work together with non-nix-users who use uv. It's easy.
There will be a file: uv.lock You can use uv2nix to get a single nix package for all the project's python dependencies, which you can then add to your devshell like anything else you find in nixpkgs (e.g. right alongside uv itself). It ends up being two or three lines of code to actually get the package, but you can just point a LLM at the uv2nix docs and at your flake.nix and it'll figure them out for you.
Your devshell will then track with changes that other devs make to that project's dependencies. If you want to modify them...
edit pyproject.toml # package names and versions
uv lock # map names and versions to hashes (nix needs hashes, finds them in uv.lock)
nix flake lock # update flake.lock based on uv.lock
nix develop # now your devshell has it too
This way you're not maintaining separate sources of truth for what the project depends on, and the muggles need not think about your nix wizardy at all.
You created a place where the interaction between nix devshells and uv was relevant, I put something there to help passers-by that might be interested in those topics. It wasn't actually for you.
It's like the (usually) interpreted equivalent to C/C++. There are lots of 'standard' package management choices.
And it seems like the package resolution is finally local by default, although that requires a 'virtualenv', which seems to be a legacy of the global packaging system.
> I started to code more in Python around 6 months ago. Why? Because of AI, obviously. It’s clear (to me) that big money opportunities are all over AI these days.
I find this depressing. Not only are LLMs covertly reducing our ability to think and make decisions, they’re now also making people voluntarily conform to some lower common denominator.
It’s like humanity decided to stagnate at this one point in time (and what a bad choice of point it was) and stop exploring other directions. Only what the LLM vomits is valid.
I have been enjoying Lisp languages since the late 1970s, and today it makes me happy using Common Lisp and Racket in the same way as when I stood in a forest very early this morning drinking coffee makes me happy.
But, Python is also a fun language and phrases like “good enough” and “actually liking it” are valid.
Agree. I think it's improved a bit but Celery is frustrating as the defacto job/queue solution. A lot of the defaults make it unreliable (it can lose jobs if workers crash or don't shutdown cleanly)
I'm hoping the existence of free-threading will push for more first-class concurrency primitives. Concurrent Futures is nice until you need a concurrent-safe data structure besides a queue
Agree that celery is a mess and it doesn't work well with async (Asyncio) python. I think version 6 maybe will support it sometime.
I also had a lot of problem due to async primitives with sqlalchemy - there's some tricky stuff with asyncio.gather vs TaskGroup and how sqlalchemy session works with it to be able to compose code easily.
There is nothing more annoying than tons of little repos all of which containing tiny projects with a few hundred lines of code but (of course) you need most / all of them to do anything. Use a mono repo until there is some obvious reason to split it up imo.
in my experience, repo by "area" works the best. This usually means by team, but you don't want a team restructure to cause code relocation issues. And, yes, only when the org pressures push for it.
On the flip side, we have an org with 50+ teams and our operations team is pinning for a monorepo. They are just fine with one team's push forcing N teams to have an unexpected deploy and recycling of caches, connections, etc. Not to mention what will happen when team A doesn't have time to deal with team B's merge due to other org pressures.
I keep meaning to write something like this but exploring the “how simple can you make it” angle - a lot of my world is kube (which is great in the right scenario) but could you shave complexity out of a stack designed for solo dev rapid iteration:
e.g. rather than:
> It’s important not to do any heavy data processing steps in the project-ui … we keep the browser application light while delegating the heavy lifting and business logic to the server
Chomp the complexity, serve HTML from the backend directly
> ty
Im curious where ty goes but for a min-complexity stack i couldnt spend complexity tokens on pre release tools
> pydantic … dataclasses
One or the t’other, plus i’ll forever confuse myself: is it post_init (dataclasses) or is it post_model_init (pydantic) - i had to check!
> docker
if we already have uv, could we get away without docker? uv sync can give an experience almost akin to static compiled binaries with the right setup. Its not going to handle volumes etc so if you're using docker features, this concept isnt going to fly. If you're not wedded to docker though, can you get away with just uv in dev and prod? in an enterprise prob not, i wouldn't expect to be able to download deps in prod. For flying solo though…
> compose
You’ve a frontend, a backend and presumably a database. Could you get away with just uv to manage your backend and a local sqlite db?
So a broadly feature comparable stack for rapid iteration with less complexity but still all the bells and whistles so you dont need to cook everything yourself, might look like:
You could probably sketch a path to hyper scale if you ever needed it:
- v1 = the above stack
- v2 = swap sqlite for postgres, now you unlocked multiple writers and horizontal scaling, maybe py-pglite for test envs so you can defer test-containers adoption for one more scaling iteration. WAL streaming would add some complexity to this step but worth it
- v3 = introduce containers, a container registry and test-containers. I dont think you really unlock much in this step for all that additional complexity though…
- v4 = rke2 single node cluster, no load balancer needed yet
- v5 = scale to triple node, we need a load balancer too
- v6 = add agent nodes to the rke cluster
- v7 = can you optimise costs, maybe rewrite unchanging parts in a more resource efficient stack
…
i experimented with it, i’m a fan of the concept. Ironically the bit i thought id like most (swap jinja extends shenanigans for just composing functions or callables more generally) is the bit that i didnt warm to. I dont really know why, on paper it ticks boxes for me. In practice i felt slow.
Java, JavaScript, and R, according to a footnote in the article. Which - to be entirely honest - sharply coloured my view of the preceding article.
"I went from being fully covered in mud to only being half covered in mud, and it's great! I don't understand why people complain about being half covered in mud."
There are many jobs where the .Net/Java/Go triad would be in the top 3. I lump them together as typed languages that are mainstream, performant, have a GC and a featured library.
Unfortunately, might not be easy in practice. It took years of effort to pass some certifications and audits and I don't think adding another language could be recouped now
I came to this comment thread expecting developer bikeshedding and I was not disappointed.
More seriously: it's fascinating and interesting to see how closely this article mirrors my own Python project layout. The tools and practices have come a long way over the last decade and good developer standards have a way of becoming the defacto in organic fashion.
IMO uv has won the race to be the new standard for Python environment management (Poetry gave it a solid try, but lost on speed).
“I started to code more in Python around 6 months ago. Why? Because of AI, obviously. It’s clear (to me) that big money opportunities are all over AI these days. And guess what’s the de facto programming language for AI? Yep, that sneaky one.”
Because you get first class support in many AI libraries like PyTorch, TensorFlow and so on ... Most of these libraries can be used from other programming languages too, but it is easier to find good documentation and examples for Python.
Possibly because many available LLMs run your Python code in a sandbox, which means less friction for vibe coders, or it may be a contributing factor at the very least.
How do you know that it is not the reason or not ONE of the reasons? Seems pretty reasonable to me to use ChatGPT, Claude, or whichever one supports it.
I am pretty sure some people (maybe this individual, too) may be using Python because their scripts can be executed in a sandbox on one of these websites.
Heck, if it was as good at Factor or Forth as it is at Python, I would be writing more of them, too.
In any case, you cannot claim that it is not one of the reasons. Can you?
I read your initial message, now edited, as this is THE reason. Of course it can be one of the reasons for which the author chose Python.
Also the vibe coding part gave me the impression that you were implying that people that use/chose Python for AI are all vibe coders which is again false. Sorry if I misunderstood you, but this is what I got from your initial message.
No worries. I think there was a misunderstanding because even with my original message, I did not intend to suggest that people who use or choose Python for AI are all vibe coders, or at least I didn't think I did hint at that.
This. You also don't really need Pydantic unless you're de/serializing data from external sources. A dataclass is perfectly cromulent if you're just passing around data internally.
Unpopular opinion: I think I’m going to wait for version 4 /jk. But honestly, I’ve been spoiled by modern languages like Rust, Go, and even TypeScript with modern tooling, strong typing, stability, and performance out of the box. Right now, I’m just interacting with LLMs, not building them.
That said, I remember writing myself a note a few years ago to avoid Python projects. I had to clean up code from all over the company and make it ready for production. Everyone had their own Python version, dependencies missing from requirements.txt, three way conflicts between 2 dependencies and the python version, wildly different styles, and a habit of pulling in as many libraries as possible [1]. Even recalling those memories makes my stomach turn.
I believe constraints make a project shine and be maintainable. I'd prefer if you throw at me a real python instead of a python project.
[1] Yes, I'm aware of containers, I was the unlucky guy writing them.
Much more capable and reliable type system, paired with comparatively sane package management. This is getting better in the Python world (thank you Astral), but it's still not anywhere near the same.
The principle of uv as much as I understand is to not tinker with virtual environments. With commands like uv run, uv sync etc you shouldn't have to worry about the virtual env and whether it's activated etc. if you are in a project, it will automatically pick up the right Python version . However if you want to do something not possible through those commands you can still get down to the virtual env. Plus for auto complete in IDEs you generally need to select the venv
I use this vscode extension called "python envy" that changes venv automatically depending on which folder of a monorepo you are in, it's great! Wish vscode had it built-in
There's only one kind of Python virtual environment, and both uv and the standard library `venv` module make them (as does third-party support like `virtualenv`). The differences are in what gets put in them (aside from the actual packages you explicitly install), and in the interface for configuring that. In particular, the standard library defaults to bootstrapping pip, but you can easily skip that. And of course uv does not bootstrap pip, because it does pip's job (and much more).
If you are working in Data Science/ML its the best bet for handling dependencies in your project compared to the rest of the tools. Its use has exploded, especially because you can do ‘uv pip whatever’ if you insist on using pip.
IMO "pythonic" is mostly about using the features the language has rather than trying to treat it as if it was Java or C++.
Having said that, in reviews you do get lazy comments like "not pythonic" or "not best practises" which often boil down to the reviewer just not liking something and being too much of an *** to say why. This is supposed to be a total shutdown that you cannot argue with and it's the kind of thing that might put you off the term "pythonic" for life.
"There should be one-- and preferably only one --obvious way to do it."
This is probably the core thing you might have issue with but I think its not really about conforming in how you write your own code but about the thing you make being easy for others to use.
I only Ruby shop I've ever worked at liberally used Rubocop to undo TIMTOWTDI. I suppose you could _write_ whatever you wanted but Rubocop would pretty aggressively rewrite it.
Just a small note on the code in the linked script:
Presenting the user with "Missing X OR Y" when there's no reason that OR has to be there massively frustrates the user for the near zero benefit of having one fewer if statement. Way better user experience, 0.00001% slower dev time.This is nitpicking, but this is a good usecase for the := operator:
For internal tools I just let os.environ["API_KEY"] raise a KeyError. It's descriptive enough.I write a decent amount of Python, but find the walrus operator unintuitive. It's a little funky that API_KEY is available outside of the `if`, perhaps because I had first seen the walrus operator in golang, which restricts the scope to the block.
This isn't really unique to the walrus operator, it's just a general python quirk (albeit one I find incredibly annoying). `for i in range(5): ...` will leave `i` bound to 4 after the loop.
Oddly enough, "except" variables don't remain bound!
So, it appears Python actually has three variable scopes (global, local, exception block)?Nope, it's more complicated than that:
It's not a scoping thing, the bound exception variable is actually deleted after the exception block, even if it was already bound before!lol - raku maybe weird, but at least it has sane variable scoping
Also true of JavaScript pre-ES5, another language that on first glance seems to only have function scope: it actually does have block scope, but only for variables introduced in `catch` blocks. AFAIU that was the standard way for a dumb transpiler to emulate `let`.
I wonder if that was ever popular, considering the deoptimization effects of try/catch, and given that block scope can also be managed by renaming variables.
exceptions being the exception is funny somehow
very recursive
you can say that again.
Exception blocks don't create a different scope. Instead, the name is explicitly (well, implicitly but deliberately) deleted from the scope after the try/except block runs. This happens because it would otherwise produce a reference cycle and delay garbage collection.
https://stackoverflow.com/questions/24271752
https://docs.python.org/3/reference/compound_stmts.html#exce...
This is the type of things that make me roll my eyes at all the wtf JavaScript posts[0], yes there are a lot of random things that happen with type conversions and quite a few idiosyncrasies (my favourite is that document.all is a non empty collection that is != from false but convert to false in an if)
But the language makes sense at a lower level, scopes, values, bindings have their mostly reasonable rules that are not hard to follow.
In comparison python seems like an infinite tower of ad-hoc exceptions over ad-hoc rules, sure it looks simpler but anywhere you look you discover an infinite depth of complexity [1]
[0] and how half of the complaints are a conjugation of "I don't like that NaNs exist
[1] my favourite example is how dunder methods are a "synchronized view" of the actual object behaviour, that is in a + b a.__add__ is never inspected, instead at creation time a's add behaviour is defined as its __add__ method but the association is purely a convention, eg any c extension type need to reimplement all these syncs to expose the correct behaviour and could for funzies decide that a type will use __add__ for repr and __repr__ for add
> yes there are a lot of random things that happen with type conversions and quite a few idiosyncrasies... the language makes sense at a lower level, scopes, values, bindings have their mostly reasonable rules
The "random things" make it practically impossible to figure out what will happen without learning a whole bunch of seemingly arbitrary, corner-case-specific rules (consider the jsdate.wtf test currently making the rounds). And no, nobody is IMX actually simply complaining about NaNs existing (although the lack of a separate integer type does complicate things).
Notice that tests showcasing JavaScript WTFery can work just by passing user data to a builtin type constructor. Tests of Python WTFery generally rely on much more advanced functionality (see e.g. https://discuss.python.org/t/quiz-how-well-do-you-know-pytho...). The only builtin type constructor in Python that I'd consider even slightly surprising is the one for `bytes`/`bytearray`.
Python's scoping is simple and makes perfect sense, it just isn't what you're used to. (It also, unlike JavaScript, limits scope by default, so your code isn't littered with `var` for hygiene.) Variables are names for objects with reference semantics, which are passed by value - exactly like `class` types in C# (except you don't have to worry about `ref`/`in`/`out` keywords) or non-primitives in Java (notwithstanding the weird hybrid behaviour of arrays). Bindings are late in most places, except notably default arguments to functions.
I have no idea what point you're trying to make about __add__; in particular I can't guess what you think it should mean to "inspect" the method. Of course things work differently when you use the C API than when you actually write Python code; you're interacting with C data structures that aren't directly visible from Python.
When you work at the Python level, __add__/__iadd__/__radd__ implement addition, following a well-defined protocol. Nothing happens "at creation time"; methods are just attributes that are looked up at runtime. It is true that the implementation of addition will overlook any `__add__` attribute attached directly to the object, and directly check the class (unlike code that explicitly looks for an attribute). But there's no reason to do that anyway. And on the flip side, you can replace the `__add__` attribute of the class and have it used automatically; it was not set in stone when the class was created.
I'll grant you that the `match` construct is definitely not my favourite piece of language design.
> methods are just attributes that are looked up at runtime
At runtime when evaluating a + b no dunder method is looked up and there is no guarantee that a + b === a.__anydunder__(b) https://youtu.be/qCGofLIzX6g
What i mean with weird scoping is
I also dislike how local/global scopes work in python but that is more of a personal preference.I agree that that Javascripts standard library is horrible the jsdate.wtf is an extreme but apt example, IMO most of these are solved with some "defensive programming" but I respect other opinions here.
> And no, nobody is IMX actually simply complaining about NaNs existing
I watched many Javascript WTF! videos on youtube and NaNs and [2] == "2" were usually 90% of the content.
> Oddly enough
It's not that odd, since it's the only situation where you cannot keep it bounded, unless you enjoy having variables that may or may not be defined (Heisenberg variable?), depending on whether the exception has been raised or not?
Compare with the if statement, where the variable in the expression being tested will necessarily be defined.
> Compare with the if statement, where the variable in the expression being tested will necessarily be defined.
Ruby does this sort of stuff, where a variable is defined more or less lexically (nil by default). Python doesn't do this. You can have local variables that only maybe exist in Python.While somewhat true, what would this be bound to?
Well, after writing my comment, I realized that a python interpreter could define the variable and set it to None between the guarded block and the except block, and implicitly assign it to the raised exception right before evaluating the except block, when the exception as been raised. So technically, it would be possible to define the variable e in GP example and have it scoped to "whatever is after the guarded block", just like what is done with for blocks.
Is there any chance this would cause trouble though? Furthermore, what would be the need of having this variable accessible after the except block? In the case of a for block, it could be interesting to know at which point the for block was "passed".
So, maybe "None" answers your question?
The answer is: it is unbound. Intellisense will most likely tell you it is `Unbound | <type>` when you try to use the value from a for loop. Would it be possible that it could be default initialized to `None`? Sure, but `None` is a destinctivly different value than a still unbound variable and may result in different handling.
Are you saying that this should result in a name error?
I stand corrected, the exception case is definitely an oddity, both as being an outlier and as a strange behaviour wrt Python's semantics. Or is it a strange behaviour?
In the case of an if like in your example, no provision is made about the existence of x. It could have been defined earlier, and this line would simply update its value.
Your example:
Same with x defined prior: What about this one? On the other hand, the notation "<exception value> as <name>" looks like it introduces a new name; what if that name already existed before? Should it just replace the content of the variable? Why the "as" keyword then? Why not something like "except <name> = <exception value>" or the walrus operator?While investigating this question, I tried the following:
In any sane language it would, yes.
Heisenbug is the word you are looking but may not find.
I may be rusty, but wasn't there a "finally" scope for those situations?
edit: writing from phone on couch and the laptop... looks far, far away...
> `for i in range(5): ...` will leave `i` bound to 4 after the loop. reply
This "feature" was responsible for one of the worst security issues I've seen in my career. I love Python, but the scope leakage is a mess. (And yes, I know it's common in other languages, but that shouldn't excuse it.)
I would love to hear about the security issue if you're able to talk about it
I don't remember the exact details, but it basically involved something along the lines of:
1) Loop through a list of permissions in a for list
2) After the loop block, check if the user had a certain permission. The line of code performing the check was improperly indented and should have failed, but instead succeeded because the last permission from the previous loop was still in scope.
Fortunately there was no real impact because it only affected users within the same company, but it was still pretty bad.
Oof that's a near miss. That's the sort of hard-to-find issue that keeps me up at night. Although maybe these days some ai tool would be able to pick them up
I find it incredibly intuitive and useful that it does that. sometimes it drives me nuts that it doesn't do it for comprehensions but I can see why.
But if something fails in a loop running in the repl or jupyter I already have access to the variables.
If I want to do something with a loop of data that is roughly the same shape, I already have access to one of the the items at the end.
Short circuiting/breaking out of a loop early doesn't require an extra assignment.
I really can't see the downside.
Python 2 actually did let comprehension variables leak out into the surrounding scope. They changed it for Python 3, presumably because it was too surprising to overwrite an existing variable with a comprehension variable.
Oh wow, maybe that's why I expect it to work that way! I can't believe it's been long enough since I used 2 that I'm forgetting it's quirks.
That almost sounds like having the "variables" eax, ebx, ecx, and edx.
Oh yeah, that's a good point.
Python really is a bit of a mess haha.
I cannot tell you how many times I've hit issues debugging and it was something like this. "You should know better" -- I know, I know, but I still snag on this occasionally.
It would be utterly nuts otherwise. For loops over all elements in a sequence. If the sequence is a list of str, as an example, what would the «item after the last item» be?
the issue isn't the value of i, the issue is that i is still available after the loop ends. in most other languages, if it was instantiated by the for-each loop, it'd die with the for-each loop
Maybe Python will get a let one day
There's no block scope in Python. The smallest scope is function. Comprehension variables don't leak out, though, which causes some weird situations:
Comprehensions have their own local scope for their local variables, but the walrus operator reaches up to the innermost "assignable" scope.This is just Pythons scoping, which is not restricted by block, but function. You have the same effect with every other element.
Wow. I had been writing Python for 15 years and I didn't even know that operator exists
It's only existed for 6 of those years so perhaps you can be forgiven :)
The last time I wrote Python in a job interview, one of the interviewers said "wait, I don't know Python very well but isn't this kinda an old style?" Yes, guilty. My Python dates me.
I always forget this syntax exists. When was this introduced?
In Python 3.8 - https://peps.python.org/pep-0572/
The Walrus operator! Introduced in Python 3.8 according to the PEP 572.
Really nice to combine 1) checking if something exists and 2) act on it
You forgot to end with "goo goo g'joob" or the less correct Simon and Garfunkel version "coo coo ca choo".
And for lambdas
October 2019 in Python 3.8
https://docs.python.org/3/whatsnew/3.8.html
Why would you use it though? I always thought it a bad thing that legacy code (such as in C or C++) had these side effects inside of checks happening
Why are people writing Python like it's Go?
Letting the KeyError go is usually fine, but if you want to log it or can recover somehow then:
no one uses walrus
It seems that way! I briefly experimented with it when it first came out, but never used it in any production code. I've never seen it used by anyone else, either.
I find it interesting that none of the voluminous python code I've had AI tools generate has ever had a walrus operator in it. Reflects the code they're trained on, I guess.
I use it very often. Avoids duplication of expensive operations in comprehensions.
I use it fairly often
Even better, check all the conditions and report all the errors. Don't make me add the first variable, run it again and get another error.
Sometimes that's inevitable, bit noisy of the time it isn't.
Even better is to try to fetch all the env variables and then report on all of the missing ones.
Why not add a boolean flag and use a single exit(1) at the end? That way the user can see all the environment variables that have not been set.
Neophytes take notice. Attention to details like this is what separates truly great programmers from merely good ones. That said, for scripts reusable by others you should use command line arguments . Environment variables in lieu of command line arguments is a huge code smell.
You don't generally want API keys accidentally recorded into someone's bash history.
What exactly is your threat model, where the attacker can read ~/.bash_history but can't execute (or capture output from) /usr/bin/env?
CI and other build systems. Where tokens have been stolen in the past, by users not caring, and a VM not being properly cleaned.
The threat model is that history is persistent while the environment isn't. That said, whenever possible you should handle secrets using file descriptors as opposed to environment variables.
How about not accidentally showing tokens and credentials when live screen sharing?
You do NOT save passwords in shell history, it is insanely insecure. Lets begin with the passwords being readable by everything that can list tasks.
You can protect passwords in a password manager. You do not need to keep the passwords in env and I do not.
> Lets begin with the passwords being readable by everything that can list tasks.
Why are processes running that can do this, that I don't already fully trust?
> You can protect passwords in a password manager.
What's your plan for supplying the password to the program, given that people will want to automate use of the program?
Threat model: shell scripts
For this example, don't just command line arguments. There's an API key there, you don't want an API key visible in your cmdline.
Then how would you SET the API key in the first place? :) The argument doesn't make any sense at all.
Using read or an equivalent, presumably. Just because you don't know why a practice is recommended against doesn't mean that there isn't a good alternative.
In some .profile or .envrc or what you'd call such a file, I suppose.
And you expect someone will be able to read your bash_history, but not your .profile?
Fair, I wasn't thinking about the details of how someone would lift out the bash history.
Typer has a great feature that lets you optionally accept argument and flag values from environment variables by providing the environment variable name:
https://typer.tiangolo.com/tutorial/arguments/envvar/
It's especially nice for secrets. Best of both worlds :)
No, that's an anti-feature. :) Sibling comments here claim that command line arguments "leak" whereas environment variables does not. It's plain wrong. An attacker with access to arbitrary processes' cmdline surely also has access to their environ. Store secrets in files, not in the environment. Now you can easily change secret by pointing the --secret-file parameter to a different file. The only reason people use BLABLA_API_KEY variables is because Heroku or something did it back in the day and everyone cargo-culted this terrible pattern.
One could write a huge treatise on everything that is wrong with environment variables. Avoid them like the plague. They are a huge usability PITA.
This is bad advice. Please don't make claims about security if you're making it up as you go.
Environment variables are substantially more secure than plain text files because they are not persistent. There are utilities for entering secrets into them without leaking them into your shell history.
That said, you generally should not use an environment variable either. You should use a secure temporary file created by your shell and pass the associated file descriptor. Most shells make such functionality available but the details differ (ie there is no fully portable approach AFAIK).
The other situation that sometimes comes up is that you are okay having the secret on disk in plain text but you don't want it inadvertently commited to a repository. In those cases it makes sense to either do as you suggested and have a dedicated file, or alternatively to set an environment variable from ~/.bashrc or similar.
Good luck storing your private keys in environment variables!
Just use
It will print message to sys.stderr and do system exit with code 1.P.S. exit() is just an alias to sys.exit(), I prefer longer form.
Thank you. I see something like this all the time on one of the sites I use for work. If you fail the 2-factor, it'll tell you your password was wrong and reset the whole thing instead of telling you the 2-factor code was wrong or expired.
Even slightly better is to first check both
That way you don't end up fixing one just come back and be told you're also missing another requirementEven better would be to only check each once and buffer the decision:
This way you only check each value once (because your logic might be more complicated than just checking it's not set, maybe it can be wrongly formatted) and you still get to do whatever logic you want. It also removed the combinatorial problems.This is a pretty general principle of separating decision from action.
Taking ephemeral arguments like channel ID from the environment is more offensive to observability and user comfort
I'm surprised there isn't an argparse like thing for documenting expected environment variables.
That's basically pydantic settings: https://docs.pydantic.dev/latest/concepts/pydantic_settings/
It would be better to do both: print out the detailed string or strings, then exit if either are printed.
Or
Do you use parens with assert? In Python, the assert statement is not a function, right?
That bit me before... it created a tuple which evaluated as true.
Assertions are disabled via `python -O` so they probably shouldn't be used like this.
On the other hand, presumably the program will just crash anyway with some terse permission/request error later on if these values aren't set
Doesn't matter. Assertions should be used for things that are always true (barring bugs in the program). That isn't the case here.
I usually go through an array of settings that must be defined and list any missing ones, to avoid repeated attempts from the user/deployer.
I feel like use case and audience matters when making these decisions. In this case, the user is probably someone interacting with a python script they're running in a console (I assume by print), then I really don't think it matters - the user will check that both things are set. Should you also give them some documentation about setting env vars? Should you customize that documentation to the OS they're running? etc.
If the user is a typical consumer using a typical consumer interface, then yes you want to handhold them a bit more.
why print then exit(1) instead of raising an exception?
Oh, I know this one:
and that story gets _a lot_ worse when some programs raise from within a "helper" module and you end up with 8 lines of Python junk to the one line of actual signal> I would like to have a tool that generates the project structure for me, but I haven’t found one that fits me yet.
I recommend cookiecutter for this. I have a few templates I've built with that which I use frequently:
python-lib: https://github.com/simonw/python-lib
click-app: https://github.com/simonw/click-app
datasette-plugin: https://github.com/simonw/datasette-plugin
llm-plugin: https://github.com/simonw/llm-plugin
You can run them like this:
My take on this (using Ruby) is https://github.com/coezbek/baker
It doesn't copy template repos, but rather creates a list of imperative steps to perform. Steps can be both manual (obtain an API key and store it here) and automatic (run 'uv init'). Markdown syntax, ruby string interpolation and bash.
It came from a deep hate for yml based configs.
Copier is the new hotness for this
https://copier.readthedocs.io/en/stable/
I made a small wrapper for cookiecutter, to handle some minimal Git integration (making a repo and doing the first commit) and to match my workflow better (write some initial code first, then transform it into a "project folder" in place): https://github.com/zahlman/cookiebaker
I'm not a huge fan of cookiecutter on aesthetic principles, though. I think it's chosen some needlessly heavyweight dependencies for such a simple task. PyYAML comes with a big chunk of compiled C, while performance is really not going to matter and I'd rather use TOML anyway. I've yet to see a project depending on Rich that uses more than a tiny portion of it; this is no exception. More to the point, Rich brings in the much larger Pygments (for syntax highlighting; most of the bulk is actual syntax definition rules for a variety of programming languages) and you can't disable that. And honestly I'm not a fan of the whole "download other people's templates using an integrated client" model anyway; Requests and its dependencies are pretty bulky, too.
Am I the only one who actually likes setting up new projects? I don't want to automate that.
It depends on your work, though…
If you work at an agency or as a freelancer and you build various similar apps with similar tooling and base setup, being able to scaffold and have them all setup quickly, not having to do it manually and waste hours id important. Similarly, if you work on various small open source packages, you want the tooling to be the same, READMEs look the same, etc, a script or tool to “spit out” the basic structure can be nice.
On the other hand, if you set up the app or larger open source package and you’ll work only on that project for potentially years, setting up a project individually, organically makes a lot of sense.
I’m legitimately curious about what you enjoy and why. Rewriting boiler plate code and trying to ensure that I get everything right with a process I don’t often execute is the very definition of toil to me.
It allows me to gain a basic understanding of the fundamentals of my code base.
I looked at the click-app repo. If you were creating that today, would you switch from uv to pip?
And to run cookiecutter do you still use pipx, or have you switched to `uv tool install`
I'm getting close to switching to uv for my templates.
I use "uvx cookiecutter" myself three days.
This type of thing seems ripe for building into agentic LLM dev workflows, doesn't it?
Hmm never heard of this. Thanks for the recommendation.
Honestly, these days, just tell AI to generate one for you.
> Not only because the syntax is more human-friendly, but also because the Python interpreter is natively integrated in all Unix distros
That's kind of very optimistic evaluation - literally anything beyond "import json" will likely lead you into the abyss of virtual envs. Running something created with say Python 3.13.x on Ubuntu 22.04 or even 24.04 (LTSs) / Rocky 9 and the whole can of worms opened.
things like virtual envs + containers (docker like)/version managers become a must quickly.
“import json” is the kind of thing which requires picking and installing libraries in batteries-not-included languages, and it’s just one of many modules which are in the standard library. That’s not a compelling basis for large projects but over the years I’ve shipped a ton of useful production code which never needed more than the stdlib and thus spent no time at all thinking about deployment or security patching.
Also, it’s not the 2000s any more. Using venv to isolate application installs is not very hard anymore and there have been decent package managers for a long time.
The official package manager is pip, it's broken, and there has been a new "permanent solution" replacement for it each year.
“broken” is hyperbole. It works fine for millions of people every day. If you have some specific scenarios where you want it be better, it’s better to say those rather than just complain about an open source project.
Millions of people put it into Docker, or they just deal with it and you see the results with tons of Stackoverflow questions
> Millions of people put it into Docker, or they just deal with it and you see the results with tons of Stackoverflow questions
Arrogantly wrong.
I've coded in Python for almost 20 years. Many of those years I've had it as my primary language at work.
2024 was the first year I actually needed a virtualenv. Before that, I'd happily use pip to install whatever I want, and never had a version conflict that caused problems.
I often encounter junior folks who default to using a virtualenv, not because they need to, but because they've been brainwashed into believing that "you're doing it wrong if you don't use one".
In any case, people should use uv these days.
>2024 was the first year I actually needed a virtualenv. Before that, I'd happily use pip to install whatever I want, and never had a version conflict that caused problems.
Okay I'll bite: how did you deal with situations where you needed to work on two different projects that required different versions of the same dependency?
Some packages are written by researchers in fields that aren’t engineering. Sometimes you go all the way and reach for a VM.
I'm not denying that some need a VM or virtualenv. They exist for a reason.
I was talking about pip, not venv. I don't use venv either, not because I think it's a bad idea but because I can't be bothered. Stuff does end up conflicting unless I use Docker (lol) or uv.
If you use uv you are using virtual environments. You're merely being shielded from a few minutes worth of education about how they work (https://chriswarrick.com/blog/2018/09/04/python-virtual-envi...).
Well I also don't have to deal with the venv then, uv does it for me. Which I suspected is how it works, cause idk how else it would.
For some reason copying a venv directory causes problems with internal links. Fun all around.
Well that explains why you think pip is broken
Because I went to the official pip "getting started" docs and did exactly what it says? It's bad even in venv though, like not managing your requirements.txt or installing conflicting stuff. The web is full of questions about this, with answers that involve installing some other thing.
pip freeze
but yes, pip predates the paradigm of package managers managing your dependency file instead of you managing your dependency file and then invoking the package manager
As another commenter alluded: It's clear your knowledge of the topic at hand is poor.
It's wise to keep silent in such cases.
Oh yeah, pip is actually right, and everyone is using it wrong
The rest of the thread makes it clear that you expect a "package manager" to do more things than Pip does.
Pip is focused on installing the packages, not really "managing" them. The problem is that there are quite a few different takes on what "management" should entail. That's where all the alternatives are coming from.
Whichever you call it, at the end of the day it's more common to encounter dep problems in Python than with NodeJS, Rust, etc.
But also, ignoring things pip isn't meant to do like version management and just focusing on installing, pip's default of installing to some unclear system location was always confusing, particularly when Py2 vs 3 was a thing.
I don't get the python hate on here.
Pip is fine. It's been fine for at least the last 5 to 10 years.
I don't hate Python, I use it by choice every day. This is merely a downside.
Poetry has been decent for years. uv is new but great and will likely continue to be.
uv is soooo much faster than Poetry, especially for dependency resolution.
I would like to try uv, but I don't find Poetry that bad for dependency resolution times. If I have to wait 2 minutes once a week it's not the end of the world. Maybe if you're constantly installing new things that are tough to resolve it's an issue.
It's not the end of the world, but it's annoying. I used to work at a place that had a huge monorepo with tons of dependencies. Poetry would sit there grinding away for 4 or 5 minutes. With smaller projects, I agree it's not much of an issue. But, uv has other cool features and it's super snappy!
Not the end of the world but why wait 2 minutes when you can wait 2 seconds?
That's what I'm saying, not too long ago people were saying "just use poetry". Which, like uv, has its own lockfile format.
Did conda ever get faster?
I'm not arguing on venv to isolate installs, I'm saying that relying on Python coming from UNIX-like OSes is close to impossible for, at least web related projects. I.e. Ansible does a lot [in terms of what code it generates] to keep itself compatible with whatever Python3 versions it may find on systems ( remote hosts )
I have a silly theory that I only half joke about that docker/containers wouldn't've ever taken off as fast as it did if it didn't solve the horrible python dependency hell so well. You know something is bad when fancy chrooting is the only ergonomic way of shipping something that works.
My first taste of Python was as a sysadmin, back in 2012 or so, installing a service written in Python on a server. The dependency hell, the stupid venv commands, all this absolute pain just to get a goddamn webserver running, good lord. It turned me off of Python for over a decade. Almost any time I saw it I just turned and walked away, not interested, no thanks. The times I didn't, I walked right back into that pile of bullshit and remembered why I normally avoided it. The way `brew` handles it on macOS is also immensely frustrating, breaking basic pip install commands, installing libraries as commands but in ways that make them not available to other python scripts, what a goddamn disaster.
And no, I really have no clue what I'm talking about, because as someone starting out this has been so utterly stupid and bewildering that I just move on to more productive, pleasant work with a mental note of "maybe when Python gets their shit together I'll revisit it".
However, uv has, at least for my beginner and cynical eyes, swept away most of the bullshit for me. At least superficially, in the little toy projects I am starting to do in Python (precisely because its such a nicer experience), it sweeps away most of the horrid bullshit. `uv init`, `uv add`, `uv run`. And it just works*.
> I have a silly theory that I only half joke about that docker/containers wouldn't've ever taken off as fast as it did if it didn't solve the horrible python dependency hell so well.
I don't think this is a silly theory at all. The only possibly silly part is that containers specifically helped solve this problem just for python. Lots of other software systems built with other languages have "dependency hell."
Back in the early days of Redhat, rpm's didn't really have good dependency management. Yes there were rpms, yes you could download them, but getting the full dep tree was a PITA. Most people installed the full Linux distro rather than a lightweight version because of this.
Debian's apt-get was very "apt" at the time when it came out. It solved the entire issue for Debian. There was a point at which there was an apt-rpm for redhat. Yum tried to solve it for redhat, but didn't really work that well -- particularly if you needed to pin packages to certain versions.
I won't touch Python either, but because I've been burned debugging large Python programs. Something that would have taken a minute in a statically typed language took hours of tracing data through the program to understand what was supposed to be in a dict. There are alternative languages that are pithy, statically typed, can write programs quickly, and can grow into large code bases that are maintainable; so there is never a reason to start a new project with Python today.
I've seen the same thing in .NET and Java where there's 800 layers of interface and impl and it's an adventure trying to find the actual business logic in all the layers of indirection
>so there is never a reason to start a new project with Python today
Nothing else has an ML/data ecosystem that compares. Perl/Go are maybe a distant 2nd
I deal with that all the time cause an adjacent team uses Java with tons of boilerplate and frameworks. At that point, your static typing isn't so static. Takes them forever to make changes, to the point where I started taking over responsibilities using Python code.
Most python written at a large scale uses types (TypedDict) and/or Pydantic for safety and never plain dict objects. That's a code smell in any language, we can stuff data into `map[string]interface{}` all day long and cause problems downstream.
There's also dataclass in stdlib
It's fine if the author doesn't choose to do horrendous things. There are types too if you want them, but I don't.
> However, uv has, at least for my beginner and cynical eyes, swept away most of the bullshit for me.
uv is _much_ better than what came before. As someone who has only had only glancing contact with Python throughout my career (terrible experiences at jobs with conda and pip), uv feels like Python trying to actually join the 21st century of package management. It's telling that it's in Rust and clearly takes inspiration from Cargo, which itself took inspiration from Ruby and Bundler.
The funniest thing for me is that you can use uv in mise to install Python cli programs in a surprisingly elegant manner.
Just curious: how?
You can do it with various mise commands, but the simplest way is to use a mise.toml that looks something like this
These are all features of the [pipx backend] for mise, and the docs discuss what to do in the case of things like python updates, etc. The advantage of doing it this way, particularly for a global mise config, is that you treat these python tools as basically any other mise tool, so their versioning is easy to control.I know mise isn't really a package manager. But with its support for things like this, be it for python, ruby, npm, or cargo, as well as more universal support from things like Ubi and the upcoming github backends, its rapidly becoming my favorite package manager. I've a number of projects that use particularly useful node based tools, like markdown-lint or prettier, that aren't JS based in any way, shape, or form. Having to carry around a package.json felt weird, and with the way mise handles all of it, now I don't have to
[pipx backend]: https://mise.jdx.dev/dev-tools/backends/pipx.html
They may be referring to uvx: https://docs.astral.sh/uv/guides/tools/
Or it could be something else, not sure.
In mise.toml I have:
``` [tools] python = "3.12.11" ruff = "latest" ```
I get ruff installed and anything else needed without any fuss.
`uv tool install` I believe
> You know something is bad when fancy chrooting is the only ergonomic way of shipping something that works.
Every language seems to have this problem. Or else how can we explain the proliferation of AppImage, Flatpak, Snap, ... ?
Yeah, if you look at a big Python vs Node vs Rust repo side by side, you'll notice Python is often the only one with a Dockerfile.
Basically in the world of C static vs. dynamic linking. A trade-off as old as programming.
You should always use virtual envs. They're a single directory, how are they an abyss? Pip now complains if you try to install a package system wide.
For shadow IT, anything that requires "installation" vs. being in the base system or being added as a file to a project is an inconvenience.
It's why I like using Bottle for small Python frontends: download the file and import.
(I'm ranting based on personal experiences with IT in the past. Yes in general virtualenv is the baseline)
Compile python to executable is always an option.
If you're dealing with a managed system, chances are, compilers are banned. You'll have to be unsafe and work outside the constrained environment - potentially violating policies, contracts, regulations and laws.
> You should always use virtual envs.
If you're not using dependencies, and are writing 3.x code, there's very little justification.
Not using dependencies is rather niche. Most people come to Python because they want to use pre-existing libraries.
I don't have to deal with this in JS or I think in other stuff like Golang. I give someone a package.json with versions of everything. npm install always sets deps up locally, but doesn't need to copy the entire NodeJS runtime.
… node_modules is your venv.
If we use uv from TFA, like the commands are nearly 1:1:
> It doesn't have to also store a local copy of NodeJS… which Node devs do have a thing for, too, called nvm, written in bash.
The difference is it's default, it always works the same everywhere, it actually writes down the deps, and I don't have to manually switch between them (or set up fancy bashrc triggers) like venvs.
> it's default
This point is true; the ecosystem simply can't change overnight. uv is getting there, I hope.
> it always works the same everywhere
`uv` works the same, everywhere?
> it actually writes down the deps
`uv add` does that, too.
> I don't have to manually switch between them (or set up fancy bashrc triggers) like venvs.
You don't have to do that with `uv`, either?
Hope they make uv default then. It's nice, but I have to separately create a project with it, and regular python commands don't work with it, both of which go back to it not being default. But even that won't fix all the old projects.
node is not the default in js projects either, it's just the currently most popular manager. Old JS projects are their own bundle of fun.
Basically is by now. This is more of a recent thing for frontend JS, but NodeJS (which is more directly comparable to Python) had npm I think from the start.
Those browser JS libs installed via <script> tags though, honestly were pretty convenient in a way.
Yes, please use virtual envs or containers. I know it seems overly heavy and hard to manage, but you don't want to end up in a situation where you're afraid to update the system because the library upgrades might break some of your Python code.
+1 - I've been shocked at how many little portability issues that I've had shipping software, even within the relatively constrained environment of fellow employees.
Minor differences between distro versions can make a big difference, and not everyone that uses a Python script knows how to use something like pyenv to manage different versions.
Also in some older cases the pre-included Python is v2 and the system relies on it, which is more of a liability.
this is solved by uv
> things like virtual envs
I consider my point as still valid with UV, what you wanted to express?
On UV specifically - say 'asdf' compiles python right on your system from official sources - means using your ssl libs for example. UV brings Python binary - I feel worried on this.
uv works just as well with whatever Python you want to bring -- you're not required to use the Pythons that uv is capable of installing on your machine.
uv really is working super super super hard to absolve decades of sins and blood in the python world. and doing a good job at redemption.
Just use uv
I agree - Python without uv is masochistic. But that really negates the "Python is already available" advantage. If you have to install uv you can just as easily install Rust or Go or Deno.
More like a snake pit than a can of worms.
I agree that the built-in Python is typically not suitable for development, especially if you're planning to distribute your code and/or care about the versions of its dependencies. (And especially on Debian and its derivatives, in my experience; they even remove parts of the standard library, like Tkinter [0].)
I disagree that virtual environments represent an "abyss". It takes very little effort to learn how they work [1], plus there a variety of tools that will wrap the process in various opinionated ways [2]. The environment itself is a very simple concept and requires very few moving parts; the default implementation includes some conveniences that are simply not necessary.
In particular, you don't actually need to "activate" a virtual environment; in 99% of cases you can just run Python by specifying the path to the environment's Python explicitly, and in the exceptional cases where the code is depending on environment variables being set (e.g. because it does something like `subprocess.call(['python', 'foo.py'])` to run more code in a new process, instead of checking `sys.executable` like it's supposed to, or because it explicitly checks `VIRTUAL_ENV` because it has a reason to care about activation) then you can set those environment variables yourself.
Creating a virtual environment is actually very fast. The built-in `venv` standard library module actually does it faster in my testing than the equivalent `uv` command. The slow part is bootstrapping Pip from its own wheel - but you don't need to do this [2]. You just have to tell `venv` not to, using `--without-pip`, and then you can use a separate Pip (for recent versions — almost the last 3 years now) copy cross-environment using `--python` (it's a hack, but it works if you don't have to maintain EOL versions of anything). If you need heavy-duty support, there's also the third-party `virtualenv` [3].
Much of the same tooling that manages virtual environments for you — in particular, pipx and uv, and in the hopefully near future, PAPER [4] — also does one-off script runs in a temporary virtual environment, installing dependencies described in the script itself following a new ecosystem standard [5]. Uv's caching system (and of course I am following suit) makes it very fast to re-create virtual environments with common dependencies: it has caches of unpacked wheel contents, so almost all of the work is just hard-linking file trees into the new environment.
[0]: https://stackoverflow.com/questions/76105218
[1]: https://chriswarrick.com/blog/2018/09/04/python-virtual-envi...
[2]: https://zahlman.github.io/posts/2025/01/07/python-packaging-...
[3]: https://virtualenv.pypa.io/
[4]: https://github.com/zahlman/paper
[5]: https://peps.python.org/pep-0723
Many of the people I coached after they asked for help would have been absolutely flabbergasted by these suggestions.
Activating a venv was at least sething they could relate to.
No and no. I don't know how you even get to this level of "making it harder for yourself".
Say you want to use a specific version of python that is not available on Ubuntu.
1. Install build dependencies https://devguide.python.org/getting-started/setup-building/#...
2. Download whichever Python source version you want, https://www.python.org/downloads/source/. Extract it with tar
3. run ./configure --enable-optimizations --with-lto
4. run make -s -j [num cores]
5. sudo make altinstall
This will install that specific version without overwriting default system python.
You can then bash alias pip to python3.xx -m pip to make sure it runs the correct one.
All the libraries and any pip install executable will be installed locally to ~/.local folder under the specific python version.
Alternatively, if you work with other tools like node and want to manage different versions, you can use asdf, as it gives you per folder version selection.
virtual environments are really only useful for production code, where you want to test with specific versions and lock those down.
Virtual environments are useful for isolating dependencies for different projects, not just isolating the interpreter.
(I mean, except on Windows, your venvs default to symlinking the interpreter and other shared bits, so you aren't really isolating the interpreter at all, just the dependencies.)
AFAIK, it's the same even on POSIX, when you create a venv, "./bin/python" are symlinks and the pyvenv.cfg has a hardcoded absolute path of the current python interpreter the venv module was using at the time of creation. It really doesn't isolate the interpreter.
(also one of the reasons why, if you're invoking venv manually, you absolutely need to invoke it from the correct python as a module (`python3.13 -m venv`) to make sure you're actually picking the "correct python" for the venv)
I meant that the symlink behavior is default except on Windows (though it may just be on POSIX platforms and I think of it as “except Windows” because I don't personally encounter Python on non-Windows non-POSIX platforms.)
> making it harder for yourself
Looking at just the first link, looks way more complicated than venv. And I'm a C++ developer, imagine someone who less experienced, or even who just isn't familiar with C toolchains.
It’s really not hard; the person you’re replying to put a lot of details in. I’ve lost track of how many times I’ve built a Python interpreter. Virtual envs used to work badly with some tools and dependencies, so I had to do it a lot. It’s gotten better and now I only compile Python to get a version that’s not provided by my Linux distribution.
The first link is a sudo apt install command that you copy paste into terminal. in what world is that more complicated than venv?
Here it is for clarity
sudo apt-get install build-essential gdb lcov pkg-config \ libbz2-dev libffi-dev libgdbm-dev libgdbm-compat-dev liblzma-dev \ libncurses5-dev libreadline6-dev libsqlite3-dev libssl-dev \ lzma lzma-dev tk-dev uuid-dev zlib1g-dev libmpdec-dev libzstd-dev
Even what you just pasted is wrong. And it would give a confusing error if you used it. Try "sudo apt install vim \ make" to see.
It's the kinda thing an experienced engineer wouldn't have that much trouble with, but you should be able to recognize how much experiential knowledge is required to compile a complex C code base and what kinda dumb stuff can go wrong.
You probably don't need to do much of the stuff on that page to build, but "What is dnf?", "Is the pre-commit hook important?", "Do I need cpython?", "What's an ABI dump?" are questions many people will the wrestling with while reading.
> The first link is a sudo apt install command that you copy paste into terminal. in what world is that more complicated than venv?
venvs also aren't complicated.
I have built Python from source before, many times. I do it to test Python version compatibility for my code, investigate performance characteristics etc.
Re-building the same version of Python, simply in order to support a separate project using the same version with different dependencies, is a huge waste of time and disk space (hundreds of MB per installation, plus the mostly-shared dev dependencies). Just make the virtual environment. They are not hard to understand. People who want a tool to do that understanding for them are welcome to waste a smaller amount of disk space (~35MB) for uv. A full installation of pip weighs 10-15MB and may take a few seconds; you normally only need one copy of it, but it does take some work to avoid making extra copies.
Personal preference, but I prefer to use 'mise' instead of 'asdf' these days: https://mise.jdx.dev/
Maybe I'm the only one that finds Python simultaneously verbose and lacking? Either you need 500 dependencies to do something in a simple way, or you need dozens (if not hundreds) of lines to do trivial things. I avoid writing Python because there's so much bullshit to add. Much prefer Perl, I can actually get things done quickly. Python feels like programming for the sake of programming.
I have a bunch of zero dependency projects. You can get a lot done with the standard library and single file projects which you can run on most systems without needing to do anything except curl it down and run it since Python is available.
For example here's a ~2k line Python project which is a command line finance income and expense tracker: https://github.com/nickjj/plutus/blob/main/src/plutus
It uses about a dozen stdlib modules.
About 25% of the code is using argparse to parse commands and flags. With that said, I tend to prefer code that yields more lines for the sake of clarity. For example this could technically be 1 or 2 lines but I like each parameter being on its own line.
Argparse is part of my problem with Python! I can never remember the weird ways it wants to collect all these attributes and features, so I'm always leafing through manual pages or googling for examples. Same for if I'm modifying existing Argparse code, I have no idea how to make it do what I want. It can do a lot for you, but it's a pain in the ass to use.
It's so much easier to write a simple usage() function that has all help text in one big block, and Getopt with a case block. Anyone can just look at it and edit it without ever looking up documentation.
Arguments can get pretty complicated.
One nice thing about argparse is it comes with a helper function for mutually exclusive groups that can be optional or required. For example you can have --hello OR --world but not both together.
> Much prefer Perl, I can actually get things done quickly
Python lets you just nest data structures without having to twist your brain. You want a tuple in a list in a dictionary value: you just write it down and can access it with a unified notation. Bam. No reference madness and thinking about contexts. It's a big part of what I typically need to get things done quickly and understand how I did it 5 years later. That has to count for something. Python is boring in that sense, Perl is fun. But that's exactly my problem with it. It's too clever for it's own good and writing it does things to your brain (well at least mine).
That's what got me off Perl. I loved it and was skeptical about Python, but one time after trying it for a week or so, I wondered what it would look like to pass a dict of lists of dicts into a function, then reference items inside it. My first try worked: I made a dict, with a list inside it, and a dict inside that list, and passed it as an argument without any sigils or reference decorators or anything.
That was about the time I stopped using Perl for any new projects. I never wanted to go back.
That is a surprisingly succinct take. That may actually have affected the demise of perl, apart from perl's breaking changes.
Powershell is also horribly offended by trying to nest arrays. As is bash.
Perl also lets you nest data structures, no brain twisting involved. Perl doesn't have Tuples natively but you can add them with a module. You can also nest objects in complex data structures.
Perl being Perl, TMTOWTDI, but you can enforce your own style using linters and formatters.Major downside to dictionaries being so useful in python... Everyone uses dictionaries everywhere. I'm so happy for dataclasses and attr, because having to check that str keys are right without ugly ass constants is miserable.
dang the backend is leaking old comments from 2007
YMMV, but in my experience, Python's top use cases tend to require fewer dependencies, not more. Many trivial tasks are already built into the language. Not saying Python is perfect, but it's definitely well-oriented in that regard. I'm curious in what kinds of use cases have you found the opposite to be true?
I think the classic Python vs. Perl question ultimately comes down to using what you feel most comfortable with. People are different, and that's totally fine.
Could you share an example, where Perl is quicker to use and more powerful?
Using it as a replacement for Bash. No environment required, Perl is already installed on every computer on earth. Perl is generally quicker to write for string-oriented workflows - which is what Bash is. Regex is quick and easy.
Perl falls apart when you need structure and even small projects. For short scripts meant to do what you would use Bash for, it's a godsend. Safer, more portable, less friction.
Perl does not fall apart when you need structure, it's OO. In fact it's better structured than Python (hello, PyPI? It's CPAN calling, we wanted to let you know hierarchical packages are a thing). If you mean "forcing people to program in an opinionated manner", it doesn't do that by default, but that's what linters and formatters are for (like the ones Python programmers use)
Perl has OO but I don't like it much.
And not write-only (https://handwiki.org/wiki/Write-only_language)
Python is a language that the standard lib is big enough that you don't need dependencies
Technically, C or assembly also fulfill this "requirement".
There seems to very few smart people who agree with this in practice, however.
Am I missing an obvious joke? C and assembly have no batteries included.
"Here's a chunk of lithium, build your own battery"
I need JSON with comments and trailing commas
When you say "I actually can get things done quickly", does that include reading that same source a year from now?
The best part about python is that there's a library for everything.
Have you met Java?
"Only in my web browser."
Those were the days…
I'm glad someone else discovered they can like python.
I got forced to learn it for a project where I was proposing Ruby and the customer insisted on Python. This was years ago when Ruby was much slower. I was annoyed but I got used to it and here I am enjoying it many years later.
I take issue with the description and use of make though! :-D What is the point of it if you're not going to use dependencies? One might as well just write a script with a case statement..... I'm adding smileys because I don't mean to be all that serious but I really do think that the failure of the youth of today to get to grips with Make is a sad reflection on our culture....... and get off my lawn, ok? :-)
Ruby's syntax is much nicer. I just can't stand the idea of using whitespace for scope delimiting.
I thought that too....but I did get over it.
I ended up feeling that it removes a lot of (internal) debate about what's the best style for braces that you get in C-like languages.
It's just weird. May be if you are used to it, it doesn't bother you, but I find it too easy to do something wrong with scope.
I wish python had support for optional braces. Maybe time I copy some snippet or integrate some other project into mine and have to worry if someone using 2 spaces, 4 spaces or tabs.
If you are worrying about that you should try to Ruff or an equivalent.
Yeah, but they seem to think it's a feature, not a hindrance. I don't really understand the idea.
Ron Garrett noticed that emacs can use "pass" like a closing brace for auto-indenting.
Genius.
I love it, so much more concise and clean without the brace noise.
> One might as well just write a script with a case statement.....
I started with this and evolved into simple flat Makefiles, because they're basically the same but Make feels more standard (there's a Makefile here vs. there's a random bash script here).
For me, python is the closest thing to writing pseudocode that functions. Every time I have the instinct to gloss over a thing when writing it down (because it feels obvious in my head), it turns out that python has an intuitive abstraction for it.
Coming from a mathy background I found it incredibly satisfying, although I’ve come around to other languages since.
As a mathy person myself, I find the OOP leaning difficult to think about. Better equational reasoning, better lambdas, fewer side effects to worry about, avoiding mutation so that I can define something and still know what it is later during runtime, those are what help me think clearly. To me OOP is about the furthest paradigm from math that I’ve used.
OOP, especially the modern take, feels pretty far from just about anything.
Inheritence should have stayed esoteric. Composition is closer to reality.
Relations and functions and values are way closer to applications than OOP seems to be.
Well it’s not about math. Stick to lines of code, functions and modules for that. Use it for abstract data types, frameworks and modeling.
You’re right, it’s not good for math, but unfortunately we have the legacy of AI / ML being entrenched into that ecosystem, with no clear leaders as alternatives.
Its quite easy to avoid OOP though.
The only reason I would use Python is for its specialized AI/ML frameworks, and unfortunately those force you to go with OOP.
From what I was told, Python was originally seen as a Swiss Army knife for sysadmins. It started gaining more traction when Canonical adopted it as the main language for Ubuntu 4.10 in 2004.
Then, in 2005, Guido van Rossum was hired by Google to work on Google Cloud. That opened the door for wider adoption in academia, since Python had strong math libraries and integrated well with tools researchers were already using, like Hadoop, right around the time big data and ML were starting to take off.
Also, between 2005 and 2006, two important things happened: Ruby on Rails came out and inspired Django, which was starting to gain popularity, and web developers were getting tired of Perl's messy syntax. That's how Python quickly became a solid choice not just for server-side scripts, but for building proper web apps. In the meantime, another language that could be embedded directly into HTML was storming the web: PHP. Its syntax was similar to JavaScript, it was easy to pick up, lowered the barrier to entry for software development, worked straight out of the box, and didn't require thousands of print statements to get things done.
The 3 Ps made history. According to programmers from 20 years ago, they were like religions. Each had its own philosophy and a loyal group of followers crusading online, getting into heated debates, all trying to win over more adopters. The new generation of devs is more pragmatic. These days it's less about language wars and more about picking the right tool for the job.
> According to programmers from 20 years ago, they were like religions. Each had its own philosophy and a loyal group of followers crusading online, getting into heated debates, all trying to win over more adopters.
It's very weird reading something you lived through described in these terms, as though it were being described by an anthropologist.
Can't help but wonder what the future will have to say about today.
"In 2025, programmers used 'frameworks' to hang 'apps' from some kind of 'web'. They believed these frameworks gave them magic powers. Many even fought in 'flame wars' on this topic, which experts believe involved the use of fire to destroy webs woven by competing programmers."
The key factor imo was Travis Oliphant merging the competing numeric and numarray libraries into numpy in 2005. That quickly became the foundation of Python as the key environment for open source numeric processing.
It brought across a ton of users from R and Matlab.
Pandas, Matplotlib and ScikitLearn then consolidated Python's place as the platform of choice for both academic and commercial ML.
Python's success is entirely due to entry-level programming courses. They all switched to Python, because you have to explain less. I don't think I heard about web servers in Python before 2012. I suppose a 2005 computer wouldn't be able to serve a Python backend smoothly.
PHP's popularity isn't really from 2005-2006. It was popular at the end of the 90s, and it looks like JS as much as it looks like a potato.
This is entirely backward. They all started switching when python was already immensely popular after the popularity boosts given by sysadmin, django and numerical/datascience users.
This is what I have seen as well. When Python became ubiquitous in the industry, they started teaching it in the colleges.
I come from a core science background. I studied Physics. And it was like this in my institute: FORTRAN -> A lot of C, small amount of FORTRAN -> C -> Python. I was taught C, but from the exact next year, it was switched to the Python ecosystem.
It was done much later when Python became the standard in research universities, the language of recent research papers, etc.
A generation of Scienctists learned C/FORTRAN/MATLAB in college/grad school as it was taught, but they switched to Python early/mid career. Teaching Python in undergrad followed.
I also taught a mid-career Economics professor Python. She used SPSS before for her papers. Humanities professors doing data crunching are now switching to Python, too. There is a clear trend.
With python you can just go: 'print "hello, world!"' and that's a full-ass program
You run it by saying `python hello.py`.
Compare that to the amount of crap you need(ed) with 2005 Java just to have something running.
The shittiness of ActivePython and generally getting python to run on Windows were a bit of a hurdle, but still it was easier than the competition
You could, but then they deprecated the paranthesis less form in 3.x :)
Sure, but you could've done the same thing with perl in the 80's and 90's.
You could also accidentally summon an Ancient One while trying to write a regex :D
Perl is a fine language, but it's like using a wood chipper with no safeties. It takes extreme care and know-how to use it without splattering ASCII everywhere and making an unmaintainable write-only mess.
For every beautiful and maintainable perl program (mostly irssi scripts) I've seen 99 abominations that are quicker to rewrite completely than decode wtf they are doing.
True, true. I remember writing full blown applications as perl CGI scripts, back in the day. I should see if I can dig up some of that stuff.
I mean, it isn't like you couldn't get similarly terse scripts in other languages. Common LISP, as an easy example people love to beat up on.
> Python's success is entirely due to entry-level programming courses.
No, entry-level courses were in a mix of Scheme, C, and other languages until Java’s industrial ubiquity ended up in it becoming a popular choice, but not completely displacing the others. Then as Python (along with the broader class of dynamic OO scripting languages) became quite popular, Python got added to the mix, but, unlike Java, it fairly quickly displaced not only Java but a lot of what had been around longer in introductory courses.
Python’s industrial success drove its initial use in introductory courses, but doesn't fully explain it, as it doing what Java never did indicates.
I think the people teaching introductory courses find it less of a compromise to industrial popularity than they did with Java.
Python’s success is multifaceted, some of it is right-place right-time, a lot of it is the ecosystem it built because of that, but a lot of it is, I think, that it turns out to be a language that has been designed (both initially and in how it is been managed over time) to be a very good language for real people solving real problems, despite not adhering to any of what various typing and performance and paradigm purists like to posit as the essential features of a good language.
1. Python was pretty popular well before entry-level programming courses adopted it. I think they adopted Python because it was a good general purpose scripting language, multiplatform, easy to install and get going, that taught programming concepts in a more approachable way for beginners.
2. Python on web servers was a thing long before 2012. You had Zope in 1998 or so, and it was pretty popular for a while, and hugely influential to subsequent web frameworks. Django came out in about 2005. TurboGears, Pylons in about 2005 or so. Flask in 2010... and these are just the more popular frameworks.
3. I think the author meant that PHP was also vaguely C-like in syntax, like JS. Keyword: vaguely. You had a common way of declaring stuff with curly braces and semi-colons.
I always loved the huge standard library that PHP offered, that and the comment section per function on the php docs pages
> I suppose a 2005 computer wouldn't be able to serve a Python backend smoothly.
Python had web servers from 2000, including Jim Fulton's Zope (really a full framework for a content management system) and in 2002 Remi Delon's CherryPy.
Both were useful for their day, well supported by web hosting companies, and certainly very lightweight compared to commercial Java systems that typically needed beefy Sun Solaris servers.
I'd forgotten about CherryPy until Turbogears was mentioned the other day in the Django birthday thread.
But yeah Python was on an upswing for webdev and sysadmin (early DevOps?) tooling, but took quite a hit with Ruby eg Rails, Puppet, Vagrant and Chef etc.
But Python hung on and had a comeback due to data science tooling, and Ruby losing it's hype to node for webdev and golang for the devops stuff.
> They all switched to Python, because you have to explain less
I think this could be generalized to ergonomics. Java 1.6 is an absolute nightmare for a newb compared to Python. No top-level statements, explicit typing, boxing, verbose declaration syntax, long import statements, curly braces everywhere... and, most importantly, no out-of-the-box REPL. Java has since made strides and borrowed from Kotlin and Lombok but my understanding is that it was too little too late.
Depending upon preference and application one might consider ~half of these things anti-features at the expense of stability but it speaks volumes to why people join and stay in a software ecosystem. If I had to work on a Python project again I'd use mise with uv and viciously lint the codebase down to a reasonable fully-typed subset of the language devoid of custom decorator magic and dunder abuse.
> but my understanding is that it was too little too late.
Too little too late to be the #1 language of choice for serious server-side software that it is today?
The weird thing about Java is that people naturally compare its popularity today to its dominance in the early '00s, which was an aberration. The ecosystem has long since returned to its fragmented self, and while Java is not nearly as dominant as it was during that very short period, no other language is, either.
> Too little too late to be the #1 language of choice for serious server-side software that it is today?
I was replying to
> Python's success is entirely due to entry-level programming courses. They all switched to Python,
not to mention that there are an awful lot of qualifiers in your statement. There are certainly plenty of Java jobs to be had but all the usual suspects like PYPL, TIOBE, SO (disregarding the old adage about damn lies and statistics) put Python squarely above Java in terms of popularity.
This is all to say that if I got conked on the head and lost all of my programming knowledge and had to make a living I'd restart with Python. This isn't a value judgment - the JVM and Python ecosystem are on roughly equal footing in my mind. It's just how things are.
Python may be somewhat more popular [1] in terms of number of people using it (partly because a lot of Python programming isn't about shipping software and many Python users aren't professional programmers), but JS, Python, and Java, the three most popular programming languages today, are each dominant in specific domains, and none of them has a huge market share in the areas where the others dominate. There isn't a lot (relatively) of client GUI software written in Python or Java, there isn't a lot (relatively) of data analysis in Java or JS, and there isn't a lot (relatively) of serious servers in Python or JS.
[1]: https://www.devjobsscanner.com/blog/top-8-most-demanded-prog...
> Python's success is entirely due to entry-level programming courses.
Yeah, after 2008. And by 2014, it had overtaken Java in many CS programs. But I was referring to the events that led to that.
I argue that Python's success was almost entirely due to it not adopting any best practices on dependency management and being installed by default on systems.
Being default installed was almost certainly the larger factor. As evidenced by how much pain it caused people when they started using dependencies that were not stable. They had to add specific support for it to not let you pip install things to the system install.
> I don't think I heard about web servers in Python before 2012
Excuse me but... what ? Django is 20 years old.
> Python's success is entirely due to entry-level programming courses. They all switched to Python, because you have to explain less.
Nonsense.
> I don't think I heard about web servers in Python before 2012.
More nonsense.
> I suppose a 2005 computer wouldn't be able to serve a Python backend smoothly.
Extreme nonsense.
https://medium.com/signal-v-noise/ruby-has-been-fast-enough-...
And this is when Python was edging Ruby performance wise.
In my experience AWS also contributes a lot to this. Boto3 is a great library for interacting with AWS services
Peak PHP was Facebook around 2008!
> The 3 Ps made history. According to programmers from 20 years ago, they were like religions. Each had its own philosophy and a loyal group of followers crusading online, getting into heated debates, all trying to win over more adopters. The new generation of devs is more pragmatic. These days it's less about language wars and more about picking the right tool for the job.
I feel like the religious wars aspects of this is completely overblown. The conversations around languages really hasn't changed all that much in the last 20 years. Look at the way you see things happening in HN, heck even in the comment thread right here. It's the exact same kinds of discussions as happened back 20 years ago, with exactly the same kind of pragmatism.
I think the gulf comes about from the fact that the primary sources are conversations occurring between people that are largely anonymous behind an online handle, interacting with people that aren't face to face. There's always been an element of exaggeration in the interactions. What might be a "Umm, no" with a maybe a shake of the head, becomes "Your mother was a hamster and your father smells of elderberries". Almost every party involved comes to recognise the particular cultural quirks (which varied from forum to forum) and how to translate what was said, to what was actually meant.
>From what I was told, Python was originally seen as a Swiss Army knife for sysadmins.
Yea, I was a sysadmin around 2000 (before that too) and I knew it as such.
>between 2005 and 2006, two important things happened:
Somewhat - I used it in 2001 for Plone which is based on Zope, which was somewhat popular around that time. Writing all the web stuff with Python made sense, since Plone provided a CMS and could include a wiki. Adding on some sql calls to it in python just made sense. The competition was between PHP and Python, though there were some other less popular choices. Ruby on Rails definitely was getting a lot more popular around those times. PHP didn't start getting popular around 2005, if anything, people started using Python more, and started criticizing the crappy code that was circulating in the PHP community.
In any case, it was a fun time, but what's the point of looking back like that?
This aligns with what I remember and Guido going to Google explains things. Mid 2000's was all about PHP (Cake/Symphony/CodeIgniter/etc) vs Ruby (Rails) in the open source world. I remember hearing about Python only in sysadmin work and Plone. Then, suddenly were suddenly a lot of articles from Google about just about everything, all in Python.
I remember saying to a coworker, "Google is single-handedly keeping Python alive."
Then bam. It was everywhere. Mid 2010's, I took a cybersec job and they told me to learn Python in the two weeks between accepting and starting. "Python is all over cybersec," I was told. It was then I realized Python took over academia, which positioned it perfectly for ML. It's features made it easy to start, but it also benefited from right place, right time.
Guido's passion and leadership were the driving force behind Python. A lot of smart people aligned with his vision.
> These days it's less about language wars and more about picking the right tool for the job
You should talk to the Java advocates in my company :) The language wars are still around, it's just Java vs the rest now.
One of my colleagues, a retired programmer who spent 20 years at Microsoft, told my students that it used to be Java vs Visual Basic, until Microsoft brought in the big guns, like Anders Hejlsberg. That's when it turned into Java vs .NET, and things really started to heat up :)
Java unequivocally won the war, though.
C# is the Java of Lua (thanks to Unity) which will never not be weird.
Having used both, I still can't understand why.
M$ was too late to open source it and too late to embrace Linux. On desktop side .NET was only for windows back then and xamarin immature. On server side in java land you had framework war so you had many choices and libraries to choose from - companies didn't want to fully embrace M$ ecosystem (windows sql server, windows deployment etc). Then Android came and adopted Java and M$ lost mobile ecosystem war.
I can see similar way why Swift eventually can loose market share - too late to open source and too late to embrace cross-platform.
I'd say the switch was learning that Google Search was written in Python (remember when error pages URL were in .py ?), and Youtube rewritten in Python. After that, reddit and dropbox followed (and later Instagram).
Apple also added Python in, I think, Mac OS X Jaguar, so it blipped on the radar.
When you have big players using it, the community automatically grows.
Django was the continuity of that and definitely contributed to the growing popularity of Python, but I think the hype started way before (and in fact we had to suffer Zope/Plone/Twisted for it :)).
Another decisive date was circa 2010 when the Belgian book "learn programming with python" came out. Granted, Django was already 5 years old at the time, but it brought many beginners who knew nothing about programming at the time.
Python was everywhere in science by earlier than that (Numeric and numarray, Numpy’s predecessors, are from the late 90s/early 2000s).
I make projects following almost identical patterns. It's a little uncanny. Maybe the people in the python developer ecosystem are converging on a pretty uniform way to do most things? I though some of my choices were maybe "my own", it seeing such consistency makes me question my own free will.
It's like when people pick a "unique" name for their baby along with almost everyone else. What you thought was a unique name is the #2 most popular name.
This sort of architecture has been in favor with python for at least 10 years or so, but I think you're right — the structure just makes sense, so many reasonable engineers converge on using it.
as though subsurface pilot waves in every spectrum hold human egos as constituent particles - becoming-being lol
It is the side of ice cream that makes all the difference for the improbability drive.
Python has done an impressive job over the years of making steady robust improvements. The typing and tooling has just gotten better and better. There are still plenty of problems though, imho async is still a much bigger pain than it should be (compared to other runtimes with a very nice experience like go or elixir, even dotnet has been less pain in my experience). Overall I like python, but it mainly boils down to the robust libraries for things I do (ML, Data munching/analysis)
I don't know what I am doing wrong but nothing written in Python has ever worked for me. I download the .py repo from from github or wherever and try to run it - errors. I try to install missing libraries pip this and that - errors. I battle fixing endless error with dependencies and when the .py finally runs - errors - wrong version of whatever library or wrong patch of this or that or the "production ready" .py does not work correctly on Windows or single file script uses excel library that has changed in incompatible ways 3 times in 2 years. I Download all the files from the flashy looking web site and follow all the instructions to the letter - errors. Python is anything but "robust". It is the most fragile environment in the universe, worse than c, c++ and javascript put togeter, at least this is my experience with it.
That’s wild. The last time I had that experience with Python must have been more than 10 years ago. I’ll admit a decade ago you definitely did need to know way to many details about way too many tools from virtualenvs to setuotools to wheels, etc. to get things working from somebody else’s project, but imho, poetry and uv have really changed all that.
This is exactly my experience too. I avoid python software like the plague, because even when it does work, I can not rely on it continuing to work when python gets updated on my system.
This is where uv comes in to save the day.
Ofcourse you can if you just learn to use it properly.
You cant really make a judgement on python from this, as your story just screams user error. I have never encountered anything of what you say, and I have been programming in Python from before I had any clue what I did.
can you give at least one example? You are probably just expecting too much. No manual checkout/install process is easy as installing an application.
What about Go?
"go install github.com/org/repo/x"
`pip install git+https://github.com/org/repo/x`
My own experience could hardly be more different. For that matter, I have never had to even look at a "flashy looking web site and follow all the instructions to the letter" in order to install something.
For example, to install yt-dlp, I followed these steps:
Actually, only the second one, because I already had pipx (https://pipx.pypa.io/ — a wrapper for pip that does basic virtual environment management) installed.Can you name some specific things in Python you have tried to use, and give more concrete descriptions of how you tried to set them up?
It's quite crazy, as Python first version was in 1991, 4 years before Java. It's a language with a ton of legacy baggage, the 2/3 transition, the BDLF being replaced by a completely different project leadership system, and it still keeps going strong.
This interview from Brett Cannon (old core dev who worked on packaging, imports, the vscode python extension...) is eye opening:
https://www.bitecode.dev/p/brett-cannon-on-python-humans-and
The guy cares SO MUCH and they have so many things not to break you can feel the sense of passion and the weight of responsibility in everything he says.
> Python has done a good job of hiding its legacy ugliness (such as __init__, __new__, and similar aberrations), swettening its syntax to accomodate developers with good taste.
What exactly is the problem with __init__ or __new__? @dataclass is very nice syntactic sugar, but are we arguing here that having access to initializer/allocator/constructor dunder methods is "legacy ugliness"? This is the core of pythonic built-in aware python. Bizarre.
Bizarre is that you don’t consider it ugly.
Kotlin: constructor is either part of class definition or keyword constructor.
Ruby: initialize
JS: constructor
Python: ______new______, _______init_______
Literally this meme: https://knowyourmeme.com/memes/three-headed-dragon
I like that magic method names generally all follow the same form so it's obvious that they are magic methods and not intended to be part of the public API. Whether that form uses double underscores or something else doesn't really matter to me as they are not being called directly.
Would you consider a constructor magic though?
"Magic methods" is just the name people use for these special methods (another is dunder methods). They're "magic" because you do not call them by name, rather they are invoked by the interpreter in specific situations.
Yes, it is magic the same way other magic methods are, that is, because it is syntactically special and callable (and primarily called by) a syntax other than <instance>.<method>(...) or <class>.<method>(instance, ...)
Specifically, in the case of constructors, via <class>(...).
> it's obvious that they are magic methods and not intended to be part of the public API
Is there an alternative API? No. This is public API regardless of anyone's intentions. Though "it's weird" is really not a very strong argument against it.
Public API refers to the user of the class, not the implementer. You never call __init__ directly. __new__ allows modifying object creation itself for the implementer, but the user of the class will never call it.
There are private and public methods. Private methods are only supposed to be called within other methods, as in privately. Public methods are the ones that are normally called through the code, repl, by the user or whatever. You are not supposed to write `myclass.__add__(x)` anywhere except where you define the class itself and its methods.
There's actually 4 kinds:
Internal is more convention as the language doesn't really do anything with it, but it does with munged, and magic methods are specifically for things implemented in the language.Internal and munged don't exactly map to private and protected, but are kinda similar ish.
Oh right. I am not really into python, had read some doc about naming conventions some poitns.
In any case I actually like how one can use underscores to point on how exposed some method is supposed to be. Makes it simpler to actually know what to skip and what not.
There are several alternative API's. @dataclass is one, Pydantic offers another, there's also attrs, plus less general things like namedtuple.
Admittedly it's obnoxious when you've got habits for one and you're on a team that uses another--totally flies in the face of the zen re: "there should be only one obvious way to do things".
...but that was always a rather ambitious goal anyway. I'm ok navigating the forest of alternative API's if it means not being locked into something that I can only change by choosing an entirely different language. I'm happy that it's very easy to tell when somebody is mucking about with python internals vs when they're mucking about with some library or other.
@dataclass does a very specific subset of what overriding dunder methods does. It’s not duplicating an abstraction it’s layering them
That's kind of the point. These methods are never meant to be called directly. They're used to desugar.
I think it's fairly short sighted to criticize these. FWIW, I also did that the first time I wrote Python. Other languages that do similar things provide a useful transparency.
I don’t think anyone is criticizing its utility, just that the syntax of 2-5(?) underscores in a row isn’t something that is DX friendly.
The ugliness is intentional. Nobody wants to name their cool new variable __hot__singleton__in__your__area__.
They are called dunder methods, meaning "double under". It's 2 underscores.
I've had snippets in my editor for approximately 15 years at this point so I don't have to manually type any of the dunder methods. Would recommend!
The syntax is exactly two, not 2-5.
I think the point was: who can know by looking at it.
Anyone using a monospacdd font, a font where underscores are separated, or a run-together proportional font that they are even a little bit familiar with, because of the relative width of underscores and other characters.
Which, between them, covers approximately everyone.
I mean, maybe you should use a font that doesn't make it hard to figure that out. Next you're gonna say we should ban 0s and Os, Is and ls, because your font doesn't allow you to figure out which one it is.
Some-other-language-user complaining about python ugliness:_____new_____ and _____init_____
Same-other-language-user:
((()))()()()({}{}{{(((())){}}}}{}{}{};;;;();)(;}}}
*not supposed to be correct syntax, it's just a joke
It's like the bright orange garments that hunters wear, the ugliness is sort of the point. It says "this is a different sort of thing."
> It's like the bright orange garments that hunters wear, the ugliness is sort of the point.
Ugliness is not the point of hi-vis vests lol, the point is to not get shot by other hunters.
...by being as shockingly contrasting with their environment as possible. Maybe your aesthetics differ, but I find it quite ugly. If it wasn't, it wouldn't grab my attention so well. Wearing something shockingly ugly is a great way to not be mistaken for a deer.
By contrast, I find a well camouflaged deer quite beautiful--once I notice it at all. The beauty comes from the way that it is so clearly a thing of its surroundings. Nothing at all like a bright orange hat.
> Wearing something shockingly ugly is a great way to not be mistaken for a deer.
Sure... yes the bright orange is ugly, but it's not the ugliness that prevents you from getting shot, it's the bright unnatural color. Other hunters aren't thinking "oh that's really ugly, it must not be something I can shoot" they're thinking "bright orange means person, I should not fire my rifle in that direction".
> If it wasn't, it wouldn't grab my attention so well.
Are you saying that if you thought the bright orange was pretty it wouldn't occur to you not to fire your gun in its direction?
If I thought that bright orange was a part of a healthy forest ecosystem, I would likely see beauty in it. And if my intent was to shoot denizens of such an ecosystem, then yeah bright orange would make a poor indicator for "don't shoot here". You'd be better off with something ugly, something which clearly doesn't belong.
Can you elaborate bit what is the problem here? Is it shape of the characters or something else?
Amazing. If you need to ask this question I am guessing you have spent too long with Python . No other language uses this underscore madness for special methods or otherwise. Because it’s just , I don’t know a more appropriate word for it , stupid.
What's actually amazing is how you're unable to answer a straight question.
If you call something "stupid" it doesn't really convey anything meaningful, especially not in the way you're using there, it comes across as "I don't actually have a reason I don't like it, I just don't".
Oh yes they do: https://www.php.net/manual/en/language.oop5.magic.php
The programming languages world is broad/varied enough that any statement like "no other language does this!" is almost certainly wrong (outside of esoteric languages, which python and php most certainly are not)
Lua, too:
Most C and C++ compilers use similarly named macros:
https://gcc.gnu.org/onlinedocs/cpp/Standard-Predefined-Macro...
I mean, Python is dynamic. Here, have this: https://gist.github.com/josiahcarlson/39ed816e80108093d585df...
Now you don't need to write your double-underscore methods ever again, if you don't want to.
You don't call __init__ directly so the ugliness is limited to the definition which seems fairly minimal to me. It seems to be a naming convention for methods that are not invoked by name anywhere and that's at least informative when you're reading the class definition.
IMO this is less horrendous than e.g. go's insistence that exported functions are indicated by a capital letter - that really affects code using the module not just the definition.
If you felt your argument about __new__ and __init__ was valid, you probably would have used the actual names and not hyperbolic exaggerations.
I have found to believe that ugly and beautiful are irrelevant. As long as it's doing the job and not cause major headaches. Life is too short to worry about syntax.
> Life is too short to worry about syntax.
Every programming language you use will annoy you in some way. You stop caring after a few of them.
Right? I'm kind of surprised that a few underscores invoke such strong emotions in people.
I had honestly not considered that the author might have been talking about the visual aesthetic of the dunder methods themselves. Honestly that has me a bit stumped; all I can say is that if you don't like underscores, don't pick python.
Fewer characters than “initialize” or “constructor”, clearly marked as being “not a normal method”. Python is better here.
I’m not sure why you’re comparing __new__ to constructors in other languages.
Ruby has the same thing but it’s called ‘new’.
Implementing the type of customization (idiomatically) that __new__ provides in Kotlin and JS isn’t any cleaner.
I would think the dumber thing is having to create an empty file called __init__.py for a package to register as a package. And relative imports being completely fubar.
> I would think the dumber thing is having to create an empty file called __init__.py for a package to register as a package.
This hasn't been true since Python 3.3. You no longer need a __init__.py for Python to recognize a module, but it can still be useful in many cases.
This is incorrect.
> The __init__.py files are required to make Python treat directories containing the file as packages (unless using a namespace package, a relatively advanced feature).
https://docs.python.org/3/tutorial/modules.html
I mean, it's never been a problem for me in my 26 years of Python, but if it is such a big deal for you, maybe you should consider this: https://gist.github.com/josiahcarlson/39ed816e80108093d585df...
I wrote it in about an hour and a half. It seems to work in Python 3.6 and 3.12. Now you never need to write another double-underscore magic method again.
The point is to keep them out of the object namespace, and works well for that.
As if other language don't have their warts.. Ruby and Javascript for example have hideous syntax, comparing this with some special naming is quite bizarre.
But who cares? It's syntax, it has its purpose.
> What exactly is the problem with __init__ or __new__? @dataclass is very nice syntactic sugar, but are we arguing here that having access to initializer/allocator/constructor dunder methods is "legacy ugliness"?
My read was that the "ugliness" was in the method naming, and specifically the double underscores, not the availability of the methods themselves.
Probably the system of giving special meaning to what are otherwise ordinary identifiers. __ isn't a reserved keyword in Python or anything. But there's a set of conventions you're supposed to follow when naming methods and attributes, and they're visibly artificial. It would be cleaner to make the features that are a special part of the language definition also a special part of the language syntax instead of running them in indistinguishably with other nonsimilar things.
In C++, if you want to define the binary + operator on a class, you give the class a method with the special name `operator+`. In Python, to do the same thing, you give the class a method with the pseudo-special name `__add__`. You don't think the Python way is worse?
> You don't think the Python way is worse?
Have you considered how much familiarity might shape your reaction to the two? Both are specific patterns with arbitrary restrictions which new learners have to internalize, and that’s a fairly advanced task most people won’t hit until they are familiar with the language syntax.
Here’s a counter argument: the Python methods are normal method declarations whereas C++ developers have to learn that “operator=“ means something different than what “operator =“ (or any other assignment statement) would mean, and that this is the only context where you can use those reserved symbols in method names.
To be clear, I don’t think either of these is a big deal - the concepts and usage are harder to learn than the declaration syntax – but I think it’s incredibly easy to conflate familiarity with ease of acquisition for things like this.
> Have you considered how much familiarity might shape your reaction to the two?
It doesn't.
> Both are specific patterns with arbitrary restrictions which new learners have to internalize, and that’s a fairly advanced task most people won’t hit until they are familiar with the language syntax.
No, the Python methods are just ordinary methods with valid names. What 'arbitrary restrictions' are you referring to?
The arbitrary restriction is that you have to just learn a specific pattern and follow it. I can’t have my Python class use “addition” if I hate underscores or my C++ code use “mathematical_division” if I’m horribly pedantic. In both cases, it’s just a particular design decision by the language developers I have to learn, and in both cases I won’t think much of it after the initial acclimation.
Python you have to learn something arbitrary for each dunder method you want to define/override. + —> __add__, * —-> __mul__, etc
C++ it’s just one pattern to learn, x -> operatorx
I'm not sure how "operator+" is supposed to be appreciably different from "__add__".
One is a sensible choice a human might pick. The other is not.
Both are doing their job just fine?
So who were the non-humans who picked the one you don’t like?
They are both sensible choices that a human did pick, that's how they got into their respective languages.
They have different locality of behavior, for one.
Funnily enough, just starting your class method/variable name with __ does do magic things as a pseudo-keyword. Specifically, it mangles the name for callers outside of that class -- `self.__foo` would need to be accessed as `obj._ClassName__foo`. It's python's approach to having private methods, while remaining somewhat ideologically opposed to them existing.
This doesn't apply to the dunder methods, though. They're magically exempt from this magical mangling, so you could call them directly if you wanted. ¯\_(ツ)_/¯
> You don't think the Python way is worse?
They seem about equivalent? I don't see any real reason to pick one or the other, beyond personal preferences.
What's the use case for __foo -> _ClassName__foo? I've used python a bunch but never this feature, so I'm curious.
It prevents accidentally overriding the member in subclasses which can prevent private implementation details of parent classes from being accidentally interfered with by child classes.
As composition over inheritance becomes more idiomatic, there are probably less places where this matters, but as long as inheritance is used at all it can be useful.
It makes it nearly impossible to reference that attribute by accident. Basically, if class Foo defines "self.__bar", the Python interpreter sees the leading "__" and mangles it. It's the lightweight equivalent of a private attribute in other languages, with the difference that Python will let you attempt to access it anyway. But if you do, and it makes your code utterly brittle and prone to explosion, no one will give you sympathy or help you fix it. They're much more likely to tell you not to do that.
It's as close as they'll get to private methods while still having the "we're all consenting adults" philosophy that doesn't prevent you doing something dumb. Or if you use this and it breaks, you can't say you weren't warned.
Not when this can be done: https://gist.github.com/josiahcarlson/39ed816e80108093d585df...
Take a deep breath.
What problem is that addressing? It doesn't solve the problem that `__add__` is a valid identifier -- so is `add` -- and it also doesn't solve the problem that there is no connection between the symbol `+` and the name `add`.
I've been doing Python professionally for a while (~10-ish years in backend web dev and file processing), and while I don't like how exactly the author described dunder methods as "legacy", it's still not a language facility I reach out to often because, well, it's sort of odd coming from a C# background. The author has blog posts about Java, which isn't that different from C#, so maybe that's the reason.
Maybe as I grow to think of the "big picture" architecture-wise with my code, I will start incorporating dunders, but until then...
Out of curiosity, why would I use Dataclass vs a Pydantic Basemodel. If we did not have a PyDantic dependency I could imagine wanting to use Dataclass. But if I have it, why not use everywhere?
To help answer your question, here's a detailed comparison from the attrs project:
https://www.attrs.org/en/stable/why.html
There's some obvious potential for bias there, but I thought most of the arguments were well reasoned.
Edit: found another that I thought was helpful: https://threeofwands.com/why-i-use-attrs-instead-of-pydantic...
Unnecessary baggage if you don't need validation or serialisation.
My rule of thumb is to use Pydantic if I need serialisation, otherwise I default to dataclasses.
This and not just for the performance reasons. A Pydantic model represents an I/O boundary of a program. This conveys a lot of information to a programmer reading the code. It would be quite misleading to find it's actually only passed around internally and never used for I/O. A bit like using an int type to store a Boolean value.
On the other hand if I see a dataclass I can tell what it's purpose is by whether it's frozen or not etc.
Always strive for self-documenting code.
Main reason is (used to be) performance. But since pydantic 2.0 it’s not an issue anymore I default to use pydantic for everything.
Performance hit from data validation at construction time. I like msgspec which is much leaner and faster.
Why would I use a Pydantic model when `TypeAdapter(MyDataclass)` exists?
One explicit difference is that dataclasses don't support validating nested objects.
I'd use it for passing flat structures as function args rather than a massive list of individual args.
My rule of the thumb is that Dataclasses are for compile-time type-checking, pydantic classes are for run time type checking.
The Chad answer is to use pydantic.dataclass
> the Python interpreter is natively integrated in all Unix distros
It's included in the default install of most desktop/server Linux distros (with plenty of exceptions), but I don't believe any of the BSDs ship it in their base system.
IIRC macOS used to have python 2 in its default install, but I vaguely recall that being deprecated and removed at some point. My only Mac is on the other side of the country at the moment, so I can't check myself.
Python 2 was removed in Monterey 12.3, which was incredibly stupid and disruptive as it caught everyone by surprise. We all knew Apple said they would remove it, but everyone was expecting them to be sensible and do it on a new major OS release, like they did with PHP, not mid-cycle.
https://developer.apple.com/documentation/macos-release-note...
I wonder if that kerfuffle is why they ended up not removing Ruby and Perl yet, despite the same promise. macOS’ Ruby is around 2.6. Perl I somehow doubt they’ll get to, as it’s such an important part of Unix admin I bet they themselves use it somewhere.
There is still a /usr/bin/python3 which is a shim. When you call it, if you don’t have the Xcode Developer Tools you’ll be asked to install them (it’s a non-scary GUI dialog which takes two clicks) and then you’re set. That is also a few versions behind the cutting edge, but it does get updated sometimes.
> [Software that was officially EOL and no longer supported by the developer as of January 1, 2020 — even though they went back on that with an emergency patch in April] was removed in [an OS released on October 25, 2021], which was incredibly stupid and disruptive as it caught everyone by surprise [despite the fact that the intent to EOL that software was declared many years ahead of time, and that sunset date already represented an extension of multiple years to the usual release schedule].
I will never understand this.
But then, I've been using Python 3 since 3.2 and my first reaction to that was a sigh of relief, and by the time I updated to 3.4 I was already wondering why everyone else was lagging behind on the switch.
> I will never understand this.
Perhaps because your interpretation of my comment is wrong.
> was removed in [an OS released on October 25, 2021]
No, no it was not! That would have been fine. Heck, it would even have been fine if they had removed it the year before. Two years. Three. I don’t care. The problem is that it was removed on a point release (not October) without warning, after setting the precedent of removing another language on a major release.
> But then, I've been using Python 3 since 3.2 and my first reaction to that was a sigh of relief
And I don’t even care about Python. But I still had to deal with the fallout from that from things I didn’t write.
> The problem is that it was removed on a point release (not October) without warning, after setting the precedent of removing another language on a major release.
My perspective is that the problem is that people were trying to use Python 2 after January 1, 2020. I left it behind years before that.
I think that by now we shouldn’t take a .0 major release as a sign of what is really going to be present in macOS and what’s not. The OS stabilizes around .4 or so, and in the meantime some deprecated things may get removed, too.
They also have a habit of sometimes add or change some APIs mid-cycle, so you may see requirements for the latest Xcode being a mid-cycle version of the OS. Or how was it with Mac App Store, that not only itself, but the relevant APIs for programs distributed in it appeared in 10.6.4?
/usr/bin/python3 is showing 3.9.6 on my Sequoia 15.5. That seems... ancient. I'm using 3.13 for all my work and personal projects.
I dont think python is pre-installed on macOS now. (uv has replaced it for me)
Edit: Unlike older versions of macOS that came with Python 2.7 pre-installed, newer macOS versions (Catalina and later) no longer include Python pre-installed.
Technically macOS doesn’t come with Python pre-installed, but it does provide you with a simple path to do it.
https://news.ycombinator.com/item?id=44580198
The removal was in Monterey. Catalina and its successor Big Sur very much still had it. Catalina was the one that removed 32-bit support.
I think they realised the security implications,
you could just take a random person macbook, open the terminal and launch python3 -m http.server 3000 --directory ~
then on the local network you could download all his files
If you have access to their MacBook and can open a terminal, even `rsync` (which comes preinstalled) would do the job.
It seems much more likely to me they were just tired of having to manage the languages (and being constantly criticised they were behind) and simply chose to remove them.
This doesn't make much sense. With physical access, I could easily just download python and do the same thing. Or anything else.
macOS dropped PHP recently too—doing a wonderful job of losing all that developer share that Apple was slowly building up.
Why? Installing python and php can be done in 2 seconds with brew, and you have control over what you install instead of using whatever is in the system, with deprecated versions, etc. It is actually much better now. System tool should be left to the system.
Virtually zero professional developers that I know use built in Python or PHP. Maybe it’s good enough for occasional scripting purposes, though.
I think it’s better for developers to not have conflicting system distributions.
Though for a while there having built in interpreters was great for kids and learners.
I much prefer installing it myself, with the required version for my project and at a known and common location.
That's fair enough. It may be less of an issue for experienced developers, but for those looking to learn the craft, it's one more barrier.
if you install commandline tools there's
/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/
Funnily enough, I made the opposite switch recently, and am also liking it.
My thoughts about python here: https://calvinlc.com/p/2025/06/10/thank-you-and-goodbye-pyth...
Next time I get into Python I’ll try uv, ruff, ty.
I switched from Python to JS for backend stuff a while back, thoroughly enjoying it. I agree that "Python installation and package management is broken," but the async stuff was the biggest improvement to productivity. Yes I know Python got asyncio, but there's a big difference between having one well-accepted way of doing things vs multiple competing, incompatible ways, where the good one has the least momentum.
The rest is small stuff that adds up like Py whitespace scoping, or Py imports somehow not taking relative paths, or JS object syntax is nicer: https://news.ycombinator.com/item?id=44544029
ty is still in alpha, I wouldn't recommend it yet. It's fast but has many false positives. In the words of charlie himself: https://bsky.app/profile/crmarsh.com/post/3lp2xikab222w
uv + ruff (for formatting AND linting) is a killer combo, though.
And the more you use uv, the most you discover incredible stuff you can do with it that kills so many python gotchas like using it in the shebang with inline deps or the wonders of "--with":
https://www.bitecode.dev/p/uv-tricks
I've avoided Python for a long time, but I'm getting roped in myself, mainly because certain tasks seem to require a lot less code than Java or Perl.
That said, call me old-fashioned, but I really take issue with "curl $URL | bash" as an installation method. If you're going to use an install script, inspect it first.
If your going to execute the code anyway, you either have to inspect everything or trust whoever is providing it. There is nothing special about bash that makes it more dangerous to execute than python.
My issue is with $URL potentially getting hijacked, or even something like the kerfluffle over the PuTTY SSH client not residing at putty.org.
do you also inspect binary installers?
I like Python for the language, and for a lot of jobs the threading model limitations do not matter. Its a great language to get stuff done. I find the package management story challenging but I will try uv next time!
I resisted pipx, poetry, anaconda and probably some other things, as I didn't see the point over venv+pip. But uv is just so damn good, there's no going back.
Feel like pipx walked so uv could run, it popularized the idea of abstracting the environment management away from the user.
pipx is still the best tool for standalone utility type of packages: "yt-dlp", "glances" (like htop), etc.
You can, however, use uv for this, e.g., `uvx yt-dlp` works like it would in pipx.
https://docs.astral.sh/uv/guides/tools/
pipdeptree is all you need for many things, but uv is such a joy to use it's hard to resist, sure.
If you haven't already you should check out uv: https://github.com/astral-sh/uv
It solves a lot of the package management headaches for me.
Good write up, and solid choices. As someone primarily working in python in the last few years, I have a very similar stack.
Two additional suggestions:
* mise to manage system dependencies, including uv version and python itself
* just instead of make; makefile syntax is just too annoying.
Mise actually has a command runner as well which I haven't tried yet, and might work better for running commands in the context of the current environment. It's pretty nice when your GitHub actions workflow is just:
* Install mise
* mise install everything else
Nice article, but this command doesn't activate the venv that's managed by uv.
The command actually creates a new venv in a dir called "activate". The correct way to activate the venv is like this:One of the key principles of uv is you don't activate the venv. You just run everything through uv every time, e.g `uv run pytest` or whatever.
If you use Make as a task runner like in the article, please make sure that you're also declaring your tasks as .PHONY targets: https://www.gnu.org/software/make/manual/html_node/Phony-Tar...
Otherwise, the existence of a file or folder with the same name as your task ("test", for example) will stop that task from being run, which might be very annoying if you're using the Makefile as part of a script or CI or something where you won't notice the "Nothing to be done for..." message.
Oh, is that the actual reason it’s for?
IIRC the info pages just say that it is for targets that lack a file. This is way easier to remember.
> So yeah, Python is powerful, and it couples very well with the now ubiquitous VSCode editor.
I always found vscode lacking for Python and C compared to pycharm and clion. The latter just work without fiddling with config files.
Funny enough I feel the other way about JetBrains IDEs. They * seem * super powerful, but there is always a lot of config that needs to go into them if I'm doing app, infra, and pipeline work. (Edit, not saying they aren't powerful, just more that as coming into them I'm never sure how to wield it the best out of the box)
In my experience (not saying this is universal), the folks that like JetBrains IDEs came from java/intellij backgrounds, where I hear it really shines.
This all might be a skill issue, as almost all my professional projects have been VSCode based, but since I've only worked at smaller places I definitely can't rule out this was because it was easier to set things up than to fight for Fin to get us all licences.
In your opinion, what makes PyCharm (or CLion if you want to add that in) 'just work'? Do you think it is because you've used it for so long and just know the ins-and-outs? Or is there something you see that they have and VSCode doesn't?
I've always been curious about this as someone who hasn't had a lot of professional exposure to the JetBrains world.
The things that just work that I look for are mouseover to get info on an identifier (local or external), right click to go to the definition, and running code in the debugger with a click. Maybe I just don’t know how to set that up in vscode; I’m a company of one and don’t have a coworker to ask how to do it. My Python work was just making requests, analyzing the data and outputting results to csv’s. This was replaced by Rust and then C. And the C is 60k loc as it does much more. In Clion when you open a cmake project it automatically understands your project structure from the cmake files and provides those need-to-have features I listed.
"Just work" is certainly subjective, but the introspection in all the JetBrains products are what make them ferocious. If one has never heard of all the 18 quadrillion lint tools before, you can immediately get value from opening a file in PyCharm and having it point out the ways it is going to fail on you, including one of my favorite tricks that no VSCode+lint horseshit has ever thought about trying:
PyCharm will spot that error immediately, no insaneo configuration requiredPyCharm Professional also gets into the SQL side of things:
instantly spotted, no configuration requiredI was going to be cute and use json.loads as an example, but it seems somewhere along the way they botched the fact that the first argument to json.loads is a fucking JSON string. But, it does allow showcasing that you can have PyCharm syntax check the "inner" language of any string literal you'd like via their language injection annotations:
Jet brains was so slow on my machines that I got to experience the speed of a physical teletype, decades after they went out of fashion.
From what? It would be useful to mention towards the front of the post so we know what is the context in which you approach python.
I've switched to python primarily (from perl) in early 2010s (I think my first "seriously" used version was 2.6. This is mostly for system management, monitoring, and data transformation /visualisation. Nothing fancy like AI back then in a work setting.
I found the biggest impact was not so much on writing code but on it remaining readable for a very long time, even if it was created hastily "just get this working now" style. Especially in a team.
Python is still one of my favourites and the first tool I reach if bash is not enough for what I'm trying to do.
I’m on board with most of this. The one suggestion I’d add is to replace “make” with “just”
I hear about it every now and again and I can't seem to grok the benefit. What's the killer feature over make?
First, I grok make. I'm saying this from a position of familiarity, not of ignorance and fear.
Make is great at compiling code in languages that don't have bespoke build systems. If you want to compile a bunch of C, awesome. For building a Rust or JavaScript project, no way. Those have better tooling of their own.
So for the last 15 years or so, I've used make as a task runner (like "make test" shelling out to "cargo test", or "make build" calling "cargo build", etc.). As a task runner... it kinda sucks. Of course it's perfectly capable of running anything a shell script can run, but it was designed for compiling large software projects and has a lot of implicit structure around doing that.
Just doesn't try to be a build system. It's optimized for running tasks. Here, that means it provides a whole lot of convenient functions for path manipulation and other common scripty things. It also adds dozens of quality-of-life features that devs might not even realize they wanted.
For example, consider this trivial justfile:
If you're in a directory with it and run `just --list`, it'll show you the list of targets in that file: That second recipe takes a required command line argument: You can do these things in make! I've seen it! Just doesn't add things that were impossible before. But I guarantee you it's a lot harder to implement them in make than it is in just, where they're happy native features.There are a zillion little niceties like this. Just doesn't try to do everything make does. It just concentrates on the smaller subset of things you'd put in .PHONY targets, and makes them really, really ergonomic to use.
You wouldn't use just to replace make in a large, complicated build. I would unhesitatingly recommend it for wrapping common targets in repos of newer languages, so that `just clean build test` does the same things whether you're in Python or TS or Rust or whatever, and you don't want to hack around all of make's quirks just to build a few simple entry points.
I don't really like Make as a build system, especially since today every language I've worked with has a much better one - cargo, uv, cmake, bazel, others.
My thing is that it's ubiquitous and if you stay simple, the syntax is perfectly readable; if you're doing anything complicated, I'll argue you don't want your logic in a Makefile anyway, you want it in a shell script or a Python script.
I get that if you want to do really complicated things, Just can be more ergonomic. But I haven't seen any real argument that it's more ergonomic, or understandable, or worth the learning curve, when your Makefiles are simple or have actual scripts doing the complicated stuff.
`just --list` is indeed missing (though I hear make is adding a --print-targets flag), but I usually need to run the same commands - make run, make test, make typecheck, make lint, make format. Note how none of these take an argument - that's also something I've never found myself needing.
That’s the thing: just is a lot simpler and cleaner for the simple things, too. Like not requiring .PHONY, for instance. And if you already have to install cargo/uv/cmake, add just to that list of dependencies.
The argument stuff is nice when you realize you’re no longer constrained by not having easy access to it. I used just to build my blog, and added a target to create a template entry whenever I updated to a new release of certain software. I could write “just newversion 1.2.3” to publish an announcement using the same mechanisms that did everything else. Without that feature, I could have scripted something up to do the same. With it, I didn’t have to. I wouldn’t have tried that with make.
If you're an expert in make and already have a Makefile, I would not recommend switching to just for that project.
The benefit of just is that it's designed to be a command runner, whereas make is designed to be a build tool. justfile syntax is much simpler and more ergonomic. It also has nice features: Private recipes, submodules, recipes that you can specify to run only in a particular OS (we use the same justfile for both Windows and Linux), writing your recipes in a language other than your shell language, and many many other niceties.
A new user can start doing "advanced" stuff in just in a couple of hours. They'll take a lot longer if trying to do it via make.
Just is fantastic for any project - also non-code ones!
It's funny, I'm the opposite: LLMs have made it easy to draft something in Python, then translate to a more appropriate language for the target problem-domain, for example Go.
Agree . We had training to learn how LLM applications can be developed and it was obviously in Python. Never liked Python but between dependencies issues (the demo project used uv but somehow the author forgot to declare some dependencies which she probably had installed globally) which cost hours of everyone’s time, and the persistent magic going on (pipe operators that do nothing similar to unix pipes, wtf!, puydantic shit just to declare a data type with metadata and lots of things like this) nearly everyone immediately switched to a typed language like Kotlin (they have a nice AI framework) as soon as we got the basics of how things work.
Is go the language that doesn't include a module to do mmap and doesn't include a module to place binary data into a struct?
If you want to read binary files, you can use C or python, but not go (unless you use segfault prone 3rd party libraries of very dubious quality, of course).
"And guess what's the de facto programming language for AI? Yep, that sneaky one."
Is this referring at all to to PyTorch. If not, any guesses what the author has in mind
"Not only because the syntax is more human-friendly, but also because the Python interpreter is natively integrated in all Unix distros."
Is this referring to GNU/Linux.
UNIX (UNIX-like) includes more than Linux; some UNIX distributions do not include Python in the base system
Where it is left as choice to the user whether to install it
I know this because I use such distributions and, unless some software needs it, I do not install Python
In such case, when I am done using that software I uninstall it^1
For example, he mentions retrieving YouTube channel metadata
I do not use Python for this; I use a 19-line shell script (ash not bash), its startup time is faster
Unlike Python, it is included in the base system of the UNIX distributions (both Linux and BSD) that I use
But if I need to test something using yt-dlp, then I might temporarily install Python
1. I compile Python from source and one annoying aspect of the project , in addition to the slow startup time, is their failure to include an uninstall target in their Makefile
"And guess what's the de facto programming language for AI? Yep, that sneaky one."
He's referring to Python in general
"Not only because the syntax is more human-friendly, but also because the Python interpreter is natively integrated in all Unix distros."
I think he means that its available or readily available in many major linux distributions like Ubuntu, Fedora, NixOS, etc. I don't think native is the right word.
I use bash too but Python is amazing. You're right, there are problems related to packaging and speed, but it is still very often the right tool for the job. It's powerful, easy to use, and open source.
"I use bash too but Python is amazing."
I do not use bash. I use ash. Bash is too slow for me, like Python.
>I do not use Python for this; I use a 19-line shell script (ash not bash), its startup time is faster
Ash doesn't do web requests unless you've implemented HTTP in ash. You're back to using 3rd party dependencies that aren't installed on all systems
For this script I have "implemented HTTP" using printf, an ash builtin.
The TCP networking is done with orginal netcat reading the HTTP from a pipe.
The TLS is handled by a TLS forward proxy listening on the loopback.
The orginal netcat and other TCP clients I use, like tcpclient from djb's ucspi or tcploop from haproxy, are not part of the NetBSD base system but are easily added when I compile the OS. For Linux I use a custom distribution I make myself, without LFS. Busybox has ash and nc together in the same binary.
These TCP client programs are stationary targets, they will work reliably year after year, and small enough that I can store and compile them quickly even on computers with modest resources. Python is constantly evolving, a moving target, and much larger.
I wrote a utility in C89 that "implements HTTP" called yy025. This is produced using GCC, specifically flex, which is intsalled on all systems I use. flex is part of the NetBSD toolchain. It's a requirement for compiling the OS. It's a requirement for building many GNU userland utilities. It's even a listed requirement when building the Linux kernel.
yy025 is what I normally use in shell scripts when I need to generate HTTP. It reads URLs on stdin and outputs customised HTTP to stdout. There is no "third party dependency" for HTTP. This is a "first party" program. I wrote it.
But this script to fetch YouTube metadata doesn't use yy025. It's just some printf statements.
In more recent times, NetBSD has added OpenBSD's version of nc to the base system.
One could thus argue that there is a TCP client "installed on every system".
For me, what is more important is an installed copy of GCC. I have yet to use a UNIX-like system that did not have the needed networking functions to create a basic TCP client.
What is the reason you prefer not to simply let python lie around? Security?
Storage space usually. Even when I have surplus space I still like to have minimal userlands with only the utilities I am actually using.
> I prefer to use a monorepo structure
Worked at a company where that approach lead to huge unwieldy structure that no one dared to touch to not break anything other teams are working on. The problem was not so much the repo, but dependencies structure (like single requirements.txt for the whole repo) and build scripts.
In theory it should've worked great -- you only need to update a dependency once and be sure all the code has the most recent security patches. In reality everyone was afraid to touch it, because it will break someone's else code. Monorepos work great only if you have serious NIH syndrome (Google).
I actually started appreciating damned micro-services there, as long as each service just reflects organization team structure.
https://en.wikipedia.org/wiki/Conway's_law
Sounds more like a monolith. You can do microservices with separate requirements in a monorepo. We have one setup at work using pantsbuild (eh the tool is a bit quirky) but it's much nicer than constantly spinning up new git repos and CI pipelines for small services/utilities
That said, I think service-per-team is a good pattern
When did Python go out of fashion? This is the second article I've seen talking about it as if it's some kind abomination.
I get that it's not the shiny new thing, but I don't understand people hating on it. Is this just junior devs who never learned it, or is there some new language out that I missed? (And please don't tell me Javascript....)
I have had a somewhat unearned distaste for Python for the last decade or so.
I mostly just don’t like some of the design decisions it made. I don’t like that lambdas can’t span multiple lines, I don’t like how slow loops are, I don’t like some functions seem to mutate lists and others don’t, and I am sure that there are other things I missed.
But it really comes down to the fact that my career hasn’t used it much. I occasionally have had to jump into it because of scripting stuff, and I even did teach it for a programming 101 course at a university, but I haven’t had a lot of exposure otherwise.
For scripting stuff for myself, I usually end up using Clojure with GraalVM (yes I know about babashka), or nowadays even just a static linked compiled language like Rust.
I don’t really understand why people think that compiled languages can’t be used for scripting (or at least task automation), honestly. Yes you have to add a compilation step. This involves maybe one extra step during development, but realistically not even that. With Rust I just do cargo run while developing, I don’t see how that’s harder than typing Python.
The most insane python feature is that for loops keep their intermediate variable.
for x in [1,2,3]: print(x)
x sticks around! So you'll always have these random variables floating around, and hope you don't use the wrong one.
And to add insult to insult to injury, if you're using mypy for type checking you'll get a nice error if you try to reuse x with a different type:
for x in ['a', 'b', 'c']: print(x) << Incompatible types in assignment (expression has type "str", variable has type "int") [assignment]
And the types I can never trust. I've used all the tooling and you still get type errors in runtime. It's also ridiculously slow.
I would also like optional chaining (e.g. foo?.bar?.baz) and a million other features that other high level programming languages have.
> The most insane python feature is that for loops keep their intermediate variable.
"Insane feature" is a generous way of describing this behavior. I would say it is just a stupid bug that has managed to persist. Probably it is impossible to fix now because of https://xkcd.com/1172/
How typecheckers and linters should deal with this is a tricky question. There is how the language ought to work, and then there is how the language actually does in fact work, and unfortunately they are not the same.
Comments like this basically should say "I want as much handholding as possible"
Lucky for you, LLMs are pretty good at that these days.
>And the types I can never trust. I've used all the tooling and you still get type errors in runtime. It's also ridiculously slow.
IDE integrated mypy checking does this in the background as you type. As for errors, it all has to do with how much typing you actually use. You can set the IDE to throw warning based around any types or lack of type annotation.
Again, handholding.
why would you want a lambda to span multiple lines? How would that be any better than a function?
Because sometimes I want to have logic that spans multiple lines and I don't want to assign it a name. An easy example might be something with a `map` or a filter. For example, in JavaScript
Obviously I know I could name this function and feed it in, but for one-off logic like this I feel a lambda is descriptive enough and I like that it can be done in-place.You're free to disagree, but I think there's a reason that most languages do allow multi-line lambdas now, even Java.
> Obviously I know I could name this function and feed it in, but for one-off logic like this I feel a lambda is descriptive enough and I like that it can be done in-place.
FWIW, you'd also have the benefit of being able to unit test your logic.
I mean, maybe, that's why your lambdas shouldn't be too long.
I have done a lot of Haskell and F#, and I'm very familiar with the concept of "lifting", and yeah being able to individually test the components is nice, but even within Haskell it's not too uncommon to use a lambda if the logic doesn't really need to be reused or is only a couple lines.
If you have a huge function, or you think there's any chance of the logic being reused, of course don't use a lambda, use a named function. I'm just saying that sometimes stuff that has 2-4 lines is still not worthy of having a name.
Off topic: you didn’t use y.
You are right! Obviously this is just an ad hoc thing I wrote to show a point but I shouldn't be using superfluous variables.
Honestly, I don't really see the appeal of unnamed functions in general. I so rarely use lambdas that I wouldn't really miss them if they were gone. Just occasionally as a sort key, or in a comprehension.
I have seen people do this in JavaScript quite often, but I always assumed there was some kind of underlying performance benefit that I didn't know about.
As I think about it I guess it makes sense if you're passing a function to a function and you just want it to be concise. I could imagine using something like that off the top of my head, but then pulling it apart and giving it a name the moment I had to troubleshoot it. Which is how I currently use nested comprehensions, just blurt them out in the moment but refactor at the first sign of trouble.
I think maybe I just have trouble seeing some of the braces and stuff, and it's easier for me to reason about if it's named. I guess that's why we have 32 flavors.
Thanks for answering me honestly I really do appreciate it, even if my tone came off as dismissive. Sometimes I don't realize how I sound until after I read it back.
Obviously it's totally fine to have a difference of opinion for something like this.
> I have seen people do this in JavaScript quite often, but I always assumed there was some kind of underlying performance benefit that I didn't know about.
I don't think so, at least I haven't heard of it if there is.
I tend to have a rule of thumb of "if it's more than 6-7 lines, give it a name". That's not a strict rule, but it's something I try and force myself to do.
Like in Python, most lambdas can be done in one line, but that also kind of gets into a separate bit of gross logic, because you might try and cram as much into an expression as possible.
Like, in my example, it could be written like this:
But now I have one giant-ass expression because I put it all into one line. Now where previously I had two extra names for the variables, I have the ad-hoc logic shoved in there because I wanted to squeeze it into a lambda.And I think that's where the reasoning behind only allowing one line lambdas came from. I believe I read a thread a long time ago where GVR didn't even want to include lambdas at all, if I have time I might look for it and edit in a link.
At it's core, I think it's fair to say Python is about forcing the user into formatting their code in a readable way. It's gotten away from it over the years for practicality reasons, and to increase adoption by people who disagree on which ways are more readable.
Sometimes I wish they would take nested comprehensions away from me, I am too lazy to avoid them in the heat of the moment, and I get a thrill out of making it work, even though I know they're disgusting.
Naming things is one of the hard problems of computer science. It's nice not to be forced into naming something.
If your style is doing a lot of functional programming, multi-line lambda is a very natural thing to do. Other times you want to use a variable or several variables without actually passing it around as an argument. It makes sense especially if you are already used to it in Java/C++/JavaScript/Go.
Is it "better" than a named function? No, of course, they work mostly the same. But we are not talking about better or not. We are talking about syntax just for the sake for syntax, because some people prefer to write code in a way you don't necessarily care about.
This makes a lot of sense, I got to a similar conclusion in my other reply.
I always thought the appeal of functional programming was more about test-ability and concurrency, it never occurred to me that people actually preferred the syntax.
Can't speak for anyone else, obviously, but part of the reason that I got into functional programming is specifically because I found the syntax very expressive. I felt like I was able to directly express my intent instead of describing a sequence of steps and hope that my intent comes to fruition.
Different strokes and whatnot, not everyone likes functional programming and of course there are valid enough criticisms against it, but I've always appreciated how terse and simple it feels compared to imperative stuff.
Even with regards to testability, if your function is pure and not mucking with IO or something, then even using a multi line lambda shouldn't affect that much. You would test function calling it.
Keep in mind, Haskell doesn't really have "loops" like you'd get in Python; in Python you might not necessarily need the lambda because you might do your one-off logic inside a for loop or something. In Haskell you have map and filter and reduce and recursion, that's basically it.
> Haskell doesn't really have "loops"
weeell, you can still do stuff like this in Haskell:
it's just in many cases the functional machinery is so accessible that you don't need to reach for for-loops.Apparently the author used to be a Java person. I have this feeling - or prejudice - that people still in the Java World are a bit out of tune with the tech industry, working on huge legacy projects on big retail or US banking. These people tend to be conservative and resistant to change, so maybe that's where these types of articles are coming from.
Java shops are certainly where I've witnessed the most disdain for Python. IME the strongest feelings tend to come from people who didn't actually have any significant experience with Python, and perhaps don't even have much practical experience with any language that isn't Java. So they tended to consider it to be objectively inferior purely because it's interpreted and dynamically typed.
At a previous job I did manage to put a chip in that when I demonstrated replacing one of our Java services with a Python implementation. It was a fraction of the code, and achieved much better latency and throughput. Obviously not every Python program is going to do that. But my point isn't that Python is better, it's that these kinds of things are never so cut-and-dried. Many non-trivial Python programs are just thin shells around a large core of very well-optimized and battle-tested C/C++/Rust code. And Java, for its part, can also accumulate a lot of dynamic language-style performance losses to pointer chasing and run-time type lookups (and GC churn) if you're not careful about how you use generics. As always, the devil's in the details. It's also less able to pull the "actually I'm a C++" trick because using a compacting garbage collector makes it difficult to interop with native code without paying a hefty marshaling tax.
I have way more experience with Python than Java. In at least 4 companies (including Google that I wouldn't call a javashop) we used mainly Python.
Still I believe Java is a better application language. Python is a better scripting language (replacement for Bash). Small apps tend to be easier on Python, but large apps are way easier on Java, both for syntax (types) and ecosystem (libs).
Mostly agreed, though I would add that I'm generally happier with Python as a default for reasonably sized services that don't have a lot of (non-numpy-friendly) compute load and therefore don't have a pressing need for multithreading. Which is a lot of what happens now that we're all trapped in the cloud. Like you say, small apps tend to be easier in Python.
That is general take as well. A lot of small apps/simulators are in python. Ops scripts tend to be python. Java for the core/data. Refactoring/tooling is easier in Java when you are dealing with a 100k codebase imo. Typescript always.
Seen plenty of coding horrors in both ecosystems...
Algorithms are a lot easier to understand when they are written in Python. I'm actually right now documenting medium size Java codebase by writing pseudocode, which looks like Python. Just by doing that, I've already discovered multiple bugs, which I didn't catch by looking at the Java code.
That might be the big thing that I think gets glossed over in a lot of these discussions. I agree that I wouldn't want to maintain 100kloc of Python. But, I don't really view that as a realistic hypothetical for a business application. Idiomatic Python tends to require a fraction as much code as idiomatic Java to accomplish the same task. The only time it even gets close is when you have code written by people who go out of their way to make things look like old-school enterprisey Java. So it ends up accumulating a bunch of stuff like
when it could have just been: The worse case for Python is when you get people doing the oldschool Python thing of acting like dynamic and duck typing means it's OK to be a type anarchist. Scikit-learn's a good one to put on blast here, with the way that the type and structure of various functions' return values, or even the type and structure of data they can handle, can vary quite a bit depending on the function's arguments. And often in ways that are not clearly documented. Sometimes the rules even change without fanfare on minor version upgrades.The reason why large Python codebases are particularly scary isn't necessarily the size itself. It's that for a codebase to even get that large in the first place it's very likely to have been around so long that the probability of it having had at least one major contributor who likes to do cute tricks like this is close to 1. And I'd take overly verbose like the Java example above over that kind of thing any day.
Hey, you forgot useless comments for each method:
And use double underscores for private fields, because, you know, encapsulation.Seriously though, in Java you don't need get*() either. Typically you either:
1. use `record` types (since Java14, 2020), or
2. use Lombok that auto-generates them [1], or
3. use Kotlin `data class`, or
4. just raw field access. I don't buy the encapsulation thing if my code is not a library for wide internet to use.
[1] https://projectlombok.org/features/GetterSetter
Try Kotlin then.
I wouldn't call it a new language, for me it's just a syntactic sugar over Java, but for any problem you would google "how to do X in Java", not "how to do X in Kotlin".
But there you can do way simpler syntax, like:
Because "0..100 meters ..." is equivalent to "(0..100).meters(...)"(0..100) is a built-in IntRange type, that you can extend:
andIMO Kotlin has some features that take it way beyond just being syntactic sugar over Java. Like, I know you could probably use some eye-watering gang-of-four pattern to achieve the same effect as extension methods. But it's going to be such a PITA to maintain that calling the Kotlin feature syntactic sugar for the same thing is about as useful as saying that function definitions are just syntactic sugar over assembly language calling conventions.
I've played with it for a bit but it doesn't excite me enough to spend time learning it properly. I also don't care about Python beyond using it for learning algorithms and maybe some smaller Pyside UIs. However, together with Mojo it might become more interesting again.
> Java, for its part, can also accumulate a lot of dynamic language-style performance losses to pointer chasing and run-time type lookups (and GC churn) if you're not careful about how you use generics.
The worst thing about Java is the average quality of Java programmer.The same could probably be said about Python. However I think that there are fewer Python programmers trying to write AbstractFactoryFactory than in Java. Java has a terrible culture of overly verbose, deep inheritance trees that make debugging and development worse, with worse performance.
I certainly think so but wasn’t sure if it was just my unfamiliarity or lack of advanced programming knowledge but I attempted what seemed like a very simple patch to a Java project (guacamole) and it was insane how I ended up having to add the new function to like a base and abstract class and interface etc. it was crazy. All the same boilerplate too.
I have a related feeling or prejudice that people not in the Java World view their corner as "the tech industry" without realizing that they are living in a village compared to the giant metropolis that is the Java World.
Java programmers may not blog as much, and Java doesn't show up on Hacker News as much, but not being Extremely Online does not mean that it isn't extremely widely used by real people whose experiences are just as valid.
you could say the same thing about a dozen other languages, but there is definitely a stereotypical cranky java dev that is frustrated writing corporate crud apps that should in theory be blazingly fast, but never quite are for reasons. They revel in the verbosity, and look down on anyone they feel is taking the easy path. If you've never met one you're lucky, but they've been at every company I ever worked for.
> you could say the same thing about a dozen other languages
No, you could not. You could say it about maybe four others: PHP, C, C++, and C#.
No other languages have anywhere near the userbase size while being fairly quiet when it comes to online tech discussion.
I agree there are crusty old Java devs (as well as crusty old C, C++, PHP, etc. devs). In a decade or two, there will be crusty old TypeScript devs. It's just the nature of technology lifecycles and career paths. Some people get relatively tired of learning when they get older and just want to capitalize on what they already know for their remaining earning years.
I think I made a conscious effort in my 30s not to become a crusty old Python dev. But I predict that in another decade or two, I (now in my 40s) will be a crusty old Rust dev.
ok I may have exaggerated, but there are quite a bit of people silently writing and maintaining mission critical code in Fortran, Lisp, Cobal, Perl, Pascal, R, Assembly, the different BASICs, Lua, TCL, Erlang, and the list goes on. Plus those are just the ones off the top of my head where I have personally met people doing it in the last decade. I am always shocked by how much
Most of those rarely make to the top of HN, and other generalized forums. If anything, Java and Python are together at the popular kids table and we forget about the silent heros keeping the ship afloat.
I worked in .Net land before switching to Pythonville. It's very much like Java metropolis, but with less CS graduates.
If the TIOBE index is an even remotely useful indicator, Python's an even bigger giant metropolis.
Yes, but the difference here is that Python is talked about all the time, so the online tech world knows that Python is huge.
The comment I was replying to seems to believe Java is a tiny backwater, which is anything but true.
I read it less as "Java is tiny" and more as "Java developers can be peculiarly insular and conservative, by the standards of other communities."
Considering that as recently as 4 years ago I was working on a project where we still had a hard requirement to support running in Java 7, and this kind of thing was not considered unusual, I can't really disagree too strongly with that. Yes, that was still inside of Java 7's extended support period, so there was really nothing unusual or surprising about this, from a Java developer perspective. But that's kind of the point.
It's also not really a bad thing, considering what kinds of things run on Java. Mainframe developers have a similar thing going on, for a similar and similarly good reason.
I'm not sure if they refer to how large the language's reach is, but more about how advanced Java is once you stop looking at syntax bloat. The JVM can do stuff that's not easy to do in other environments.
I recall things like updating packages/code on the fly, recompiling fast paths on the fly. Maybe that's not necessary in a borg/kubernetes world where you can just restart things and have your load balancer infra take care, or just run slower code because compute isn't that expensive once you accelerate your CPU-heavy libraries, but cool anyways.
Lest we forget that Python is ~4 years older than Java.
This is absolutely correct as someone who has worked in a few Java shops. Although, the same thing is Java's failing, as it is well nigh impossible for people external to the Java ecosystem to learn what's inside it without having work-related exposure.
You may know all this and are just singling out the Hacker News crowd. But I read your comment and thought "surely he doesn’t think Java is much bigger than Python?" I’m not even sort of sure Python is smaller.
Edit more succinct
The reason that people still code in Java (or derivative) is because legacy code that they are working on is in Java, and nobody has either the skill or time to go through and translate it. Which means that the jobs where its used are basically just big enterprise, low tech software that just been around for a while.
The Log4shell incident is the perfect demonstrator of what kind of people are in Java world.
> nobody has either the skill or time to go through and translate it.
To what? Java is still a very efficient, productive language. Updating a legacy codebase to use newer Java features would probably be good, but migrating to another language is unlikely to significantly move the needle in terms of runtime performance or developer velocity.
Most of the use cases for java are in places where network latency dominates. If Python is fast enough for Uber and Youtube, its fast enough for your service.
When you work with a codebase that doesn't need compilation, the development velocity is quite fast. You don't need to compile, you can prototype features on the fly, you can even write real time updates while the web server is running.
And standard compilation in Java is done in a VERY inefficient manner with groovy - you have groovy that gets compiled to bytecode, which then gets JIT compiled to native, which then actually runs the compilation of the source code to bytecode, and then you have a first startup latency as it gets JIT compiled to native.
All you really need to write any modern app is Python + C. Anything that needs to go fast, just encapsulate in a small C program, and launch that from Python.
I feel like the opposite is also true. People view Java as enterprise Java from 2008; a clunky outdated language for bad, verbose spaghetti code.
In fact, a lot of the most interesting plt and compiler r&d going into real world applications is on the jvm (project loom, graal etc), and the features of modern Java (pattern matching, records, etc) make it a great choice for lots of projects that aren’t legacy enterprise apps.
I gave a talk recently at a conference about how modern Java doesn’t suck very much, and that the worst part of Java is Java developers, who seem to be completely intellectually unambitious.
I don’t think Java makes anyone unambitious, I think it’s that Java is taught in schools and unambitious people don’t feel the need to learn anything else, and they get jobs at unambitious corporations. It selection-biases towards unambitious people who don’t want to learn more than they have to.
Compare this to something like Clojure or Haskell or something, which isn’t routinely taught at schools and is not very employable. People who learn these languages generally seek out these things because they’re interested in it. This selection-biases towards intellectually ambitious people.
As a result, Java people can be insufferable for people like me.
The people who make Java have actually made the platform and language pretty ok in the last two decades, but I had to fight at a previous job to use NIO, which I think was introduced in Java 4, but none of my coworkers had really heard of it or used it because the regular Java blocking IO has been “good enough”.
>how modern Java doesn’t suck very much,
Given fact that Lombok is still pretty much widely used, with its under the hood functionality of essentially hacking the AST, or the fact that annotation processors write out Java code to files, or the fact that you could be using a standard library like Log4j and have a massive vulnerability on your system because someone decided that it would be a good idea if log statements could execute code and nobody said anything otherwise, or the fact that Kotlin and Groovy were literally made to address inefficiencies in Java, amongst other things....
Yeah not really sure how you came to that conclusion.
That gets to my point though. For example, Lombok isn't really necessary for a lot of stuff now, because Records give you a lot of what you would use with Lombok.
Kotlin and Groovy did come and address problems with Java, you should use use them if your employer allows it. I'm just saying that Java 21 is actually kind of fun to write.
Yes, some of the libraries have been unsafe, but that's one example of the 30 years of Java.
I just feel like Java has improved in the last twenty years. It's the engineers that haven't.
link to talk please if recorded.
I think one comment I saw here on HN that Java is better if written in Pythonic way. I agree completely with that stance.
but yeah within Java you've 'merchants of complexity' people who wanna do things in the most abstract way rather than the simple way.
btw Java can be as simple as Go.
I don't think that the talk has been released on public YouTube yet (and they made it clear to not share the unlisted links publicly), but here is the posting: https://www.lambdadays.org/lambdadays2025/thomas-gebert
And the slides are available here: https://github.com/Tombert/lambda_days_2025/blob/main/slides...
I'm afraid that my humor isn't really reflected in the slides, but imagine everything here is said kind of snarkily.
Java can be mostly as nice as Go, the BlockingQueues and Virtual Threads can get you pretty far, though they're not quite as nice as Go channels because there's no real way to select across multiple BlockingQueues like you can with Go channels.
Overall though, I think Java 21 is actually not too bad. Shockingly, I even sometimes have fun writing it.
Github link is 404
Oops! Forgot to make the repo public!
Should be fixed now. Sorry about that.
> I have this feeling - or prejudice - that people still in the Java World are a bit out of tune with the tech industry
Other than being ageist, it’s wrong; or misattributed to Java. I work with Python every day, and what’s missing is static typing and IDEs that make use of it to greatly reduce the amount of code and context I have to store in my head. Python (a dynamically typed language obviously) is exhausting to maintain. But easy to write. Java/C#/whatever statically typed language with great IDE is easy to write and maintain by comparison.
Of course there are IDE for Python and dynamically typed languages, but everyone I’ve tried has fallen short compared to the best Java/c# IDEs.
Static vs dynamic used to be a huge flame war on the internet, but over the past few years I’ve encountered people who’ve never heard of it. This is it.
I think at least half of amazon.com and AWS run on Java.
I've always felt X was far superior to Y, and don't get me started on Z or W.
I personally dislike Python, but it does surprise me that anyone is acting like it's generally disliked because that has definitely not been my impression since time immemorial. I doubt that it's gone out of fashion. More likely, acting like something actually doesn't suck after all has become the clickbait framework du jour.
A decade ago, Python was pretty rough around the edges in production. Of course you could make it work, but you had to sacrifice a lot in terms of:
- Environment / dependency management
- Type safety
- Performance
As the author points out, these have largely been addressed now by uv, type hints, pydantic, FastAPI, etc.
>A decade ago, Python was pretty rough around the edges in production.
Not really.
Environment/dependency management is/was never an actual problem. People act like you update a version and stuff breaks. Even then, venv existed for this reason, and you could specify version in the setup.py or requirements.txt.
Type safety should in theory be covered by unit tests. If you assign a variable to another one in a dynamic setting accidentally, like a string to a dict, then your functionality breaks. In reality though, its really not that hard to use different variable names. In my experience, the only time things start getting confusing is if you start using Java concepts with things like Factories and a lot of OOP and inheritance in Python. None of that stuff is really necessary, you can get by with dicts for like 90% of the data transfer. And in very type safe languages, you spend a lot of time designing the actual type system instead of just writing code that does stuff.
Performance is still slow, but language performance doesn't really matter - you can launch natively compiled code easily from Python. Numpy was built around this concept and is also 10 years old. You will never beat C (with explicit processor instructions for things like vector math) or CUDA code, but most of your code doesn't require this level of performance.
Typing still sucks big time. Same for perf, unless you are working with numerics that fit with something like NumPy or JAX.
I haven't tried uv but my biggest pain point with Python has been the way env works. I always find it painful and odd.
Backward compatibility, which I suppose is closely related to needing to use env, is also a pain. In my experience you can't go forward or backward in many cases. It's especially painful on projects that don't change very often. I'm sure that an active codebase can probably be kept updated from version to version, but if you've waited a bunch of versions, it seems painful.
But, I'm not sure I've given it a fair shake because I haven't needed to. It's use in AI does make it an attractive option, now more than ever.
I think a combination of:
- statically typed languages got better, reducing the relative benefits of dynamic typing. people realized they didn't really hate static types, they hated the way [insert 90s-00s enterprise language here] did static types
- the GIL became more and more painful as computers got more cores
- it used to be a language for passionate hackers, like a latter-day lisp (cf that old pg essay). "written in python" used to be a signal of quality and craftsmanship. now it's the most commonly taught beginner language; millions of bootcamp devs put it in their CV. average skill level plunged.
- PSF was asleep at the wheel on the packaging / tooling problems for years. pip/venv are dinosaurs compared to cargo, nix, or even npm.
> Is this just junior devs who never learned it
Seems more like it's fallen out of favor with senior devs who have moved to Go/Rust.
I don’t know anything about go. But Rust is more of a competitor to C and C++, right? It is sort of bizarre if these languages are butting heads with a scripting language like Python.
Python compares fairly well to Bash or JavaScript or whatever, right? (Maybe JavaScript is better, I don’t know anything about it).
Rust has language features (often inspired by functional programming languages) that allow you to write pretty high level code.
I consider this self-inflicted. Pythons Async functionality is … unfortunate.
JavaScript has much more intuitive async syntax, which was actually borrowed from a Python framework.
For whatever reasons, the Python folks decided not to build on what they had, and reinvents things from scratch.
you'd be crazy (senior or not) not to use Go for Go stuff and Python for Python stuff
I use both, with a preference for Go but I feel like I should be doing more Python just to keep it fresh.
It seems like two of the main entries under “Python stuff” are “working with people who only know Python” and “AI/ML because of available packages.”
What are some others?
I mean... to oversimplify a bit, Python is for scripting and Go is for servers.
I think for a lot of us, it was very frustrating when it was being pushed with no real analog for the practices we had in other languages. Poetry used to be pushed as how to do dependency management, but it was not obvious that it was not necessarily your build manager, as well. Even today, I'm not entirely clear what the standard approach is for how to manage a project. :(
If I were to try again I'd go with uv, it seems way way better
I have been sticking with poetry for a while, now. What would make me want/need to move to uv?
- Performance: uv is so much faster that some operations become transparent. poetry's dependency resolver is notoriously slow. uv being a native binary also means it has a faster startup time.
- Interpreter version management: uv can handle separate python versions per project automatically. No need for pyenv/asdf/mise.
- Bootstrapping: you only need the uv binary, and it can handle any python installation, so you don't need to install python to install a management program.
- Environment management: uv will transparently create and update the environment. You don't need to source the venv either, you use `uv run...` instead of python directly.
Overall, it makes common Python project management tasks simple and transparent. It also follows standards (like project definition metadata and lock files), which poetry often doesn't.
I'll pay attention to poetry soon. As is, I don't recall my builds ever going slow because of poetry. I don't think I've noticed it have any impact on speed, at all.
I did just update the dependencies of some of my projects. I could see how that could be faster. I don't do that often enough for me to care about it, though. `poetry run pytest` is the slowest thing I have, and I'm confident most of that slowness is in my direct control already.
I'm intrigued on the lock file point. I thought that was literally one of the main reasons to use something like poetry, in the first place? Does uv have a better lock file mechanism?
uv supports the standardised lock file format. This means that you're not tied to uv for anything, as most project settings it uses (eg metadata, dependencies, extras, etc.) are based on PEPs. poetry has its own format for many things.
https://peps.python.org/pep-0751/
Ah, I see that this is a newer format than the projects I've been using poetry for. Are there advantages to move to it?
Interoperability being the major one. You can switch between uv and other tools (such as pdm and hatch) with minimal effort.
Maybe you'll find this series of articles interesting: https://www.loopwerk.io/articles/tag/uv/
Thanks! I'm not entirely sure I see a reason to change on there, oddly.
I'm very fortunate that my python projects are all relatively small, so maybe that colors things a bit. Certainly looks like something that would have swayed me to uv at the start, but as things are, I think I mainly just wish there was a more standard/accepted work flow for build and release.
Makes sense. I've stopped using python for development but I run into python utilities I want to use all the time. uv/uvx are perfect for someone like me.
yeah Poetry is fine
I coded happily in python for many years and fell out of love with it.
It doesn’t ship with a first party package manager so you got the community trying to fill this gap. Use any other language with good tooling like golang or rust and it is a breath of fresh air.
Python used as an actual PL is a footgun because it’s dynamic scripted. (Don’t tell me about using tools X, Y, Z, mypy, …) You essentially become the compiler checking types, solving basic syntax errors when uncommon paths are executed.
A programming language that’s only good for < 100 line scripts is not a good choice.
I honestly wish python were in a better state. I switched to rust.
> Don’t tell me about using tools X, Y, Z, mypy, …
What's wrong with using tools that improve on common issues? I don't think I'd use Python without them, but ruff and pyright make Python a very productive and reliable language if you're willing to fully buy into the static analysis.
> A programming language that’s only good for < 100 line scripts is not a good choice.
What a bunch of crap. It's so trivial to show very popular and useful programs written in Python that far exceed this number I'm not even going to do the work.
What a lazy criticism.
Hi rsyring, I made this comment out of experience.
As python projects grow and grow, you need to do lots of support work for testing and even syntactic correctness. This is automatic in compiled languages where a class of issues is caught early as compile errors, not runtime errors.
Personally I prefer to move more errors to compile time as much as possible. Dynamic languages are really powerful in what you can do at runtime, but that runtime flexibility trades off with compile time verification.
Of course, every project can be written in any language, with enough effort. The existence of large and successful python projects says nothing about the developer experience, developer efficiency, or fragility of the code.
All perfectly valid perspectives and I agree with most of what you wrote. But the comment above is pretty different from the tone/effort behind the comment I took issue with. :)
In hindsight, I should have just left it alone and not replied which is what I usually do. But Python's popularity isn't an aberration. It's tradeoffs make sense for a lot of people and projects. The low effort bad faith swipes at it from subsections of the HN community got me a bit riled today and I felt I had to say something. My apologies for a less than constructive critique of your comment.
Best.
Thanks for your comment. I definitely could have worded mine better too with a bit more effort and context.
From the community guidelines:
> In Comments
> Be kind. Don't be snarky. Converse curiously; don't cross-examine. Edit out swipes.
> Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
> When disagreeing, please reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3."
> Please don't fulminate. Please don't sneer, including at the rest of the community.
> Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
https://news.ycombinator.com/newsguidelines.html
There are massive numbers of devs who would never even think of trying to code in a dynamically-typed language, even though the big players all have gradual typing. They don't know what they're missing.
I have no problem with dynamically-typed languages, my main problem with Python is the significant whitespace. I really do not like it. I can deal with everything else in Python, or any other programming language, but significant whitespace is what kills it for me.
Even after writing tons of python the whitespace is still the reason why I never reach for it by default.
Today the IDEs got much better, but I still can’t see the significant whitespace as anything than downside. With brackets I can indent, but also have automatic indenting succeed every single time.
Having gotten back into Python for a project after some time away, I found the solution (for me) is to automatically fix the whitespace every time I run the code.
I just have a line in my Justfile that does this. Probably would be better to format on save but I use different editors and haven’t gotten around to it.
Still doesn’t fix the creeping doubts about everything in a language conceived by people who made that whitespace call, but it removes the specific pain point.
do you mean requirement of whitespace or how much space that takes up? Because you can change how many spaces python needs for its indentation to get some reduction in whitespace
I've been writing code for 40+ years. I know how to set up my editor, thanks.
If you copy and paste some Python code and it isn't indented properly, it breaks the python program. It's the stupidest thing I've seen in any language, including javascript (which isn't as stupid as many claim it is).
Yes, I'd say Python's crown as the go-to lightweight scripting and web development language was mostly ceded to JS.
It's still reigning champion of data science, and of course it has a huge number of uses and users still around, but it's not really cool or in vogue outside of those circles.
Python has never been THE web development language, that's always been JavaScript. Python is one of the more popular server-side languages, and JavaScript has move from frontend to backend in the last decade too. Not sure whether this has really taken the crown of any backend-language.
And what is lightweight scripting? Isn't scripting by definition lightweight?
But Python overall is still very popular outside the Data Science-circles. Not sure where this claim is coming from.
> It's still reigning champion of data science, and of course it has a huge number of uses and users still around, but it's not really cool or in vogue outside of those circles.
This is a wild take. You're never going to get a fully accurate measurement but every source I've seen[0][1][2] puts Python as the most common programming language by a country mile.
If it doesn't seem "cool" or "in vogue", that's because everyone is already using it.
[0] https://www.tiobe.com/tiobe-index/ [1] https://pypl.github.io/PYPL.html#google_vignette [2] https://www.pluralsight.com/resources/blog/upskilling/top-pr...
Do people really turn to JS (instead of Python) for lightweight scripting? Are we talking about the "better Bash" use case?
JS devs do, and there are a lot of them
I use "tsx script.ts" often and it's very nice good due to gradual typing you can use (plus I know lang well)
I mean if its really light weight, and I want to avoid bash, then JS is not too bad. The downside being that the built-ins are not really there compared to Python.
The famous answer.... _it depends_.
that would be very surprising
When it got big enough that companies were forcing people to learn it. People automatically hate things they didn't choose themselves.
Plus there has been a rising sentiment against dynamic typing by masochists over the last decade or so.
HA! Alright Fair enough :-)
It's not out of fashion. We're just seeing the long tail finally picking it up. It's becoming somewhat of a lingua franca of programming.
After using Rust's tooling, Python's tooling is obscenely painful. Like intolerable. Which makes some of us feel overwhelmingly frustrated with the language itself. The amount of time I've squandered on Python because of its tooling... I'm never getting that back.
But then uv came along. It's not just another poetry or pipenv or whatever. It works well and it has uvx and `uv tool install` and other nice things.
Previously when I saw that something was written in Python I'd curse under my breath because I don't have the mental energy to make a virtual environment and source its shell script and remember where I put it just to try something out. On arch Linux I can't just pip install something without going through some crazy setup that I used to have patience for but as I get older it's been better to just curse that snake language altogether and spend time doing something more chill like writing rust.
But now I can just type "uvx" and it works. I'm probably not going to start writing python again any time soon, but at least now I have less reason to be disappointed with people who themselves choose to write Python.
The v2 to v3 transition threw a lot of people.
Can we please stop reiterating the same old stories? That was years ago. Most Python devs these days never wrote a single line of Python 2. I learned it 16 years ago and, while frameworks like Django were still in the middle of the transition back then, I pretty much started learning & writing Python 3 right away.
This might be true but I still find myself typing:
print "xyz"
now and then. It's not a big deal, but it reminds me of struggling with pip->pip3 and many other things for a long time years ago.
Hah that brings back memories of me smugly insisting to my freshman roommate that using Python 3 tutorials to learn programming was a complete waste of his time and he should be using 2.7 for life like the rest of us l33t hax0rs.
Tbf at that point Django was still pretty shaky with 3 and basically none of the 3rd party Django libraries supported it at all, plus I was using Google AppEngine which at the time was tightly coupled to the 2.7 runtime. But really I was just parroting the Slashdot hivemind which was 100% convinced the transition was the new Perl 6 and would kill Python, and that Python.org was dishonestly teaching newbies who didn't know better a dead language and worthless skill when they changed the default.
Fortunately for him he ignored me and most of the big Django libraries were ported like a year later at which point I had to switch anyway to get updates. Fully agreed that in 2025 it's pretty much irrelevant, and honestly despite some legitimate pain the transition was much more successful than the cynics assumed it would be at the time.
Python is probably too awesome and versatile to go out of fashion, but as a long time user.. the ecosystem is frustrating. Dependencies and packaging have been, and still are a nightmare after thousands of years. Problems are fixable sure but the thing is that they never end. After you get plenty of practice fixing all the related problems, you'll have to keep fixing them for yourself, your less savvy teammates, and in third-party code pretty much forever. This is worth it in exchange for "import antigravity" for the first 100 years, but it's frustrating eventually.
Everyone will mention uv/pyenv/poetry/conda/virtualenvs, so fine, let's pretend it's not a problem that you tried each of those in desperation and they are all household names. Suppose the packaging wars actually ended and everyone uses what you use without you needing to tell them, and suppose further that every pypa problem isn't blaming debian maintainers for obvious regressions. Pypi will still yank[0] packages at the source to perhaps randomly break deterministic behaviour, smashing anything in the blast radius rather than only clients using a --strict flag or something. You can pin your own dependencies but who knows what they will pin (nothing, or the wrong stuff, or the wrong way probably!) or what they will yank. Now for repeatability you need to host a mirror for everything you use- which is fine for corporate but a nonstarter for the novice and annoying for FOSS projects.
If you have zero dependencies or a slowly changing environment that is receiving constant care and feeding, you'll probably never notice how bad things are. If you put down most projects for a month though and pick them back up, perhaps with a different environment, machine, or slightly different python version it's broken, bitrotted, and needs serious attention to be rehabilitated.
People might argue.. that's just software dev. Nope. I say it with lots of love and gratitude for the efforts of the community.. but most langs/ecosystems would never tolerate this level of instability. One has to eventually just admit that working, portable, and actually repeatable environments with python basically just require docker. Can we talk about how "--break-system-packages" is hilarious? After you retreat to docker you can type this a few times a day, push the container up, pull it down months/years later, and then realize that literally the only way to get a stable working environment is to request a broken one. QED
[0]: https://docs.pypi.org/project-management/yanking/
I think during the python 2-> 3 transition period (which was almost a decade long) a lot of people started looking into alternatives, because frankly that mess was the last straw. In the scientific domain this is when Julia started growing.
Of course then crypto bros happened and the rest is history.
I love writing Python. However, there are caveats:
1: I don't like dealing with language crossing boundaries with it - it's painful, especially if to/from a compiled language - there's just too much friction. It's easy to write but hard to maintain.
2: Debugging python can be...painful. If you have a pure perfect environment dedicated to pure python and all the tooling set up, it can be breezy. But if you have something messier like C++ that calls Python or vice-versa and are using an editor not made for Python like QTCreator then suddenly it becomes hell.
Who owns this data!? Where was this initialized!? What calls this function!? What type is this data!?!?!?!? These questions are so effortless in C++ for me and so very painful in Python. It slows efforts to a crawl.
It feels exhausting to have to go back to print statements, grep, and hand-drawing a graph that shows what functions call what just to figure out WTF went wrong. It's painful to a degree that I avoid it as much as possible.
...and don't even get me started on dealing with dependency problems in deployments...
I've long lusted after a smooth debugging workflow writing Cython that would let you seamlessly debug between Python and C/C++. I think there are some hacks to do it, but nothing really well integrated.
Python debugging is great when you're strictly debugging Python code. Placing a `breakpoint` or spinning up an IPython instance to investigate the issue via `import IPython; IPython.embed()` makes life breezy. For neovim, you can send code to the ipython REPL easily via vim-slime.
I've never found myself debugging the underlying C++ code unless developing a C++ extension. But is it really that hard? Just point lldb to the python process and run your script in lldb.
If the C++ is not yours & assuming it's a mature lib (e.g. PyTorch): it's probably an error caused by you in Python land.
Both the C++ and Python are mine. I have cases of both C++ calling Python, and Python calling C++. Problems could be on either side.
The problem is how to step into one from the other, you really can't as far as I know.
The python parts aren't standalone enough that it could just be run on its own, there's so much setup involved prior to that point that it can really only be run from the top-level user controls.
I really don't find the Python language elegant at all. I prefer the Ruby syntax.
But Python's tooling, particularly with what Astral is doing (uv, ruff, ty), is so good, I'm always using Python and virtually never using Ruby. And yeah, the rich libraries, too.
Jupyter is also great thing in python ecosystem + colab etc.
YAML, Python, Make... — only I see issues and a pattern with this choice of toolset?
They occupy a space where things work most of the time but jank and footguns are a way of life. That seems to be where most people live unfortunately.
> And guess what’s the de facto programming language for AI?
I thought nodejs/typescript seemed to be the default that most LLMs choose? Or is that just v0/lovable/replit? (although replit seems better about going non-js sometimes)
Python as a language is nice. Python's version and package management is nothing short of a nightmare.
This hasn't really been true for a while now. `uv` has radically improved the experience.
For the last 20 years that has been the mantra. Some X "solves" all the problems.
Except it doesn't. It just creates another X that is popular for a while, and doesn't somehow retroactively "fix" all the chaotic projects that are a nightmare to install and upgrade. Yes, I understand people like Python. Yes, I understand the LLM bros love it. But in a real production environment, for real applications, you still want to avoid it because it isn't particularly easy to create robust systems for industrial use. You may survive if you can contain the madness in a datacenter somewhere and have people babysit it.
I think that's true until it isn't, and people are really rallying around uv.
Here's to hoping it manages to actually solve the Python packagig issue (and lots of people are saying it already has for their use cases)!
Solving it at least involves the Python maintainers making a choice, integrating it into the Python binary and sticking to it. At least. But that requires possibly annoying some people until a) whatever solution becomes mature and b) people get over it.
I've ignored the trends and just used the bog standard requirements.txt with pip and a virtualenv and have had no problems in the past 10+ years. Either way, you always want to catch production deploys if something breaks.
"production deploys" sounds like something that is in a datacenter.
>Except it doesn't.
That is only true if you never reexamine the universality of your statement. I promise that it is possible to "solve" the mess that was Python's ecosystem, that uv has largely done so, and that your preconceptions are holding you back from taking advantage of it.
Here's the thing: there has never been a lack of people who have declared this problem as solved in the 20 or so years since Python started Poking its way into my professional life. (And for about 12-13 years before that I could gladly ignore it since nobody did much of anything in it). People have said this since the days of the Blackberry.
Multiple times people have explained why they think whatever they are madly in love with now is the definitive solution. And none of those times, over those couple of decades did it turn out to be true.
I understand that you are enthusiastic about things. I get it. But perhaps you might understand that some people actually need to see things stick before they declare a winner? I'm not big on wishful thinking.
I would take a look at uv adoption. That's what makes it different. It nails everything that all the other tools have done and it does it fast. So it's been what people have been using for a while now. Even poetry never seemed to get this ubiquitous of support.
Some of the biggest codebases in the world are in Python, this is a bizarre statement that reeks of the hn superiority complex.
Every single language enthusiast says that some of the biggest codebases in the world are whatever their favorite major language is. And here's the thing: it is completely irrelevant whether the codebase is small or large. What counts is what it is like to use and maintain programs.
Python isn't the only language that has poor tooling. C/C++ is even bigger than Python in terms of established code base, and its tooling is nothing short of atrocious.
What helps is people realizing where tooling and production readiness should be. They can learn a lot from Rust and Go.
The it's big so therefore it must be right argument is nonsense. Worse yet: it is nonsense that excuses lack of real improvement.
> But in a real production environment, for real applications, you still want to avoid it because it isn't particularly easy to create robust systems for industrial use.
This is silly and seems to discount the massive Python codebases found in "real production environment"s throughout the tech industry and beyond, some of which are singlehandedly the codebases behind $1B+ ventures and, I'd wager, many of which are "robust" and fit for "industrial use" without babysitting just because they're Python.
(I get not liking a given language or its ecosystem, but I suspect I could rewrite the same reply for just about any of the top 10-ish most commonly used languages today.)
I can get that Pythons not for everyone, it certainly has its flaws and maybe uv is just another transient solution which will come and go and others have. I might disagree, but I can accept that. What I can't accept is the idea that it should be avoided for real production environments, which is frankly a bit ridiculous considering all the real applications and real production environments running on Python.
There’s still 20 years of projects using everything that became before uv. They didn’t upgrade the moment uv came into existence. Data science-land still uses other rubbish too.
> They didn’t upgrade the moment uv came into existence.
There's also projects that can't use `uv` because it doesn't like their current `requirements.txt`[0] and I have no bandwidth to try and figure out how to work around it.
[0] We have an install from `git+https` in there and it objects strongly for some reason. Internet searches have not revealed anything helpful.
Unrelated to uv but the problem with having a git ref in requirements.txt is that pip will treat it as a fixed version so it will strictly narrow the other dependencies it can resolve, which gets exceptionally difficult to reason about once that packages also loads a package from another git ref. Throwing everything into codeartifact (or equivalent on other clouds) is better longterm.
If you open even a brief issue and tag me @zanieb I'm happy to take a look!
Ta, will give that a go when I've got some free time.
Python's package management is OK. The main culprit is .so libraries
Think JNI or cgo management.
Yes, I've never really understood the complaint about python packaging - building native code is not something that is ever easy to guarantee across multiple distributions and operating systems.
Those native packages can be in any language and require any odd combination of tools to build. Who has truly solved that problem?
If you don't need to link C lib, you can build any combination of arch and OS for a golang program. The default tooling allows you to do so easily.
If you need to link a C lib, there are ways to set it up to compile other OS (and maybe other archs).
take numpy as an example, it's gfortran mixed with C. How does cgo handle that?
And there's ffmpeg....
If you got object files you can compile whatever if you have the compiler toolset. I have made a go program that links ffmpeg and is built from linux to Windows and Macos. It was not super easy but it's doable.
wheels, conda, docker, firecracker, unikernel, flatpak
flatpak, docker ? i.e. include an almost complete distribution just to make one package work? but what if you want that package to work in the distribution you have now? What if you need 2 different packages that are in different flatpacks or different docker images?
Set them up, each in its own container, and use GRPC to communicate between them...
(Not even kidding, I've seen people do this)
Goodness gracious!
Well, if you're using GRPC anyway, and have the protocol buffers already... It's hard to resist the temptation after a few hours of installing broken package versions.
Incidentally, I suspect this is the spiritual origin of microservoces...
obviously don't try those first I'm just saying there's a stack of options to work with for any level of isolation you desire.
start with wheels if you just want pre-built binaries, move up to conda if you need arch and OS-specific C libs that depend on system packages, flatpack or docker if you dont want those to mess up your host, or unikernel/firecracker/VMs if you need kernel modules or hardware virtualization.
I'll throw in BLAS and LAPACK. Fuck, what a nightmare it always has been to get scipy running. I've always ended up with a wild mix of conda and pip installed shit that just wouldn't work.
Really? `pip install scipy` in a new environment just works for me. What concrete issues are you encountering?
It is not okay (maybe uv fixes this?)
pure .py libs can be "vendorized" directly into project.
I expect that for every two python users who know what `pip` is, there are three who are discouraged from thinking about the environment at all. Given that, focusing on the language and not the packaging is the right choice. Packaging is most often somebody else's problem.
It's just HN users are more likely to be that somebody else. Probably we have to deal with non-python dependencies anyway so we're reaching for bigger hammers like docker or nix. It would be nice if there wasn't a mess of python package managers, but whichever one I use ends up being a somewhat unimportant middle-layer anyway.
Going to be honest: With uv it really isn't that bad anymore. Sure, the whole packaging ecosystem is still really bad, but uv abstracts over most of it. Unless you're doing some really esoteric stuff you'll be just fine.
Using a nix devshell has worked out well for scripts for me so far. I haven't figured out what that workflow is like for larger projects though. I'm not interested in learning Uv
As a nix user, you hardly need to know uv, but you might later need to know how to work together with non-nix-users who use uv. It's easy.
There will be a file: uv.lock You can use uv2nix to get a single nix package for all the project's python dependencies, which you can then add to your devshell like anything else you find in nixpkgs (e.g. right alongside uv itself). It ends up being two or three lines of code to actually get the package, but you can just point a LLM at the uv2nix docs and at your flake.nix and it'll figure them out for you.
Your devshell will then track with changes that other devs make to that project's dependencies. If you want to modify them...
This way you're not maintaining separate sources of truth for what the project depends on, and the muggles need not think about your nix wizardy at all.Didn't I say I wasn't interested?
You created a place where the interaction between nix devshells and uv was relevant, I put something there to help passers-by that might be interested in those topics. It wasn't actually for you.
There's nothing to learn. Get Claude code to install uv into your devshell and cut over your requirements. 5 minutes.
AI? :vomit_emoji:
Or do it yourself, 10 minutes
It's like the (usually) interpreted equivalent to C/C++. There are lots of 'standard' package management choices.
And it seems like the package resolution is finally local by default, although that requires a 'virtualenv', which seems to be a legacy of the global packaging system.
> I started to code more in Python around 6 months ago. Why? Because of AI, obviously. It’s clear (to me) that big money opportunities are all over AI these days.
I find this depressing. Not only are LLMs covertly reducing our ability to think and make decisions, they’re now also making people voluntarily conform to some lower common denominator.
It’s like humanity decided to stagnate at this one point in time (and what a bad choice of point it was) and stop exploring other directions. Only what the LLM vomits is valid.
Eh, I just wrote a chrome extension for the first time through AI. If it automates the boring stuff for us, I'm more okay with this than not.
humanity tried to save time and allocate it to other pursuits, don't worry
I don't see how the OP layering utilities on top of python remediates on the claim that the language is not "production ready". Which it clearly is.
I get the “actually liking it part.”
I have been enjoying Lisp languages since the late 1970s, and today it makes me happy using Common Lisp and Racket in the same way as when I stood in a forest very early this morning drinking coffee makes me happy.
But, Python is also a fun language and phrases like “good enough” and “actually liking it” are valid.
I was like, switched from what, then eventually found a footnote "I used to be mostly a Java/JavaScript/R kind of guy"
If you're working on machine learning the most economic choice is Python.
But weiting a processing pipeline with Python is frustrating if you have worked with C# concurrency.
I figured the best option is Celery and you cannot do it without an external broker. Celery is a mess. I really hate it.
Agree. I think it's improved a bit but Celery is frustrating as the defacto job/queue solution. A lot of the defaults make it unreliable (it can lose jobs if workers crash or don't shutdown cleanly)
I'm hoping the existence of free-threading will push for more first-class concurrency primitives. Concurrent Futures is nice until you need a concurrent-safe data structure besides a queue
Agree that celery is a mess and it doesn't work well with async (Asyncio) python. I think version 6 maybe will support it sometime.
I also had a lot of problem due to async primitives with sqlalchemy - there's some tricky stuff with asyncio.gather vs TaskGroup and how sqlalchemy session works with it to be able to compose code easily.
>> I prefer to use a monorepo structure
There is nothing more annoying than tons of little repos all of which containing tiny projects with a few hundred lines of code but (of course) you need most / all of them to do anything. Use a mono repo until there is some obvious reason to split it up imo.
in my experience, repo by "area" works the best. This usually means by team, but you don't want a team restructure to cause code relocation issues. And, yes, only when the org pressures push for it.
On the flip side, we have an org with 50+ teams and our operations team is pinning for a monorepo. They are just fine with one team's push forcing N teams to have an unexpected deploy and recycling of caches, connections, etc. Not to mention what will happen when team A doesn't have time to deal with team B's merge due to other org pressures.
Why do they want a mono repo though? What are the problems with the current situation.
consistent CI/CD, code discovery and sharing, and (I contend falsely) protection against backward compatibility issues.
I got the impression that TFA speaks of a monorepo in the sense of not splitting the backend and the frontend into two different repos.
For personal projects, though, I get the value of an actual small projects monorepo.
I'm not talking about personal projects. If you have <100 people, mono repo imo.
Ofcourse the most used programming language in the world is not a pain to use. How could it be #1 if it was?
I keep meaning to write something like this but exploring the “how simple can you make it” angle - a lot of my world is kube (which is great in the right scenario) but could you shave complexity out of a stack designed for solo dev rapid iteration:
e.g. rather than:
Chomp the complexity, serve HTML from the backend directly Im curious where ty goes but for a min-complexity stack i couldnt spend complexity tokens on pre release tools One or the t’other, plus i’ll forever confuse myself: is it post_init (dataclasses) or is it post_model_init (pydantic) - i had to check! if we already have uv, could we get away without docker? uv sync can give an experience almost akin to static compiled binaries with the right setup. Its not going to handle volumes etc so if you're using docker features, this concept isnt going to fly. If you're not wedded to docker though, can you get away with just uv in dev and prod? in an enterprise prob not, i wouldn't expect to be able to download deps in prod. For flying solo though… You’ve a frontend, a backend and presumably a database. Could you get away with just uv to manage your backend and a local sqlite db?So a broadly feature comparable stack for rapid iteration with less complexity but still all the bells and whistles so you dont need to cook everything yourself, might look like:
You could probably sketch a path to hyper scale if you ever needed it:if you want to serve HTML from the backend, why not use FastHTML then ;-)
i experimented with it, i’m a fan of the concept. Ironically the bit i thought id like most (swap jinja extends shenanigans for just composing functions or callables more generally) is the bit that i didnt warm to. I dont really know why, on paper it ticks boxes for me. In practice i felt slow.
What was your language of choice prior to Python?
Python is the 2nd best language at everything.
And the first best is python3!
Is there a Rust equivalent to this? I'd like to dive into a rust "api_starter" as someone with mostly NodeJS experience!
It feels like most people are coming into python from other languages. Wonder what is the emigration rate.
I wonder what the poor guy is switching from.
Java, JavaScript, and R, according to a footnote in the article. Which - to be entirely honest - sharply coloured my view of the preceding article.
"I went from being fully covered in mud to only being half covered in mud, and it's great! I don't understand why people complain about being half covered in mud."
"the second best language for any job"
Meaning the first best language depends on the job?
Of course :)
There are many jobs where the .Net/Java/Go triad would be in the top 3. I lump them together as typed languages that are mainstream, performant, have a GC and a featured library.
I'm at a java shop. All customer facing apps are in java. Our tools and glue code is mostly in python. Similar for data processing workflows
Yes that might demonstrate the point. Would Python be your second choice for the customer facing apps?
Unfortunately, might not be easy in practice. It took years of effort to pass some certifications and audits and I don't think adding another language could be recouped now
I don’t understand “switching” to anything. The job picks the tool, not the other way around.
I think it's mostly boredom-driven career choices.
I came to this comment thread expecting developer bikeshedding and I was not disappointed.
More seriously: it's fascinating and interesting to see how closely this article mirrors my own Python project layout. The tools and practices have come a long way over the last decade and good developer standards have a way of becoming the defacto in organic fashion.
IMO uv has won the race to be the new standard for Python environment management (Poetry gave it a solid try, but lost on speed).
“I started to code more in Python around 6 months ago. Why? Because of AI, obviously. It’s clear (to me) that big money opportunities are all over AI these days. And guess what’s the de facto programming language for AI? Yep, that sneaky one.”
Why is that?
Why Python for AI?
Because you get first class support in many AI libraries like PyTorch, TensorFlow and so on ... Most of these libraries can be used from other programming languages too, but it is easier to find good documentation and examples for Python.
Possibly because many available LLMs run your Python code in a sandbox, which means less friction for vibe coders, or it may be a contributing factor at the very least.
> Because many available LLMs run your Python code in a sandbox, which means less friction for vibe coders.
This is false, a lot of non "vibe coders" are using Python for AI because of PyTorch and a many other AI libraries have first class Python support.
How do you know that it is not the reason or not ONE of the reasons? Seems pretty reasonable to me to use ChatGPT, Claude, or whichever one supports it.
I am pretty sure some people (maybe this individual, too) may be using Python because their scripts can be executed in a sandbox on one of these websites.
Heck, if it was as good at Factor or Forth as it is at Python, I would be writing more of them, too.
In any case, you cannot claim that it is not one of the reasons. Can you?
I read your initial message, now edited, as this is THE reason. Of course it can be one of the reasons for which the author chose Python.
Also the vibe coding part gave me the impression that you were implying that people that use/chose Python for AI are all vibe coders which is again false. Sorry if I misunderstood you, but this is what I got from your initial message.
No worries. I think there was a misunderstanding because even with my original message, I did not intend to suggest that people who use or choose Python for AI are all vibe coders, or at least I didn't think I did hint at that.
Switched to python from what? R? Java? Javascript?
pydantic basemodel has made dataclasses redundant.
Dataclasses have the one massive benefit of not being an additional dependency.
This. You also don't really need Pydantic unless you're de/serializing data from external sources. A dataclass is perfectly cromulent if you're just passing around data internally.
Ubiquitous VSCode - honestly a tragedy.
Which package manager should I be using in my Python? I only just heard of UV
Unpopular opinion: I think I’m going to wait for version 4 /jk. But honestly, I’ve been spoiled by modern languages like Rust, Go, and even TypeScript with modern tooling, strong typing, stability, and performance out of the box. Right now, I’m just interacting with LLMs, not building them.
That said, I remember writing myself a note a few years ago to avoid Python projects. I had to clean up code from all over the company and make it ready for production. Everyone had their own Python version, dependencies missing from requirements.txt, three way conflicts between 2 dependencies and the python version, wildly different styles, and a habit of pulling in as many libraries as possible [1]. Even recalling those memories makes my stomach turn.
I believe constraints make a project shine and be maintainable. I'd prefer if you throw at me a real python instead of a python project.
[1] Yes, I'm aware of containers, I was the unlucky guy writing them.
I don't have a strong love for python, but all the tooling the author mentioned has actually made python fairly decent to use now.
Still could be better, but I think Python's really hit its stride now.
> Unpopular opinion: I think I’m going to wait for version 4
In my personal timeline, people giving up waiting for Perl 6 were a huge source of early Python developers.
What makes TypeScript better than Python? I don't get it..
Much more capable and reliable type system, paired with comparatively sane package management. This is getting better in the Python world (thank you Astral), but it's still not anywhere near the same.
Not sure if it's worth selling your soul to Microsoft.
I will stick to other languages when I need a better type system.
switching from what?
have fun!
So, i don't get it, now i still need to have a uv venv ?
The principle of uv as much as I understand is to not tinker with virtual environments. With commands like uv run, uv sync etc you shouldn't have to worry about the virtual env and whether it's activated etc. if you are in a project, it will automatically pick up the right Python version . However if you want to do something not possible through those commands you can still get down to the virtual env. Plus for auto complete in IDEs you generally need to select the venv
I use this vscode extension called "python envy" that changes venv automatically depending on which folder of a monorepo you are in, it's great! Wish vscode had it built-in
Never heard of it, will try. Thanks
You install uv not python, you use "uv run <file>" instead of "python <file>", you dont think about anything else.
There's only one kind of Python virtual environment, and both uv and the standard library `venv` module make them (as does third-party support like `virtualenv`). The differences are in what gets put in them (aside from the actual packages you explicitly install), and in the interface for configuring that. In particular, the standard library defaults to bootstrapping pip, but you can easily skip that. And of course uv does not bootstrap pip, because it does pip's job (and much more).
If you are working in Data Science/ML its the best bet for handling dependencies in your project compared to the rest of the tools. Its use has exploded, especially because you can do ‘uv pip whatever’ if you insist on using pip.
Said with a note of surprise?
Made me think this is probably normally a Ruby developer indoctrinated against Python. The article doesn’t seem to say what they have come from.
> Ruby developer indoctrinated against Python
Ruby devs think about code differently. Like Perl, they embrace TIMTOWTDI.
https://perl.fandom.com/wiki/TIMTOWTDI
Also, there's a pride in writing elegant code as opposed to following "Pythonic" conventions. Excellence is not conformity.
I use Python a lot more often now because it's seen as simpler and more approachable and easier to force compliance. I miss Ruby.
IMO "pythonic" is mostly about using the features the language has rather than trying to treat it as if it was Java or C++.
Having said that, in reviews you do get lazy comments like "not pythonic" or "not best practises" which often boil down to the reviewer just not liking something and being too much of an *** to say why. This is supposed to be a total shutdown that you cannot argue with and it's the kind of thing that might put you off the term "pythonic" for life.
"There should be one-- and preferably only one --obvious way to do it."
This is probably the core thing you might have issue with but I think its not really about conforming in how you write your own code but about the thing you make being easy for others to use.
I love this sentiment:
> There should be one-- and preferably only one --obvious way to do it.
But I often don't think the Pythonic way is a very good way to do it. And this leaves you a bit stuck!
I only Ruby shop I've ever worked at liberally used Rubocop to undo TIMTOWTDI. I suppose you could _write_ whatever you wanted but Rubocop would pretty aggressively rewrite it.
> Like Perl, they embrace TIMTOWTDI.
Yeah, and it's the wrong approach. Of course, you can have whatever preference you want, but in terms of engineering, it's plain wrong.
It's in the footnotes, they're a java/javascript/R dev.
There’s a footnote:
> If you know me, you know I used to be mostly a Java/JavaScript/R kind of guy. ↩
So, if you don't fancy Python you must be labeled a Ruby developer? Weird take.