I'm rather surprised that it doesn't take the z-buffer as an input. I would have thought that would have provided useful information, it's one of the more useful forms of contolnet.
I think this is mainly so it can use the existing hooks for DLSS upscaling without requiring changes to the renderer, AMD is working on a comparable method which uses adapter networks to slot normals and material properties from the renderer into the diffusion model: https://gpuopen.com/learn/temporally-stable-generative-illum...
The official one seems to do, as well as other info from the engine (I think remember their mentioning LOD/UV map hints in one of the public demos, or articles, a few months back--or it might have been an Unreal Engine podcast)
The Nvidia video presentation on DLSS 5 says that the model was only trained with various G-buffers as input (including the depth buffer) but during inference, the model only uses the rendered frame. As well as the previous rendered frame reprojected via motion vectors, if I understand correctly, likely to improve temporal stability.
> The inference interface uses the engine-rendered RGB image as a dense, registered observation of visible scene appearance. It provides dense, pixel-aligned evidence for object support, occlusion boundaries, composition, and local material properties; engine motion vectors separately provide temporal correspondence.
> Existing image generative models commonly rely on text embeddings, exemplar images, or spatial control fields such as depth, edges, segmentation, and pose [...] These conditions are effective for general-purpose generation and editing, but they do not uniquely determine the object identities, materials, visibility relationships, lighting decisions, and pixel-aligned detail contained in an engine-rendered frame. DLSS 5 is therefore conditioned on the rendered frame itself.
Obviously they tried it multiple ways have brought receipts, but nonetheless it seems surprising that it wouldn't be of benefit to bring as much of that kind of metadata to the model as possible. You'd think depth and segmentation in particular would basically just be a straight shortcut without which the model spends its own time and effort re-deriving that stuff.
I'd also be interested in how post-processing fits in with this. Like if you've got weather effects, film grain, tone mapping, etc, I would have thought the model would do better working on the image before those processes.
Hmm well the depth buffer only has accurate depth for opaque objects (and even that's not really true). Things like hair wouldn't be in there. Its not ground truth depth.
I think it has more to do with what kind of data they have access to at runtime - IIRC DLSS upscaling has only required the previous frames and motion vectors, so requiring depth buffers would mean it was no longer a "drop-in" replacement
> I'd also be interested in how post-processing fits in with this.
I think screenspace effects like film grain and tonemapping are excluded in the same way UI elements are rendered separately from the game.
Even relatively small RGB -> depth models are pretty good. Which kind of implies depth is well encoded in the RGB, and adding depth would not really reduce entropy, while costing bandwidth.
Well it's doing the same math as the original, apparently. Hard to do but makes enough sense.
With LLMs you can do whatever you want pretty much. I have upstream CUDA running llama.cpp under unmodified Nouveau on one of my boxes. Why? Well, why not?
I also have a modified Nouveau driver that, with the help of more and newer blobs, gets reclocking working for at least most of Pascal/GTX 10 series. I would love to try to upstream it but it desperately needs to be rewritten with that intent. Too much ugly garbage. Still, I wanted to know how possible it is. Possible, it turns out. Modern LLMs can blackbox analyze the real driver quite well, and debug the Falcons themselves. It's very interesting. People say coding is dead; I think it's probably not really true. However, it is certainly changing. I think someone less skilled than me could beat me to the punch with enough determination. That is interesting.
With LLMs getting good, I've noticed people are more akin to fixing and new adding features to open-source projects that will never see the light of day.
Given that you bring the weights from NVIDIA's DLSS. So basically, the repo contains reverse engineered machinery that produces the exact same output given the same model.
Yes, see the numbers below. In most games, it's basically unusable if you want to play on 60 FPS or above unless you have a 5090.
The current implementation is more of a tech demo than a practical way to play games (+ officially it's available in 1 game). It's _fast enough_ to make some impressive YouTube videos but you most likely won't want to play anything with it yet.
Nvidia has stated that they're still working on improving the performance. No doubt future hardware generations will also include further hardware optimisations.
The potential for this kind of technology is pretty awesome, especially given that people have also found ways to add this to emulators.
Modders have added the ability to use the upscaler after DLSS 5, personally I don't know how sound this method is but the quality is pretty good, and allows to play games with DLSS 5 at 4k 60FPS with something that is not a 5090.
I really hope they do, but the market for gaming cards is looking mighty bleak right about now. 5090 is up over 80% since November last I checked and my 5080 is up 50%.
NVIDIA removing all mentions of gaming in their financials doesn’t bode well either, and from a fiduciary standpoint it would be negligent to sacrifice any capacity for higher-margin AI chips to make gaming cards.
Again, I hope I’m wrong and we see new cards summer/autumn 2027, but I would not bet my savings on it.
> from a fiduciary standpoint it would be negligent to sacrifice any capacity for higher-margin AI chips to make gaming cards
Corporations do not have a fiduciary duty to seek maximal profits. This is a myth.
They are given wide latitude to decide what's in the best interests of the shareholders. Keeping a less-profitable offering alive just in case the current big offering doesn't pan out in the long term would easily be defensible in court.
It wouldn't even be a challenge. Courts are loathe to question the judgment of directors and executives. The reasoning is obvious: why in the world would a judge have better knowledge of how to run a company than the people whose jobs are to run the company?
You are correct, I should’ve used financial not fiduciary. They make more money selling AI hardware than gaming and it’s a zero-risk transfer because if AI collapses gamers will not “vote with their wallets” and not buy a new card from NVIDIA, and they know it.
And I don’t even disagree with them here, if you can make 10-100x more money doing less work, why wouldn’t you?
It does OK on most of the 5 series cards, but you can only do 4k on the 5090 basically. And there are lots and lots of knobs to tune and experiment with.
In some cases, it seems that lowering resolution and graphics actually produces better DLSS5 output (but it varies)
It's really good at making older gen games look remade/remastered.
Worth remembering it is running a single-step diffusion model working in pixel space to generate each frame, it's a technical feat in itself that people are even using the words "frames per second"
RTX 5070, 9.6 ms with NR of Optiscaler in Stalker 2. It's payable and I enjoy new visual. Especially shadows and faces are incredibly detailed and precise. Landscapes not so much.
If they take Neural Rendering far enough, they can get rid of those useless raster and RT cores completely and ship compute and tensor cores only on all their chips
Rasters are far too good for simple textures + simple geometry. Even the best neural rendering works better when it has the bones of geometry, texture, motion and depth to work with. The baseline of spatial coherence you get from a conventional rendering engine is hard to beat with pure neural techniques.
Even in a "neural rendering optimistic" future, I can't see a way in which traditional rendering doesn't survive as a "control channel" that informs what the neural rendering does. To do otherwise would require making the bulk of the game logic neural too.
ppl dont understand sacrefices made and tradeoffs in making high fidelity game engine and them make such a comment. Deferred renderer is likely still most accurate because it actually does the thing faithfully. Forward+ is already stretching it yet much further having lot of cached results etc which make things less accurate. In my mind neural rendering is a step in that direction further. it will suffer accuracy problems only hidden by the fact people dont understand what it should or could look like.
ofc, getting performance is the real problem. I like neural rendering ideas in the sense DLSS can make certain things happen on older machines that were definitely out of scope for deferred pipeline on same machine.
Its like a Boost u really dont want to use if u dont need it.
It might become that CGI will he the place for very accurate rendering more than games, but i sure hope not. (they can easily afford accuracy as they dont need realtime).
games need to slow down a notch on getting better graphics and go for a stabilization and expectation management pass. so people dont expect things that are only feasible through neural rendering techniques. It forces a lot of players to use it. like most AI. fomo or something...
Neural-assisted rendering tech is awesome because it actually exploits the spatial/temporal redundancy of the image stream to save render compute. There are very few techniques that can do it, and none I know can match the neural quality.
Frame 2 is highly redundant if you already have Frame 1 rendered, and even more so if you can supply the motion vectors. If you already have Frame 3 too? It becomes extremely redundant.
And yet, a conventional rendering pipeline would spend as much computation on it as it would on Frame 1. There is no "just reuse the previous work for cheap" primitive in conventional rendering. Every frame has to be built step after step, with the full depth of the rendering pipeline.
The same is true for neural upscaling. Going from 1080p to 4K means pushing 4x the pixels - and for a conventional renderer, that's nearly 4x the work. But a neural upscaling pipeline can exploit the redundancy of image data - and "fill in" the missing detail from a "ground truth" 1080p render, in a way that flies below the radar of human perception. Using cheaper neural operators instead of the full pipeline for it.
Neural rendering is a welcome optimization to conventional rendering techniques, in my eyes. As long as it's implemented right. Which is hard, but not impossible - we've come a long way already.
Also, I think "hard generative AI" is more useful in CGI than in real time rendering. Because real time rendering with user input demands a degree of repeatability, but CGI only has to "look good once". So you can accept the lowered accuracy of a largely untethered generative process with very few keyframes, and pick the "better" (more accurate, if that's what you want) outputs out of it.
> it actually exploits the spatial/temporal redundancy of the image stream to save render compute
This is great until the redundancy no longer exists and you have to rebuild the entire state machine from zero (a humble scene transition or rapidly turning a corner).
>exploits the spatial/temporal redundancy of the image stream
Gbuffers have been using motion vectors forever. For realtime global illumination, techniques like ReSTIR already allow for temporal reuse and spatial coherence.
I just don't see the purpose of replacing the traditional rendering pipeline for neural rendering techniques. It would be one thing if we were constrained on the number of triangles we could push per second, but we're not. The bottleneck is usually elsewhere in the pipeline: animating characters/objects, particle systems, physics updates, etc.
This. This is what they are trying to do. If they can flip the entertainment industry to use neural rendering using their "proprietary" api's then they would have cornered a market from the bottom.
But I thought Nvidia “loves open source” (they don’t) and they are now a supporter for open source and open weight models by acquiring Huggingface? (They don’t actually care)
But the line is drawn when it involves CUDA and any part of their closed source compilers (nvcc).
There are obvious reasons why they are closed source, but it’s becoming pointless since Deepseek have open sourced their AI compiler and compute libraries with DeepGEMM and eventually they will catch up.
I assume this is meant to run with the weights people extracted from the latest NBA game, where it was first trialled.
> Isn't the mote that Nvidia has is they work with studios to generate the training data from the game, then they ship a model per game?
That was true for the very first version of DLSS, from DLSS 2 on the models have been universal - the per-game adjustments are done on the inference end by changing the effect intensity or masking out objects
They used to ship one model per game but now there is a single model, however they still do minor updates to it presumably to fine-tune it on new games
I don't really understand why the weights aren't included... US law says they can't have a copyright, no? Maybe they're concerned about other countries or cautious about a litigious Nvidia.
It used to be in dlss1. I think it's completely been put to pasture now though, it's way too much work and can't really cover some of the main things people actually want to use dlss5 on, for instance Morrowind.
I think it's interesting that Nvidia is so interested in producing the hardware that fuels the future of software development, given that their primary business advantage is their software moat. This is an interesting project for sure, but turning a bunch of GPUs at Zluda[0] (an open implementation of Cuda) could be far more destructive for them, right?
Am I the only one who feels a sense of disinterest in a project where the main README is LLM-generated? Does the author not have time to write what they did and how it's used?
I'm more upset about it being factually wrong, e.g. both mentions of "git-ignored" are absurd (why would you mention it if it's not in the repo?) and wrong (they are in the repo).
I notice this, that AI likes to write about things that are not in there. Like i review AI generated output, notice unnecessary things, and asks AI to remove that. So AI removes that and adds that "this and that, that was used or described like this, was removed because bla bla bla" to the document.
I think its somehow needs to talk (write) about the things that are in the context and removal is there so AI predicts that it should be there.
Yeah, I call it bugfix storytelling. Once upon a time this class far far away had this red hooded method...
Especially egregious if both adding and removing the thing happens in one commit. Git should be telling the story, and if it can't then there _is_ no story!
I spend so much time cleaning up AI comments in the codebases I work on, it's maddening. I could have an instruction to not allow it to write comments at all, but some of them are useful.
Exactly. The code, whatever, it's for machines so I don't really care if it's by machines as long as it works. But if you can't even be bothered to think about the human-facing parts of your thing like docs and UX, I'm not really interested. It just feels cheap (in the bad sense) and offputting.
I do feel like there's merit to having an open source implementation of anything, no matter who/what wrote it. I'm just hoping the results are validated well.
If the README is >90% AI generated and it is as long as a novel, I am not going to read it and will assume that the author did not read or write it either.
Unfortunately it is slop, beyond the comprehension of the author unless they are experienced with DLSS internals to explain it in depth.
"Am I the only one who feels a sense of disinterest in a project where the code is LLM-generated? Does the author not have time to code the project?"
This is how I feel about every single project announcement on HN recently, they are already bragging about models all over the place, why shouldn't they go full way down being replaced by the Borg?
Something will this would historically guarantee a Senior Staff+ position at Nvidia. Wondering why Jensen doesn't put money where his mouth is ("were seeking exceptional engineers blabla") and offer him a job?
> It takes one rendered frame (a low dynamic range proxy of it, three lanes of Gaussian noise, the previous frame's output reprojected, and five conditioning scalars) and produces four f32 channels per pixel: an RGB residual and one temporal-blend logit.
> The temporal path is implemented, but in the demo: the network's history input lanes and its per-pixel blend logit drive a reprojected feedback loop (docs/frame.md). The dlss5vk tool runs single frames with no history, which is what the reference captures were made with.
From this I assume the network uses the (via motion vectors) reprojected previous frame in order to increase temporal stability, i.e. similarity over adjacent frames. But this isn't strictly necessary, and apart from it, DLSS 5 is a pure post-process filter. So you could apply it to an old animated CGI movie like Final Fantasy (2001) [1]. Which should make it look significantly more realistic, at the cost of some flicker or other temporal instability.
One could also apply it to still images, like old renders from Tomb Raider [2], where temporal stability is not a factor. The difference to conventional text-to-image models with a "make it photorealistic" prompt would be that DLSS 5 strongly adheres to the underlying geometry.
The maanHimself repository appears to be 35 hours older than the aloshdenny one based on `created_at` from the GitHub API. maanHimself's `pushed_at` predates aloshdenny's `created_at` too.
That doesn't guarantee maanHimself is the original author, but it's looking likely.
I'm rather surprised that it doesn't take the z-buffer as an input. I would have thought that would have provided useful information, it's one of the more useful forms of contolnet.
I think this is mainly so it can use the existing hooks for DLSS upscaling without requiring changes to the renderer, AMD is working on a comparable method which uses adapter networks to slot normals and material properties from the renderer into the diffusion model: https://gpuopen.com/learn/temporally-stable-generative-illum...
The official one seems to do, as well as other info from the engine (I think remember their mentioning LOD/UV map hints in one of the public demos, or articles, a few months back--or it might have been an Unreal Engine podcast)
The Nvidia video presentation on DLSS 5 says that the model was only trained with various G-buffers as input (including the depth buffer) but during inference, the model only uses the rendered frame. As well as the previous rendered frame reprojected via motion vectors, if I understand correctly, likely to improve temporal stability.
The technical report suggests it does not use the depth buffer: https://research.nvidia.com/labs/adlr/files/DLSS5_Report.pdf
> The inference interface uses the engine-rendered RGB image as a dense, registered observation of visible scene appearance. It provides dense, pixel-aligned evidence for object support, occlusion boundaries, composition, and local material properties; engine motion vectors separately provide temporal correspondence.
> Existing image generative models commonly rely on text embeddings, exemplar images, or spatial control fields such as depth, edges, segmentation, and pose [...] These conditions are effective for general-purpose generation and editing, but they do not uniquely determine the object identities, materials, visibility relationships, lighting decisions, and pixel-aligned detail contained in an engine-rendered frame. DLSS 5 is therefore conditioned on the rendered frame itself.
Obviously they tried it multiple ways have brought receipts, but nonetheless it seems surprising that it wouldn't be of benefit to bring as much of that kind of metadata to the model as possible. You'd think depth and segmentation in particular would basically just be a straight shortcut without which the model spends its own time and effort re-deriving that stuff.
I'd also be interested in how post-processing fits in with this. Like if you've got weather effects, film grain, tone mapping, etc, I would have thought the model would do better working on the image before those processes.
Integration cost is very important to DLSS.
More information is better, but more games is also better.
Hmm well the depth buffer only has accurate depth for opaque objects (and even that's not really true). Things like hair wouldn't be in there. Its not ground truth depth.
I think it has more to do with what kind of data they have access to at runtime - IIRC DLSS upscaling has only required the previous frames and motion vectors, so requiring depth buffers would mean it was no longer a "drop-in" replacement
> I'd also be interested in how post-processing fits in with this.
I think screenspace effects like film grain and tonemapping are excluded in the same way UI elements are rendered separately from the game.
That doesn't make sense; every game has a depth buffer, but not every game has motion vectors.
Because the depth buffer isn't always accurate or accessible.
WoW goes to lengths to hide its depth buffer.
Even relatively small RGB -> depth models are pretty good. Which kind of implies depth is well encoded in the RGB, and adding depth would not really reduce entropy, while costing bandwidth.
> bit-exact against the original
What kind of sorcery is this ? Very impressive work !
Well it's doing the same math as the original, apparently. Hard to do but makes enough sense.
With LLMs you can do whatever you want pretty much. I have upstream CUDA running llama.cpp under unmodified Nouveau on one of my boxes. Why? Well, why not?
I also have a modified Nouveau driver that, with the help of more and newer blobs, gets reclocking working for at least most of Pascal/GTX 10 series. I would love to try to upstream it but it desperately needs to be rewritten with that intent. Too much ugly garbage. Still, I wanted to know how possible it is. Possible, it turns out. Modern LLMs can blackbox analyze the real driver quite well, and debug the Falcons themselves. It's very interesting. People say coding is dead; I think it's probably not really true. However, it is certainly changing. I think someone less skilled than me could beat me to the punch with enough determination. That is interesting.
Please please please publish those things somewhere, so that no one has to reinvent the wheel even with an llm
With LLMs getting good, I've noticed people are more akin to fixing and new adding features to open-source projects that will never see the light of day.
Given that you bring the weights from NVIDIA's DLSS. So basically, the repo contains reverse engineered machinery that produces the exact same output given the same model.
Claim != Reality most of the time
The load-bearing kind.
LLMs are really good at deobfuscating or even decompiling code.
If done by a human, yes.
Nowadays it takes one well written prompt to a frontier LLM to produce something like this.
That would be still impressive, even though more distributed among the AI builders and all those nameless code contributers etc.
So… that kind of sorcery, I guess.
> A Vulkan reimplementation of NVIDIA's DLSS 5 Neural Rendering network, bit-exact against the original.
Bit-identical, I swear I heard that somewhere before.
Almost 8ms on 1080p resolution seems extremely expensive, does the original also eat into the rendering budget as much?
Yes, see the numbers below. In most games, it's basically unusable if you want to play on 60 FPS or above unless you have a 5090.
The current implementation is more of a tech demo than a practical way to play games (+ officially it's available in 1 game). It's _fast enough_ to make some impressive YouTube videos but you most likely won't want to play anything with it yet.
Nvidia has stated that they're still working on improving the performance. No doubt future hardware generations will also include further hardware optimisations.
The potential for this kind of technology is pretty awesome, especially given that people have also found ways to add this to emulators.
I was able to run it in Skyrim on my 5060ti at about 60fps. First game that almost used all 16gb of vram
Modders have added the ability to use the upscaler after DLSS 5, personally I don't know how sound this method is but the quality is pretty good, and allows to play games with DLSS 5 at 4k 60FPS with something that is not a 5090.
I really hope they do, but the market for gaming cards is looking mighty bleak right about now. 5090 is up over 80% since November last I checked and my 5080 is up 50%. NVIDIA removing all mentions of gaming in their financials doesn’t bode well either, and from a fiduciary standpoint it would be negligent to sacrifice any capacity for higher-margin AI chips to make gaming cards.
Again, I hope I’m wrong and we see new cards summer/autumn 2027, but I would not bet my savings on it.
> from a fiduciary standpoint it would be negligent to sacrifice any capacity for higher-margin AI chips to make gaming cards
Corporations do not have a fiduciary duty to seek maximal profits. This is a myth.
They are given wide latitude to decide what's in the best interests of the shareholders. Keeping a less-profitable offering alive just in case the current big offering doesn't pan out in the long term would easily be defensible in court.
It wouldn't even be a challenge. Courts are loathe to question the judgment of directors and executives. The reasoning is obvious: why in the world would a judge have better knowledge of how to run a company than the people whose jobs are to run the company?
You are correct, I should’ve used financial not fiduciary. They make more money selling AI hardware than gaming and it’s a zero-risk transfer because if AI collapses gamers will not “vote with their wallets” and not buy a new card from NVIDIA, and they know it. And I don’t even disagree with them here, if you can make 10-100x more money doing less work, why wouldn’t you?
It does OK on most of the 5 series cards, but you can only do 4k on the 5090 basically. And there are lots and lots of knobs to tune and experiment with.
In some cases, it seems that lowering resolution and graphics actually produces better DLSS5 output (but it varies)
It's really good at making older gen games look remade/remastered.
Worth remembering it is running a single-step diffusion model working in pixel space to generate each frame, it's a technical feat in itself that people are even using the words "frames per second"
Yes the original is very expensive. It depends on the resolution and the GPU used:
RTX 5060: 9.9 ms at 1080p
RTX 5070: 10.2 ms at 1440p
RTX 5080: 13.7 ms at 2160p
RTX 5090: 8.2 ms at 2160p
Source: https://www.youtube.com/watch?v=3EfLjmdG29Q&t=600
RTX 5070, 9.6 ms with NR of Optiscaler in Stalker 2. It's payable and I enjoy new visual. Especially shadows and faces are incredibly detailed and precise. Landscapes not so much.
The original does tend to reduce the FPS by half or more
probably, going by the reported massive performance hits
Jensen : Nobody needs to code anymore...
Programmer: OpenDLSS...
Jensen : Wait. Not like that! (╯°□°）╯︵┻━┻
You think Jensen even thinks about the gaming market anymore?
If they take Neural Rendering far enough, they can get rid of those useless raster and RT cores completely and ship compute and tensor cores only on all their chips
Rasters are far too good for simple textures + simple geometry. Even the best neural rendering works better when it has the bones of geometry, texture, motion and depth to work with. The baseline of spatial coherence you get from a conventional rendering engine is hard to beat with pure neural techniques.
Even in a "neural rendering optimistic" future, I can't see a way in which traditional rendering doesn't survive as a "control channel" that informs what the neural rendering does. To do otherwise would require making the bulk of the game logic neural too.
ppl dont understand sacrefices made and tradeoffs in making high fidelity game engine and them make such a comment. Deferred renderer is likely still most accurate because it actually does the thing faithfully. Forward+ is already stretching it yet much further having lot of cached results etc which make things less accurate. In my mind neural rendering is a step in that direction further. it will suffer accuracy problems only hidden by the fact people dont understand what it should or could look like.
ofc, getting performance is the real problem. I like neural rendering ideas in the sense DLSS can make certain things happen on older machines that were definitely out of scope for deferred pipeline on same machine.
Its like a Boost u really dont want to use if u dont need it.
It might become that CGI will he the place for very accurate rendering more than games, but i sure hope not. (they can easily afford accuracy as they dont need realtime).
games need to slow down a notch on getting better graphics and go for a stabilization and expectation management pass. so people dont expect things that are only feasible through neural rendering techniques. It forces a lot of players to use it. like most AI. fomo or something...
I don't think I fully agree?
Neural-assisted rendering tech is awesome because it actually exploits the spatial/temporal redundancy of the image stream to save render compute. There are very few techniques that can do it, and none I know can match the neural quality.
Frame 2 is highly redundant if you already have Frame 1 rendered, and even more so if you can supply the motion vectors. If you already have Frame 3 too? It becomes extremely redundant.
And yet, a conventional rendering pipeline would spend as much computation on it as it would on Frame 1. There is no "just reuse the previous work for cheap" primitive in conventional rendering. Every frame has to be built step after step, with the full depth of the rendering pipeline.
The same is true for neural upscaling. Going from 1080p to 4K means pushing 4x the pixels - and for a conventional renderer, that's nearly 4x the work. But a neural upscaling pipeline can exploit the redundancy of image data - and "fill in" the missing detail from a "ground truth" 1080p render, in a way that flies below the radar of human perception. Using cheaper neural operators instead of the full pipeline for it.
Neural rendering is a welcome optimization to conventional rendering techniques, in my eyes. As long as it's implemented right. Which is hard, but not impossible - we've come a long way already.
Also, I think "hard generative AI" is more useful in CGI than in real time rendering. Because real time rendering with user input demands a degree of repeatability, but CGI only has to "look good once". So you can accept the lowered accuracy of a largely untethered generative process with very few keyframes, and pick the "better" (more accurate, if that's what you want) outputs out of it.
> it actually exploits the spatial/temporal redundancy of the image stream to save render compute
This is great until the redundancy no longer exists and you have to rebuild the entire state machine from zero (a humble scene transition or rapidly turning a corner).
>exploits the spatial/temporal redundancy of the image stream
Gbuffers have been using motion vectors forever. For realtime global illumination, techniques like ReSTIR already allow for temporal reuse and spatial coherence.
I just don't see the purpose of replacing the traditional rendering pipeline for neural rendering techniques. It would be one thing if we were constrained on the number of triangles we could push per second, but we're not. The bottleneck is usually elsewhere in the pipeline: animating characters/objects, particle systems, physics updates, etc.
This. This is what they are trying to do. If they can flip the entertainment industry to use neural rendering using their "proprietary" api's then they would have cornered a market from the bottom.
Hope it will get a Linux port soon
But I thought Nvidia “loves open source” (they don’t) and they are now a supporter for open source and open weight models by acquiring Huggingface? (They don’t actually care)
But the line is drawn when it involves CUDA and any part of their closed source compilers (nvcc).
There are obvious reasons why they are closed source, but it’s becoming pointless since Deepseek have open sourced their AI compiler and compute libraries with DeepGEMM and eventually they will catch up.
They only care because open-weight helps to drive the GPU business notably thanks to inference providers (Baseten, Together, Mistral, ...).
At least their support helps the open-weight ecosystem.
Depends on which open source you are talking about, like every single company contributing to FOSS.
They care when the agendas align, and they don't when they won't.
They certainly do _care_. Embrace, extend, extinguish.
They don't. I got threatened with a lawsuit after I suggested (submitted patches) they fix some of their buggy kernel code.
At what point does this neural rendering take away the human touch on the art styles?
that's essentially the whole thing
How useful is this without weights?
Isn't the mote that Nvidia has is they work with studios to generate the training data from the game, then they ship a model per game?
Or is my knowledge outdated here and they're just using a single generalised model?
I assume this is meant to run with the weights people extracted from the latest NBA game, where it was first trialled.
> Isn't the mote that Nvidia has is they work with studios to generate the training data from the game, then they ship a model per game?
That was true for the very first version of DLSS, from DLSS 2 on the models have been universal - the per-game adjustments are done on the inference end by changing the effect intensity or masking out objects
They have a technical report on the neural rendering part of DLSS 5 which goes into it: https://research.nvidia.com/labs/adlr/files/DLSS5_Report.pdf
They used to ship one model per game but now there is a single model, however they still do minor updates to it presumably to fine-tune it on new games
I don't really understand why the weights aren't included... US law says they can't have a copyright, no? Maybe they're concerned about other countries or cautious about a litigious Nvidia.
The US has a copyright law.
> they ship a model per game?
there's no way that's true!?
It used to be in dlss1. I think it's completely been put to pasture now though, it's way too much work and can't really cover some of the main things people actually want to use dlss5 on, for instance Morrowind.
its not, from what I've heard the difference in quality was not worth it
>they ship a model per game?
They don't. Only DLSS 1 was trained specifically per each game.
I think it's interesting that Nvidia is so interested in producing the hardware that fuels the future of software development, given that their primary business advantage is their software moat. This is an interesting project for sure, but turning a bunch of GPUs at Zluda[0] (an open implementation of Cuda) could be far more destructive for them, right?
[0] https://github.com/vosen/ZLUDA
Am I the only one who feels a sense of disinterest in a project where the main README is LLM-generated? Does the author not have time to write what they did and how it's used?
I'm more upset about it being factually wrong, e.g. both mentions of "git-ignored" are absurd (why would you mention it if it's not in the repo?) and wrong (they are in the repo).
I notice this, that AI likes to write about things that are not in there. Like i review AI generated output, notice unnecessary things, and asks AI to remove that. So AI removes that and adds that "this and that, that was used or described like this, was removed because bla bla bla" to the document.
I think its somehow needs to talk (write) about the things that are in the context and removal is there so AI predicts that it should be there.
Yeah, I call it bugfix storytelling. Once upon a time this class far far away had this red hooded method...
Especially egregious if both adding and removing the thing happens in one commit. Git should be telling the story, and if it can't then there _is_ no story!
I spend so much time cleaning up AI comments in the codebases I work on, it's maddening. I could have an instruction to not allow it to write comments at all, but some of them are useful.
Yes it's constant. And in any article it's full of "The X does Y. The Z does A. But the B stays silent."
AI writing is just bad, this things is really noticable - but even in READMEs they look superficially OK until you read them.
AI probably should not be writing docs, commit logs or comments.
If you think about it, actually the author did write what they did (nothing), and also how it’s used (it isn’t).
Seems like the owner of the github repo claims copyright, though. Since they provide a license.
Exactly. The code, whatever, it's for machines so I don't really care if it's by machines as long as it works. But if you can't even be bothered to think about the human-facing parts of your thing like docs and UX, I'm not really interested. It just feels cheap (in the bad sense) and offputting.
Agree.
I do feel like there's merit to having an open source implementation of anything, no matter who/what wrote it. I'm just hoping the results are validated well.
> what they did
bold of you to assume the code wasn't llm generated as well.
you think a human wrote the code? in a month? Do you think that's air you're breathing? might be time to challenge preconceptions
I'm fine with it
If the README is >90% AI generated and it is as long as a novel, I am not going to read it and will assume that the author did not read or write it either.
Unfortunately it is slop, beyond the comprehension of the author unless they are experienced with DLSS internals to explain it in depth.
"Am I the only one who feels a sense of disinterest in a project where the code is LLM-generated? Does the author not have time to code the project?"
This is how I feel about every single project announcement on HN recently, they are already bragging about models all over the place, why shouldn't they go full way down being replaced by the Borg?
Slop is a new language and you will learn it read it
What kind of dataset is used to train DLSS 5? Do they need to generate synthetic image pairs first?
So that means AMD implementation is on the horizon?
Something will this would historically guarantee a Senior Staff+ position at Nvidia. Wondering why Jensen doesn't put money where his mouth is ("were seeking exceptional engineers blabla") and offer him a job?
Because this was most likely written entirely by an LLM?
> It takes one rendered frame (a low dynamic range proxy of it, three lanes of Gaussian noise, the previous frame's output reprojected, and five conditioning scalars) and produces four f32 channels per pixel: an RGB residual and one temporal-blend logit.
> The temporal path is implemented, but in the demo: the network's history input lanes and its per-pixel blend logit drive a reprojected feedback loop (docs/frame.md). The dlss5vk tool runs single frames with no history, which is what the reference captures were made with.
From this I assume the network uses the (via motion vectors) reprojected previous frame in order to increase temporal stability, i.e. similarity over adjacent frames. But this isn't strictly necessary, and apart from it, DLSS 5 is a pure post-process filter. So you could apply it to an old animated CGI movie like Final Fantasy (2001) [1]. Which should make it look significantly more realistic, at the cost of some flicker or other temporal instability.
One could also apply it to still images, like old renders from Tomb Raider [2], where temporal stability is not a factor. The difference to conventional text-to-image models with a "make it photorealistic" prompt would be that DLSS 5 strongly adheres to the underlying geometry.
1: https://www.imdb.com/title/tt0173840/
2: https://www.tombraiderchronicles.com/images/artwork-high-res...
Seems quite similar to this repository?
https://github.com/aloshdenny/open-dlss
The maanHimself repository appears to be 35 hours older than the aloshdenny one based on `created_at` from the GitHub API. maanHimself's `pushed_at` predates aloshdenny's `created_at` too.
That doesn't guarantee maanHimself is the original author, but it's looking likely.
Pretty sure the author was Claude
Same exact commits at the same time as well, but different repository names and authors. What the hell?
From the LICENSE
Copyright (c) 2026 maan
So... Either alooshdenny stole the commits, or it's an alias for maan.
It's not stolen. It's MIT licensed and the attribution is still there.
Rewriting the commit author is very much "stolen".
Morally, I completely agree, but legally that's not something you can prevent with a weak license like MIT.
It might be possible to pursue this as illegal under general copyright law but it would be difficult.
If you don't want something like this to happen to your work, use a stronger license.
unsavory, yes. stolen a bit strong.
bit-exact against the original )
just unsure who's original ))
It's Dragostea Din Tei all over again