The version selection part sounds weird. The versions in nixpkgs make sense when you're running/building the system. If you're providing runtimes/compilers as a platform, you really want something like what devenv does - provide the versions yourself. You really don't want to end up building an old system to provide an old nodejs - you're leaving security patches in dependencies behind. Devenv does it for example through https://github.com/cachix/nixpkgs-python "All Python versions, kept up-to-date on hourly basis using Nix."
> Railway injects a deployment ID environment variable into all builds.
They could've done it in the next layer after installation. Also, you can split packages into different layers. There's even automation for it if you need batches to keep the number of layers down.
kesor 2 hours ago [-]
Looks like they are trying to force versions into where there are none. Just like trying to force a square cube into a round hole.
"Default versions" breaking things that depend on them? What is that? It is like using docker's ":latest" tag and being surprised each time that a new server falls on its face because the "default" image is actually a different version from the previous "default" image.
I don't understand any of the explanations in this blog post. Seems like people who have zero clue about what a "version" of a software is.
"no way of splitting up the Nix dependencies into separate layers" - Why? Of course you can split /nix/store into as many layers as you need. Do they even know how to use containers and how to use Nix in the first place?
With the clear incompetence of these people, no wonder that their proposed solution smells like a decomposed fish.
Classic NIH syndrome. There is going to be no surprise to see them meet the exact same problems they didn't solve with Nix to infest their new "solution".
jbverschoor 1 hours ago [-]
What works better to raise VC (which for many is the goal)
A nix wrapper or a deployment platform
mrbluecoat 2 hours ago [-]
> we don’t have any problem with Nix itself. But there is a problem with how we were using it.
A good example of 'use the right tool for the right job'. Nix is great for some use cases and awful for others. The problem is the Nix learning curve is so high that by the time you've grasped it enough to make a decision you feel you've invested too much time to back out now and pivot to something else so you try to shoehorn it to solve the original need.
teekert 2 hours ago [-]
I feel it like that as well, but in some ways Nix is a more normal programming paradigm than other OS’s. We’re just not used to thinking about an OS that way. Nix expressions have inputs (a package repo, lots of key-value pairs) and outputs (a Linux system). Idk perhaps in a couple of years it will be much more normal.
Ie it is very easy for an AI to create a to-spec shell.nix (some Python packages, some Linux packages, some env vars, some path entries etc), or configuration.nix because of this paradigm.
I do this a lot to include envs with repos that fully support the package. It would probably be more reproducible with flakes (a flake.nix is like a shell.nix but with version pinning… or something, I’m still climbing that learning hill).
ris 2 hours ago [-]
The main problem here is wanting to hang on to the "bespoke version soup" attitude that language package managers encourage (and is totally unsustainable). The alternative Mise doesn't appear to have any ability to understand version constraints between packages and certainly doesn't run tests for each installed package to ensure it works correctly with the surrounding versions. So you're not getting remotely the same thing.
k__ 1 hours ago [-]
"the "bespoke version soup" attitude that language package managers encourage"
Care to elaborate what that means and what the alternative is?
setheron 1 hours ago [-]
Nix gives you a commit guarantee rather than arbitrary versions. You're going to put yourself in a bad time when you have edge cases: glibc changes or conflicting shared libraries.
It sounds like it's a little bit too late, but I'm happy to provide some consulting on how you can get it to work idiomatically with Nix.
Product looks cool!
nialv7 57 minutes ago [-]
nix solves the shared library incompatibility problem by being extremely conservative. every time anything changes, consequential or not - a comment got modified, documentation changes, a testcase got added, etc. - it will rebuild all dependents. and not just that, but all dependents of dependents, and dependents of dependents of dependents, on and on. this often results in massive massive rebuilds.
sure you are not going to get shared library conflicts, but i think this solution is extremely wasteful, and can make development painful too - look at nixpkgs' staging process.
Cloudef 2 hours ago [-]
I dont see why they couldnt made their own derivations instead of relying on nixpkgs hashes.
droelf 1 hours ago [-]
We've been working on Pixi, which uses Conda packages. You can control versions precisely and also easily build your own software into a package to ship it. Would be curious to chat if it could be useful as an alternative to `mise`.
wg0 33 minutes ago [-]
> We also changed the codebase from Rust to Go because of the Buildkit libraries.
Go is the best choice at the moment for such tools. These tools start a process, do lots of IO and exit.
Very pragmatic choice.
Onavo 18 minutes ago [-]
Go also statically links all dependencies and reinvents all the wheels usually provided by the system land. Cross compilation is trivial. It is unrivaled when it comes to deployment simplicity.
xvilka 14 minutes ago [-]
Rust does the same.
femiagbabiaka 44 minutes ago [-]
The new design is very similar to Dagger, interesting.
lloydatkinson 3 hours ago [-]
As someone with only a little experience with Nix, the points here don’t really seem right?
> This approach isn’t clear or maintainable, especially for contributors unfamiliar with Nix’s version management.
> For languages like Node and Python, we ended up only supporting their latest major version.
What is not maintainable about this? That they need to make a list of available versions? So, can this not be automated?
Furthermore, why is Railway defining how a user uses Nix?
Surely one of the points of Nix is that you can take a bare machine and have it configured with exactly what versions of packages you want? Why would Railway need to get in the way of the user and limit their versions anyway?
Or did I misunderstand and they don’t even expose Nix to the user? If so, the original question still stands: can’t they automate that list of package versions?
elbear 2 hours ago [-]
The version limits come from the fact that the Nix cache doesn't maintain older versions. So, if you use an older version, you will have to compile from sources. It sounds like they didn't want to take it upon themselves and provide a cache with older versions, even though it doesn't sound like much effort.
Honestly, the reasons given don't feel very solid. Maybe the person who introduced Nix left and the ones remaining didn't like it very much (the language itself is not very nice, the docs weren't great either in the past).
Still, I'm not familiar enough with the stack they chose, but does it provide a level of determinism close to Nix? If not, it might come to bite them or make their life harder later on.
e3bc54b2 1 hours ago [-]
Nix cache (cache.nixos.org) does in fact maintain older versions[0]. In fact, they maintain so much older stuff (binaries and associated source), that they are used in both research[1][2] and are having issues with gigantic cache size[3][4].
And yes, their reasoning implies NIH and just unfamiliarity combined with unwillingness to really understand Nix.
> The way Nixpacks uses Nix to pull in dependencies often results in massive image sizes with a single /nix/store layer ... all Nix and related packages and libraries needed for both the build and runtime are here.
This statement is kinda like “I’m giving up on automobiles because I can’t make them go forward”. This is one of the things Nix can do most reliably. It automates the detection of which runtime dependencies are actually referenced in the resulting binary, using string matching on /nix/store hashes. If they couldn’t make it do that, they’re doing something pretty weird or gravely wrong. I wouldn’t even know where to start to try to stop Nix from solving this automatically!
I wouldn’t read too much into their experience with it. The stuff about versioning is a very normal problem everyone has, would have been more interesting if they attempted to solve it.
codethief 2 hours ago [-]
> Or did I misunderstand and they don’t even expose Nix to the user?
That's at least my understanding, yes.
notnmeyer 2 hours ago [-]
you can not have a dockerfile in your project at all, push your code to them, and they’d build an image for it with nixpacks. you’d see nix stuff in your build logs, but it’s behind the scenes for the most part.
Rendered at 15:33:29 GMT+0000 (Coordinated Universal Time) with Vercel.
> Railway injects a deployment ID environment variable into all builds.
They could've done it in the next layer after installation. Also, you can split packages into different layers. There's even automation for it if you need batches to keep the number of layers down.
"Default versions" breaking things that depend on them? What is that? It is like using docker's ":latest" tag and being surprised each time that a new server falls on its face because the "default" image is actually a different version from the previous "default" image.
I don't understand any of the explanations in this blog post. Seems like people who have zero clue about what a "version" of a software is.
"no way of splitting up the Nix dependencies into separate layers" - Why? Of course you can split /nix/store into as many layers as you need. Do they even know how to use containers and how to use Nix in the first place?
With the clear incompetence of these people, no wonder that their proposed solution smells like a decomposed fish.
Classic NIH syndrome. There is going to be no surprise to see them meet the exact same problems they didn't solve with Nix to infest their new "solution".
A nix wrapper or a deployment platform
A good example of 'use the right tool for the right job'. Nix is great for some use cases and awful for others. The problem is the Nix learning curve is so high that by the time you've grasped it enough to make a decision you feel you've invested too much time to back out now and pivot to something else so you try to shoehorn it to solve the original need.
Ie it is very easy for an AI to create a to-spec shell.nix (some Python packages, some Linux packages, some env vars, some path entries etc), or configuration.nix because of this paradigm.
I do this a lot to include envs with repos that fully support the package. It would probably be more reproducible with flakes (a flake.nix is like a shell.nix but with version pinning… or something, I’m still climbing that learning hill).
Care to elaborate what that means and what the alternative is?
It sounds like it's a little bit too late, but I'm happy to provide some consulting on how you can get it to work idiomatically with Nix.
Product looks cool!
sure you are not going to get shared library conflicts, but i think this solution is extremely wasteful, and can make development painful too - look at nixpkgs' staging process.
Go is the best choice at the moment for such tools. These tools start a process, do lots of IO and exit.
Very pragmatic choice.
> This approach isn’t clear or maintainable, especially for contributors unfamiliar with Nix’s version management.
> For languages like Node and Python, we ended up only supporting their latest major version.
What is not maintainable about this? That they need to make a list of available versions? So, can this not be automated?
Furthermore, why is Railway defining how a user uses Nix?
Surely one of the points of Nix is that you can take a bare machine and have it configured with exactly what versions of packages you want? Why would Railway need to get in the way of the user and limit their versions anyway?
Or did I misunderstand and they don’t even expose Nix to the user? If so, the original question still stands: can’t they automate that list of package versions?
Honestly, the reasons given don't feel very solid. Maybe the person who introduced Nix left and the ones remaining didn't like it very much (the language itself is not very nice, the docs weren't great either in the past).
Still, I'm not familiar enough with the stack they chose, but does it provide a level of determinism close to Nix? If not, it might come to bite them or make their life harder later on.
And yes, their reasoning implies NIH and just unfamiliarity combined with unwillingness to really understand Nix.
[0]: https://discourse.nixos.org/t/how-long-is-binary-cache-kept-...
[1]: https://hal.science/hal-04913007
[2]: https://luj.fr/blog/is-nixos-truly-reproducible.html
[3]: https://discourse.nixos.org/t/nixos-foundations-financial-su...
[4]: https://discourse.nixos.org/t/the-nixos-foundations-call-to-...
> The way Nixpacks uses Nix to pull in dependencies often results in massive image sizes with a single /nix/store layer ... all Nix and related packages and libraries needed for both the build and runtime are here.
This statement is kinda like “I’m giving up on automobiles because I can’t make them go forward”. This is one of the things Nix can do most reliably. It automates the detection of which runtime dependencies are actually referenced in the resulting binary, using string matching on /nix/store hashes. If they couldn’t make it do that, they’re doing something pretty weird or gravely wrong. I wouldn’t even know where to start to try to stop Nix from solving this automatically!
I wouldn’t read too much into their experience with it. The stuff about versioning is a very normal problem everyone has, would have been more interesting if they attempted to solve it.
That's at least my understanding, yes.