If this "save a backup file" functionality was built into the OS, instead of having to be redone by every application, this probably would never have happened – it is much harder to do the wrong thing when you are just calling an OS API than when you have to implement it all yourself. And if the OS API is implemented wrong, you can just change the OS API, instead of having to change every application.
I like how Vim uses swap files to warn you if you are editing a file you already have opened (or have autosaved data from a previous crashed execution) – but it only works for Vim, if you open the file in another editor you don't get the warning. If that functionality was in the OS instead, it could easily work cross-application.
Unfortunately it's deprecated, I have used it for around an year for my home as a test, the idea is good, the implementation not much:
- the cleaner daemon is damn slow, so on write-intensive (as a modern WebVM profile dir, improperly named browser for legacy reasons profile dir) you'll easily end up in a filesystem full, and recovery is not that easy;
- navigating through the write history is manual, a manual mount of a snapshot that might disappear while you actually go through a bunch of them, so you need to mark them to stop garbage collection, mount them, diff/see the change of something you are interested in etc.
No shiny meld-like UI is there, there were some project in the past but so long abandoned that I've just found references to them, not even their code. Nowadays the sole Linux log-based fs is Samsung f2fs, witch is far less developed than nilfs2, to a point you can't even go through the write history. The sole good log-based fs I know is DragonFlyBSD Hammer, witch is "a better zfs" but being a DragonFly-only show with very little devs it's not really useful if you are outside the HPC world.
Personally these days I have zfs with auto snapshot service well backed by ZnapZend to sync data to a homeserver backup, it's not write()-triggered but time triggered so going through history is less useful, but as a raw protection against accidental disasters it's generally enough, zfs diff is not much detailed (just tell about changed files) but might be of help, snapshots are auto-mounted typically under the volume root in a .zfs/snapshots/$snapName dir so reasonabily easy to traverse with meld, a file manager etc to directly grab some files.
Unfortunately since the '80s no one except for SUN (zfs) have done much on generic data storage/file storage side...
You don't really need that much functionality, a copy-on-write filesystem with fine-grained access to historical data.
Once you have that, you can easily access the old version of a file and that is probably 90% of the functionality that you need without many changes to the software (regular automatic saves and ability to access old versions from the program itself).
While there are a number of copy-on-write filesystems around (ZFS, btrfs), most of them don't have fine-grained access to historical data, but only through snapshots. One exception is them HAMMER fs from DragonFlyBSD [1], though I don't if this is still true (HAMMER2 seems to be quite a different beast)
Sadly that’s not what copy on write means. When you have file a and you copy it to file b. COW means the copy b will only be truly materialized once you write something to it/edit it. Before that, file b will be aliased to file a.
If you have file A and a make a snapshot of it, let's call it A@2022-04-01-12:00, you have now two different files pointing at the same blocks.
If you write to one of the blocks of A you will not overwrite any data of A@2022-04-01-12:00 because the new data will be saved to a different block. I would call that copy on write.
I admit that the description is not perfect because it doesn't always apply (like in the example you make), but I couldn't come up with a more appropriate description and for the discussion at hand I thought it was quite apt.
Also, for what is worth, Wikipedia seems to agree with me [1]
The main feature of VMS which seems relevant here is file version numbers–it is a great idea, and it is sad more mainstream operating systems haven't copied it (if only Dave Cutler had taken that VMS idea to Windows NT... if only RMS' plan, announced in the GNU Manifesto, to include a file versioning filesystem in the GNU system had panned out...)–but I don't think it fully addresses the issue of editor autosave/etc:
(1) File version numbers are great to make a backup of the old file on saving, not so relevant to creating an autosave of a modified editor buffer / open document / etc. You generally want autosave to not create a new version of the file you are editing, because that would make your changes visible to other applications/users, and you haven't decided yet whether you want to "commit" your in-progress changes to the actual file.
(2) Doesn't really address the case of warning you that you (or somebody else) has the file open in another editor, or that you have recovered changes from a prior editing session. I know VMS has some file locking features that might help with the "warning" part, but other platforms do too. As far as I know, VMS still leaves it up to each app to build a sensible UI around those capabilities, rather than providing something built in to the OS.
We already have tools for this. Important work should be in version control and application authors who want more out of open have sqlite. People need open/read/write to work with the file directly and I don't want shell tools scattering backup files everywhere.
Leaving aside the correct advice to avoid such hard coded backup files, I wish people would recommend at least considering "sudo -e" instead of "sudo $EDITOR" when editing something important with root permissions.
sudo $EDITOR launches the editor itself as root, sudo -e launches the editor as a regular user on a temporary copy of the original file, and copies the contents back over to the original file when you are done. The less done as root, the less opportunity to mess things up.
Perhaps, but you will still overwrite it when you exit the editor and let sudo copy the contents back (that being said, there might be a race condition between closing the editor and sudo noticing that the process has terminated). Not sure if this would work, but a possible workaround could be to use /dev/shm (https://www.kernel.org/doc/gorman/html/understand/understand...).
The temporary file should be created in such a way that other users cannot modify it. If that does not happen, if other users can modify it, I would regard that as a bug.
Malicious processes running as the same user could potentially modify the file, but if you have malicious processes running as a user with sudo privileges you have probably already lost.
The vi editor is, or was just ed. All ed commands are vi commands.
Likewise Emacs: originally it was just a bunch of TECO macros plus the “^R mode” (visual) addition to TECO. I frequently used to casually type raw TECO into a mini buffer to perform certain edits (meta-altmode — my hand just made the gesture automatically when I thought about it).
I suspect Bill Joy knew about this when he added visual mode to ed about a decade later.
PS if you haven’t checked out the MIT TECO language you should — it will blow your mind.
In 1999, after installing Linux on a mainframe for the first time, I needed to edit a few files. Being in the Linux console, ISPF wasn't an option. But the copy of Linux we were working with (Marist College's release) didn't yet have vi available. So I had to go with ed. That, as they used to say, was "a real trip".
I was once stuck on a box that didn’t even have ed, and I needed to edit a file. This is maybe 1995 and it’s an HP/UX machine that was used as nothing more than an X console, but its fstab or automount config was messed up so it couldn’t boot anything but single user mode.
I managed to cobble together an “editor” with just the shell that as I recall read each line of the existing file and echoed it to stdout. I could press return to keep it as is or retype the entire line with whatever changes I needed. The output went to a new file. When I was done I copied the new file into place.
Another time I needed to transfer an executable to a server I could access only over serial console. On the receiving end, I think uudecode was all I had to safely get the binary over serial console.
Damn. That's the equivalent of having to walk to school uphill both ways during a snow storm. It makes me feel really grateful for the hundreds of thousands of man-hours that have been spent on giving us good and reliable tools.
Just another reason I don't mess with unix utils outside of the most trivial scripts one can imagine.
I really don't care if the software I use is "high quality" or not, anywhere near as much as how many edge cases they've covered.
If your app is small and has very few dependencies, as a heuristic I generally expect it to somehow be trouble one way or another.
The only time I've had major trouble with VS Code is when I installed a buggy extension that helps add licenses to things, without noticing that it modifies every file in the entire workspace, even if they have a license already, and sometimes gets the syntax wrong.
It wasn't a bit popular extension, so I probably should have known better.
> If your app is small and has very few dependencies, as a heuristic I generally expect it to somehow be trouble one way or another.
Wow, that’s extraordinary! I have precisely the opposite view, especially the “few dependencies.” The more dependencies the higher the probability of failure in my experience.
Oh you're probably totally right. The smaller apps are less likely in and of themselves to fail.
The problems I expect to see are not really the app itself failing, its the inability to deal with some other failure, or some changed requirements that make some ugliness, or maybe it doesn't have hardware acceleration and the performance is a little worse.
Maybe it doesn't detect malformed packets right. Maybe there's a UI issue that makes it easy to delete files, as in all the rm and dd problems. Maybe it only supports bitmap and wav files and now your app needs 5x the storage when you start adding media.
Very small things rarely outright fail. It's the rest of the whole system from users to hardware that does wrong by those very small things, and in their perfect bug free innocent perfection they're all "OK I guess the user knows what he's doing, one deleted home folder, coming right up!"
...and that never made any sense to me. Because you don't pronounce "sed" as "ess-ee-dee". So if you pronounce "sed" as "said", you must surely pronounce "ed" as "ed".
Found first half good but cannot say the same about second half. (Though a kid's opinion may differ.) Wonder how a Unix-y version of Aesop fables would look like.
The code fragment that the author complains about has an even more serious fault: it doesn’t check for the existence of an `ed.hup` file. If you happen to have one, because you’re unaware of `ed`’s behavior and you like to give weird names to your files, `ed` will wipe it out when trying to help you.
Vim does similar irresponsible things with its backup files¹ in its default configuration. It not only destroys existing files without checking, but prevents many auto-build systems from working correctly. All these behaviors are reckless, especially as there are obvious alternatives that don’t make assumptions about the users' files.
ed was a lifesaver if your Unix system didn't fully boot because unlike ex/vi, it was statically compiled and had no dependencies on other filesystems. Plus it was cool to learn just as a bit of history and in a very small way follow in the footsteps of some amazing people.
Same here. I'm listening to & reading a lot of computer history lately. From Knuth, Kernighan, Thompson, Ritchie, Hopper to more 'modern' pioneers like James Gosling, Chris Lattner, David Patterson, Rich Hickey and Rob Pike.
I'm just amazed by how much of the foundation of the software world was laid by these brilliant minds.
We need a series of books which would chart out all of the software, innovations and the people involved during that era. I would definitely buy and read it.
I live upgraded a remote server from a much older redhat release to gentoo. This was a system that didn't have a nice console interface, so no fake local terminal. I messed up the mount paths, so nothing that required shared libraries worked. I was able to fix it with ed!
The right solution would be to make a log file at startup, and append a copy of the original text and then every last bit of input, exactly as it is typed, interspersed with some metadata. Then, if anything happens, you just exit on the spot, and absolutely nothing is lost.
The type of stuff edited doesn't matter: could be audio, video, 3D design, vector drawing.
All editors should operate this way. It is disgraceful none do.
Sublime Text does. When you start it, all unsaved modifications and files from your last session will conveniently be there, just as you left them. Over years of usage, I've never had Sublime lose any data, even after sudden loss of power.
ed is the way that it is in part because many users in the late 70s had a really slow modem and a printing terminal, and memory was minimal, so just ? was seen as a good error message.
Early 70s, and it wasn't with a modem. I think the typical Teletype baud rates were 45.45 baud, 110 baud, and 300 baud. Having to wait half a second for your Teletype to type out a 15-character error message that tells you what you already know is the opposite of user-friendly. (Later, yes, people were using 300-baud modems, but that wasn't how Unix was originally developed.)
May be. However, my memory is of 2 factors that made line editors like ed necessary.
1. The keyboard mechanics and having paper as the output medium made typing very slow. That's probably why all the commands were short (ls, mv, rm, etc). Coding was not the stream of consciousness you might experience today. You would have to hold a load of stuff in your head (and ignore interruptions) as you slowly typed it out.
2. The paper only ever advances upwards. If you make a mistake there is no up arrow. Your easiest option may be to use the delete command and type the whole line again. Hopefully someone will give you one of those paper cheat sheet summarizing all the editor commands soon. I wonder how many potential computer geniuses dropped out of CS at this point.
Imagine you are at the command line and you take a few minutes to quickly type:
$ cp inupt_KJuusiwbHSgh.txt output_HswbXVfgawYuq.txt
cp: cannot stat 'inupt_KJuusiwbHSgh.txt': No such file or directory
Rats! You didn't notice that you mistyped "input". What now? You can't do copy and paste from paper! Well you could use the line editor:
When I started programming as part of my job in 1967, there were no screen editors. I used a teletype terminal (ASR33) with a paper tape to load and save my work.
Line editors like ed were the way to go. You’d keep the state of the program in your head and type out sections of it once in awhile to confirm your conception. Maybe print the whole thing once an hour.
Who doesn't reflexively save when pausing to think/etc.? To someone who does save early and often, this seems to be a non-issue --- if the original file wasn't writable for whatever reason, you'd also know pretty quickly.
If you want to do that in ed or ex, you have to exit insert mode, save, go back into insert mode, and then possibly exit insert mode again, find the extra line break you inserted to save, and delete that line break. So I could be wrong but I think probably no ed or ex users save by reflex when pausing to think. Also, saving a file could take a significant amount of time in the 01970s.
> This is a lesson on why not to use static backup filenames.
Yes, that would address this bug, and this bug only.
What if there's another bug? Or the file system is full? Or `ed` gets kill -9'd? Or the PC loses power?
Assuming it's possible to predict every single failure mode is foolish. It's better to make a system safe by default. In the case of `ed`, simply periodically store the text buffer in a file and restore from it on startup.
I am glad vim has the option to place the swap files where you want them, and make them unique (by setting 'directory' to end with "//").
For vim and ed and every other console program, I would run them inside screen(1) to avoid losing work due to network troubles, and network disconnects is usually where I get my SIGHUP.
this seems to have some advantages: for example, i can do a find / -name '*.swp' to figure out what i was working on when most of the files are readme.md
(i should note, real ed doesn't create ed.hup files; that's a weird gnuism)
> For vim and ed and every other console program, I would run them inside screen(1) to avoid losing work due to network troubles, and network disconnects is usually where I get my SIGHUP.
i don't like screen very much, even though i do continue to use it (owing more to its ubiquity and utility); it's full of all kinds of questionable ui decisions, and so many features i use so infrequently i need to constantly be re-reading the manual. i think if you have learned it and liked it okay, but all i ever want is terminal mobility, and i get a much nicer experience out of two tools:
using mosh means i don't get SIGHUP, and if i lose my local display, i can open another mosh, reptr my old session (it's still there!) and keep on hacking.
I like how Vim uses swap files to warn you if you are editing a file you already have opened (or have autosaved data from a previous crashed execution) – but it only works for Vim, if you open the file in another editor you don't get the warning. If that functionality was in the OS instead, it could easily work cross-application.