Hacker Read top | best | new | newcomments | leaders | about | bookmarklet login

Storing things "in files" still writes a CHS-addressed sector on a disk, but it's less reliable and the user doesn't know where it is.

Files are currently used to implement apps, but that can be seen as a transitional measure, like an OS that supports both files and raw disk access. A fully app-based OS without files, though not existing currently, would be possible.

Another idea the industry discarded was to make the disk a big SQL database, again without files.



sort by: page size:

I think there would still be a need for "files", ie, a way to store a self-contained document format in memory that is independent from the code that is working on the data at one exact moment: to allow moving documents between computers, but also to allow updating the software in a safe way (ie without losing data), to save older version of a document, but most importantly IMHO to be able to create snapshots that can be safely reloaded in case of a system crash. We're not ready to have systems that can't be rebooted.

Current file systems are impressive - flexible, robust, close the hardware performance. But I'm disappointed that we are still using such low level models for our day to day computing. Files = everything is an array of bytes and every program/library has to interpret and manage those bytes, "manually", individually, and slightly differently to other programs!

It's understandable to use "files" when running retro apps, but it's way past time that a high level model rendered the concept of files obsolete.

(I can be hopeful but I hold not outlook for such better models. Too many backwards compatible apps and too much depends on our existing code.)


The concepts are orthogonal. Von Neumann machines can be used without file systems and vice versa.

Persistent state has its pros and cons - for me the ability to reset the machine's state is an important feature. For the author this doesn't seem to be the case.

As for abandoning the concept of a file system, that's something that has been worked on for decades. Microsoft even wanted to include this as the main feature for Windows Vista, back in 2003 (dubbed "WinFS"). They basically wanted to replace traditional files with a single relational database backed by schemas for describing objects.

This would've been a truly revolutionary concept, as it would've allowed for features like searching, sorting, grouping, and versioning of objects without any complex application-level code.

Another advantage of such system would be the ability to exchange data between apps by just passing an id. All required meta-information would be available through the schema and the app could choose a view that suits its needs: providing the id of an audio file could result in the title and artist for a word processing program, the audio data for a music player, or the associated cover art in case of an image editing program.

There are many options for replacing the concept of files and directories that are just as robust and provide extra features. The general architecture of the CPU is not affected by or related to this.


File systems are just nosql databases, hierarchical key-value blob stores. There are obviously ton of other ways to model databases that could be used. For the other extreme end I think Oracle DB runs quite happily on raw disks, or at least did so at some point.

Of course I'm not sure if parent was meaning files as a way to structure/store data (having that hierarchical blobstore) or as a way to access data (something you `open`, `read`, `seek` etc), as they are slightly different things.

For a more real world example, take a look how mainframes, especially AS400 (edit: meant System/360 successors), managed data. At least afaik they fundamentally work on a more structured level.


File systems are not a replacement for a database.

File systems can be used similar to a database with user locking as well. I'm not suggesting it doesn't come with new challenges but the apps I've used that write and read from files are pretty phenomenal and much easier to scale.

Files are not databases.

Ignores the obvious benefit of the classical filesystem, which is that all data inherits an enormous amount of generic functionality from it: organization, interop, sharing, backup, versioning, etc.

File systems don't need to die, they need to grow up and provide the things that apps always have to implement over top of them:

- user-centric interface that shows data in a way that makes sense to them: documents, apps, messages, contacts, etc.

- non-hiearchal structure, views based on search, filter, sort

- database-like performance and semantics e.g. indexes, transactions

- structured data that is trivial to load/save from any programming language

I believe that all this can be had while preserving the benefits of flat files, but it's an enormously ambitious undertaking and the only sort-of-attempt that I'm aware of (WinFS) never even shipped, let alone caught on.


The file storage approach is unfortunately super common and makes a bunch of really basic things difficult and slow. Need to phase it out more heavily, especially from new implementations.

> I could see such an interface lacking files entirely

Now that's a great idea. If you think about it, current file systems are OLD. No transactions (I'm aware of the exceptions, but even those filesystems capable of transactions don't always expose that to the user), we rely on the file name to determine its type, no-or-severely-restricted access to metadata, possible infinite link loops.... I'm sure that the filesystems are implemented excellently, but I really hope we rethink the way we store data. I would much prefer a "flat" filesystem jailed by app and organizable by labels. Or whatever.


We haven't gotten rid of files, we've just obfuscated their existence. Akin to putting a splash screen, or allegorically sticking our heads in the sand.

If we had actually devised an alternative to hierarchical file systems and moved to something like a persistent object store (as with Newton OS, PalmOS and some other platforms), that would have been impressive, though in the long term might not turn out well due to relative novelty and lack of battle testing to the idea.

In fact, trying to pretend files don't exist in an environment dominated by them is foolish.


Files are central to almost any OS with storage of some sort.

That's exactly right. Files don't exist. The "filesystem" just a big fat KV store.

Somewhere along the way someone decided it'd be a useful abstraction to imagine a hierarchical organization system (directories) on top, so that was glommed on, but it's not real either.

It was never a perfect system, but it worked well enough when most users were fairly technical and had a humanly comprehensible set of labelled data streams.

I appreciate the effort to insulate users from the complexity that has grown up underneath them. The simple fact is that most people fail at large scale taxonomy and organization. It's hard. And it's a lot of work to maintain even if you're good at it. See: library science. So I don't think there is another model that will succeed as well as "files" have.

iOS hides the filesystem, but it's still there obviously. So far all we've seen is insulation for those who need it, as a byproduct of huge control loss for everyone. The other (valuable) byproduct is security.

We haven't found the compromise yet. There might not be one.


but.. file systems

In some mainframes and micros, the filesystem is based on database model, there are no files.

I'm not saying the physical file as fundamental computing construct is going anywhere.

I'm saying software is getting to the point where the details of file storage won't be relevant, even to producers of content. Whether your resource is one file, five files or five parts of five larger files will be as relevant 'tomorrow' as where the disk controller has put physical bits on a platter or memory cell 'today'.

The details still matters and always will; there are certainly good and bad ways to store those bits depending on the varying contexts. But no-one's thinking about it when they save a file; no-one's hand-tweaking each decision for individual performance or organizing it so you'll remember physically where you put it.


Files as an interchange format, sure. But as a primary storage system for application structured data they leave a lot to be desired:

- Portability of metadata is lacking (and can be sneakily removed when you least expect it), as other commenters have pointed out.

- Filtering sets of files from out of a (possibly deep) directory hierarchy based on different criteria requires writing a lot of subtly different loops to check metadata. Querying e.g. SQLite handles that part for you once you express what you want, without as much risk of messing up one of those loops.

- Similarly, a schemaful database can prevent your writing incorrectly-shaped (meta)data up front, where filesystems are flexible enough that bad writes may not be noticed until your program tries to read that data back out.

- The accessibility of file-based internal storage systems to human users can sometimes be too high, a la the joke about someone "organizing and renaming things in the win32 folder". Cracking open and messing about with a flat-file all-in-one DB is a higher barrier to screwing around. To be fair, permissions mitigate this risk substantially.

- Intermediate failures with some single-flat-file DBs are much less impactful than with many filesystems. Two parts to this: one is that a more rigid structure in a DB prevents certain invalid writes entirely; the other is transactionality. While plenty of local-flat-file "myapp.library" DBs don't have a good atomicity story underneath (I'm always saddened when I poke at a proprietary data library format and find that it contains a bug-ridden, informally-specified implementation of half of SQLite), and while some file systems make logical atomicity possible to achieve (e.g. via CoW copying data/directories, doing mutations, atomically swapping a source-of-truth link to the new version, and dropping the old), filesystem-as-database systems tend to fail-corrupted often due to unexpected issues (from bugs to "oops, don't have write access on 1/1000 files" to SIGKILL/power loss/drive failure) during data modifications. While I wish more file-based systems were as robust as maildir, I won't hold my breath when SQLite is right there.


I think you're talking about file systems as an implementation detail, whereas the author seems to be talking about file systems as a user experience.

For example, iOS devices have a file system, but for 99%+ of iPhone/iPad users it doesn't. Presumably, the author's "Digits" object database is persisted via a file system.

Also, a user-facing file system isn't needed for things to have a location, and things can have a location without needing a user-facing file system.


So, a file system?
next

Legal | privacy