Security Advancements at the Monastery

“I’ve decided to change the copyright to have the same set of rules as the GNU copyleft – I got some mail asking about it, and I agree..” — Linus Torvalds

Linux creator Linus Torvalds talked with Nick Miller from the Sydney Morning Herald about his work and ambitions for his operating system. Linux expressed his preference for Mac OS X over Windows. The part that I found most entertaining was when he described the Mac OS X file system HFS+ as “complete and utter craps.” When asked, what he thought about the regular hype about the release of a new version of Microsoft Windows or Apple OS X, Linus responded:

An o/s should never have been something that people (in general) really care about: it should be completely invisible and nobody should give a flying f*** about it except the technical people.

It’s stupid – when you make a big deal about something like Vista or Leopard a lot of it is about things I don’t consider to be the operating system. It’s about the visual shell around it. The fact Microsoft tied the two together so much actually caused them problems, not just the legal problems. If you manage a thousand clients, or a hundred thousand clients which is not at all unheard of, you sure as hell don’t want to point and click at them. In many ways Microsoft has had to fix the design mistakes they made when they thought the graphical approach should be a very intimate part of (Windows).

To Microsoft and Apple the o/s is important as a way to control the whole environment, from a marketing and money-making standpoint, to force people to upgrade their applications, and your hardware

When asked, if he had a favorite between Leopard and Vista, Linus responded:

I don’t think they’re equally flawed. I think Leopard is a much better system. On the other hand, (I’ve found) OS X in some ways is actually worse than Windows to program for. Their file system is complete and utter crap, which is scary. I think OS X is nicer than Windows in many ways, but neither can hold a candle to my own (Linux). It’s a race to secondplace!

In my last post “Bro on FreeBSD 7: Should You Care?”, ZFS was listed as one of the improvements of FreeBSD 7. The posts “The Advantages of Sun’s ZFS Filesystem” and “Ten Reasons to Reformat Your Hard Drives” discuss some of the advantages of ZFS. Blogger jamesd_wi posted a very nice collection of links concerning ZFS titled “Interested in ZFS.” Paul Venezia’s article in InfoWorld titled, “Sun ZFS breaks all the rules” does a real nice job of hitting the high points.

To address Linus comment on HFS+, Apple is developing ZFS for Mac OS X. Noel Dellofano, who is part of the ZFS development team, did release binaries and source code of ZFS for Mac OS X on the MacOSForge site. Noel cautioned in his post, “As I mentioned, ZFS is still BETA, so there are (and likely will be) some issues turn up with compatibility with the upper layers of the system if that’s what you’re referring to.” The Mac OS forge has created a “Known Issues and Features in the Works” area to help keep people apprised of the development of ZFS under Mac OS X.

What does Linus have to say about recent file system development, and in particular ZFS? Swapnil Bhartiya talked to Linus in his article,”Linus Torvalds: I Have Never Really Talked To Microsoft!” On the question of ZFS, Linus responed

Actually, just yesterday we had a git performance issue, where ZFS was orders of magnitude slower than UFS for one user (not under Linux, but git is gaining a lot of traction even outside of kernel development). So I think a lot of the ‘new file system’ mania is partly fed by knowing about the issues with old file systems, and then the (somewhat unrealistic) expectation that a ‘new and improved’ file system will make everything perfect.

In the end, this is one area where you just let people fight it out. See who comes out the winner — and it doesn’t need to be (and likely will not) be a single winner. Almost always, the right choice of file system ends up depending on the load and circumstances.

One thing that I’m personally more excited about than any of the file systems you mention is actually the fact that Flash-based hard disks are quickly becoming available even for ‘normal’ users. Sure, they’re still expensive (and fairly small), but Flash-based storage has such a different performance profile from rotating media, that I suspect that it will end up having a large impact on file system design. Right now, most file systems tend to be designed with the latencies of rotating media in mind.

A very interesting point concerning technological innovation. Linus does in a post state that ZFS is “one of their very very few bright spots.” By “their” he means Sun. Linus problem with ZFS is that “ZFS and the other things that people are drooling about in a way that lets Linux use them on an equal footing. I can pretty much guarantee that. They don’t like competition on that level. They’d *much* rather take our drivers and _not_ give anything back, or give back the stuff that doesn’t matter.” Linus goes on to say, “And yes, maybe ZFS is worthwhile enough that I’m willing to go to the effort of trying to relicense the kernel. But quite frankly, I can almost guarantee that Sun won’t release ZFS under the GPLv3 even if they release other parts. Because if they did, they’d lose the patent protection.”

Jonathan Schwartz, Chief Executive Officer and President Sun Microsystems, Inc., responded in his blog post “An OpenSolaris/Linux Mashup.” Jonathan writes, “We want to work together, we want to join hands and communities – we have no intention of holding anything back, or pulling patent nonsense. And to prove the sincerity of the offer, I invite you to my house for dinner. I’ll cook, you bring the wine. A mashup in the truest sense.” We will leave Jonathan and Linus to settle their issues. If you are interested, you can listen to Linus on the Linux Foundation Podcast. They have two part interview (part 1, part 2) where Linus comments on a variety of topics including patents, internal and external competition, the broader adoption of Linux, Microsoft and much more. If you want to know more about GPLv3, Eben Moglen, a leading advocate of open source rights, gives the talk “The Renaissance of Invention: Free Software and the Next American Century” on the IT Conversations podcast. In the talk, Eben addresses the controversy about GPLv3 and Linux, and whether software patents are more dangerous than other patents. To some, licensing issues might not be the most exciting topic, but it is important to be aware of the issues. Development and adoption of projects, such as ZFS, can be affected.

Drew Thaler has a few things to say about ZFS compared to HFS+. Drew worked in Apple’s CoreOS filesystems group. Before that, he worked on DiscRecording.framework, and singlehandedly created the content subframework that streamed out HFS+, ISO-9660, and Joliet filesystems. Before that, he worked on the same thing for Mac OS 9. And before that, he worked on mass storage drivers for external USB/FireWire drives and internal ATA/ATAPI/SCSI drives. Currently he works on a file I/O subsystem for PlayStation 3 games. In short, Drew knows what he is talking about. He wrote the post, “Don’t be a ZFS Hater” and a follow up post “ZFS Hater Redux” which contain some amazing information. Drew’s makes a very important point when he writes:

Some very smart people at Sun started the ball rolling by putting an awful lot of thought into the future of storage, and they came up with ZFS.

After they announced it and started talking about it, other brilliant people at Apple (and FreeBSD, and NetBSD) paid attention to what they were doing. And they listened, and thought about it, and looked at the code, and wound up coming around to the side of ZFS as well.

The original HFS design is twenty years old. It has had a good run, but so much has changed in the world of computers. As Linus pointed out, technology changes and file systems need to be redesigned for these changes. Some brilliant folks have worked on ZFS . It is not a patched file system trying to handle the latest requirements of computers. ZFS is a file system designed with the flexibility to handle data requirements for the next ten years. It will not be the final word in file systems. The world continues to innovate and change. The point is, ZFS is here now after seven years of development. It sets a new standard for future file systems. I look forward to seeing ZFS implemented in the latest Mac and FreeBSD operating systems.