Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
An update on bcachefs
An update on bcachefs
Posted May 23, 2018 23:26 UTC (Wed) by doublez13 (guest, #122213)Parent article: An update on bcachefs
Bcachefs is starting to sound like a very promising fs, but btrfs has a hell of a head start.
(Log in to post comments)
An update on bcachefs
Posted May 23, 2018 23:29 UTC (Wed) by Cyberax (✭ supporter ✭, #52523) [Link]
An update on bcachefs
Posted May 23, 2018 23:51 UTC (Wed) by rahvin (guest, #16953) [Link]
btrfs gets a bad rap for reasons I don't quite understand. All these new filesystems end up with tons of corner cases once they get broad deployment, btrfs was moving pretty fast until it got pushed out of beta then it slowed down as they spent and continue to spend tons of time fixing all these corner/edge cases. Although I'd argue significant prior available resources were pulled away when Oracle bought Sun as they were a prime backer.
Personally I'll be surprised if bcachefs doesn't run into the same long tail of corner cases once it's mainlined. I remember reading something saying it takes at least 10 years after a filesystem is mainlined before they deal with all these issues, after all it was pretty much 10 years after XFS was brought in before it really got stable.
An update on bcachefs
Posted May 24, 2018 1:21 UTC (Thu) by simcop2387 (subscriber, #101710) [Link]
An update on bcachefs
Posted May 24, 2018 10:30 UTC (Thu) by pizza (subscriber, #46) [Link]
I've had two [might-as-well-be-]total filesystem losses with btrfs, both after clean shutdown/reboot cycles on lightly-loaded, battery-backed hardware that hasn't so much as hiccupped before or since.
An update on bcachefs
Posted May 25, 2018 15:54 UTC (Fri) by drag (guest, #31333) [Link]
1st time was when my cat yanked out a PCI card that wasn't screwed in. Corrupted xfs metadata, had to sort through tens of thousands of files in the recovery bin.
2nd time was while using btrfs.
There was also a couple times while using btrfs on external storage device I expanded it wedged itself into a corner. Ran out of inodes or something like that. It was a long time ago. Didn't lose data, but the file system was effectively rendered worthless and needed to be rebuilt.
Now these things are all partially my fault, but I have used Linux on garbage-level (sometimes literally) hardware for years and it's been proven to be tough.
I am looking forward to bcachefs getting support from various distro installers.
An update on bcachefs
Posted May 24, 2018 13:11 UTC (Thu) by dcg (subscriber, #9198) [Link]
As someone who has been using btrfs for many years without problems I also feel that btrfs is not appreciated enough (and that bcachefs is over-hyped - Overstreet has criticized btrfs for its stability, but I have seen several cases of people having corruption issues in the bcachefs IRC channel). But I wonder how much can that last if btrfs keeps not progressing in basic features like RAID5/6?
An update on bcachefs
Posted May 24, 2018 6:14 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]
From my experience, I'm using BTRFS on my personal external RAID tower. It got corrupted every single time I tried its multi-device support. Mostly because of botched RAID rebuilds after power failures or disk replacements.
I've got tired of this and switched to a good old MD-based RAID. I'm still using BTFS for snapshots, but there's no way in hell I'm going to touch its multi-device snapshot within the next 5 years or so.
An update on bcachefs
Posted May 24, 2018 7:48 UTC (Thu) by khim (subscriber, #9252) [Link]
An update on bcachefs
Posted May 24, 2018 7:53 UTC (Thu) by Cyberax (✭ supporter ✭, #52523) [Link]
An update on bcachefs
Posted May 25, 2018 19:26 UTC (Fri) by khim (subscriber, #9252) [Link]
Ok, I'll bite. You say that "back then" page looked nice and cheerful. Here's the full history.
When exactly RAID56 wasn't marked with red and word "Unstable"?
When you are ignoring warnings written in red... bad things happen... that's just how life is it...
An update on bcachefs
Posted May 25, 2018 20:54 UTC (Fri) by Cyberax (✭ supporter ✭, #52523) [Link]
Also, I wasn't aware of this page's existence. RAID56 search on Google gives another page that looked OK.
An update on bcachefs
Posted May 31, 2018 21:45 UTC (Thu) by Wol (subscriber, #4433) [Link]
Even md raid has only just fixed it, with the addition of a journal (a feature that might still be experimental). The journal is intended primarily as an optimisation, flushing updates to SSD before saving them to the full raid on spinning rust. It fixes the write hole almost as a side-effect.
(Note, understanding the journal is on my to-do list, I need to learn more about it before I can document it ... :-)
Cheers,
Wol
An update on bcachefs
Posted Jun 2, 2018 17:52 UTC (Sat) by koverstreet (subscriber, #4296) [Link]
An update on bcachefs
Posted May 24, 2018 21:50 UTC (Thu) by rahvin (guest, #16953) [Link]
It just seems like everyone wants to work on something new rather than try to fix what we've got. I get that, fixing bugs is hard and boring rather than developing something new and shiny. But that new and shiny is still going to take 10 years to stabilize just like every filesystem Linux has had.
This is actually a question I'd like to see answered at one of these filesystem conventions, Why is it so hard to get experienced developers to work stabilizing the fs's we have rather than building new ones?
An update on bcachefs
Posted May 24, 2018 22:51 UTC (Thu) by nivedita76 (guest, #121790) [Link]
XFS is around 25 years old, and has been in Linux since the early 00's, not new and shiny. BTRFS is the "new and shiny" fs by comparison. XFS history
An update on bcachefs
Posted May 24, 2018 7:18 UTC (Thu) by vadim (subscriber, #35271) [Link]
Eg, the "raid1 volumes only mountable once RW if degraded" issue was there until very recently and is not the kind of thing one wants to deal with on a RAID setup.
Compression until recently came with warnings.
Compression is still quite half-assed on the UI side. Why can't I easily check how well a file is being compressed? Why do I have to 'defrag' to compress, on a SSD? I don't care if it's fragmented, I just want it compressed. There's no need to move already compressed data around.
Snapshots are in theory very cool, but in practice easily cause horrible performance issues. Eg, I unwisely had installed snapper and allowed DNF to make a couple dozen snapshots of my filesystem. Cleaning that up took an entire day, on an SSD, with the first snapshots taking hours to remove and completely locking up the machine. I don't want to imagine how that would work on an HDD.
Despite my best efforts so far I haven't figured out a way to do a btrfs scrub in such a way that it doesn't make the entire desktop lag, on a computer with a SSD.
An update on bcachefs
Posted May 25, 2018 2:03 UTC (Fri) by zlynx (guest, #2285) [Link]
One thing to point out about slow btrfs scrubs is that it depends on your IO scheduler. Use the standard CFQ or BFQ. If you use deadline or noop then of course the scrub is going to use up all the IO because nothing will stop it.
It still causes some extra latency. On my email server I use multi-queue BFQ and I can feel a bit of extra delay when loading each IMAP message from it during a scrub, but I don't think it's that serious.
And on my laptop with NVMe SSD the btrfs scrub is done in about 30 seconds, so it isn't a big deal there either. It reads and verifies at hundreds of megabytes per second.
An update on bcachefs
Posted May 23, 2018 23:49 UTC (Wed) by EdwardConnolly (guest, #123865) [Link]
An update on bcachefs
Posted May 24, 2018 6:00 UTC (Thu) by zdzichu (subscriber, #17118) [Link]
Incidentally, my main setup is bcache+btrfs ( https://enotty.pipebreaker.pl/dżogstaff/2016.05.25-opcja2... ) but it doesn't seem that bcachefs could be a feature-complete replacement.
An update on bcachefs
Posted May 24, 2018 11:36 UTC (Thu) by mchouque (subscriber, #62087) [Link]
If you do RAID 1 on more than 2 devices (say n devices), btrfs only guarantees you have 2 copies of your data, not n.
https://btrfs.wiki.kernel.org/index.php/FAQ#What_are_the_...
More over, you only read from one disk (IIRC I think they do load balancing with the PID or something like that so you really use RAID 1 / 10 for redundancy more than performance).
When you loose a device in RAID1, you can mount the FS but only once... https://btrfs.wiki.kernel.org/index.php/Gotchas#raid1_vol...
As for RAID10, it inherits the design features of RAID1. From the doc, it says "RAID-10 is built on top of these definitions. Every stripe is split across to exactly 2 RAID-1 sets and those RAID-1 sets are written to exactly 2 devices". The issue is you do not control on which devices your stripe is mirrored.
So a double disk failure on a btrfs RAID10 is more likely to end up with data loss data than on a traditional RAID-10.
There was a long discussion about that years ago: https://www.mail-archive.com/linux-btrfs@vger.kernel.org/...
>The example below is probably a pathological case - but here goes. Let's say in this 4-disk example that chunks are >striped as d1,d2,d1,d2 where d1 is the first bit of data and d2 is the second:
>Chunk 1 might be striped across disks A,B,C,D d1,d2,d1,d2
>Chunk 2 might be striped across disks B,C,A,D d3,d4,d3,d4
>Chunk 3 might be striped across disks D,A,C,B d5,d6,d5,d6
>Chunk 4 might be striped across disks A,C,B,D d7,d8,d7,d8
>Chunk 5 might be striped across disks A,C,D,B d9,d10,d9,d10
>
>Lose any two disks and you have a 50% chance on *each* chunk to have lost that chunk. With traditional RAID10 you >have a 50% chance of losing the array entirely. With btrfs, the more data you have stored, the chances get closer to 100% >of losing *some* data in a 2-disk failure.
>
>In the above example, losing A and B means you lose d3, d6, and d7 (which ends up being 60% of all chunks).
>Losing A and C means you lose d1 (20% of all chunks).
>Losing A and D means you lose d9 (20% of all chunks).
>Losing B and C means you lose d10 (20% of all chunks).
>Losing B and D means you lose d2 (20% of all chunks).
>Losing C and D means you lose d4,d5, AND d8 (60% of all chunks)
>The above skewed example has an average of 40% of all chunks failed. As you add more data and randomise the >allocation, this will approach 50% - BUT, the chances of losing *some* data is already clearly shown to be very close to >100%.
The bottom line is RAID1 or 10 as defined by btrfs is not what you think they are if you haven't read the fine prints...
An update on bcachefs
Posted May 31, 2018 21:50 UTC (Thu) by Wol (subscriber, #4433) [Link]
I won't say more than that mdraid-10 is NOT raid 1+0. Look it up if you can be bothered.
Cheers,
Wol
