Setup all with ansible-playbook and it worked fine untill I tested the performance for git clone, create file, rm file. I setup a disk of 50gb on each server and I configure zfs with RAID-0 and glusterfs with replication and no distribution. I setup a three node GlusterFs cluster on OVH with sandbox VM (2GB ram and 1CPU). I want to do a feedback about my own experience. Thanks you for sharing your experience about GLUSTERFS over ZFS. That turns off glusters io cache which seems to insulate the VM’s from the lack of O_DIRECT, so you can leav the vm caching off. You have to switch them to write through or write back caching before they will work.Īnd if you are running glusterfs on top of ZFS, hosting KVM images its even less obvious and you will get weird i/o errors until you switch the kvm to use caching. One thing – ZFS doesn’t support O_DIRECT, which can give you grief if running KVM images as by default they require direct access. Much better to have it integrated in the fs and of course the management/reporting tools are much better. I fiddled a lot with dm-cache, bcache and EnhanceIO – managed to wreck a few filesystems, before settling on zfs. One of the big advantages Im finding with zfs, is how easy it makes adding SSD’s as journal logs and caches. Thanks for the various articles on gluster, zfs and proxomox, they have been most helpful. ZFS handles disk level corruption and hardware failure whilst GlusterFS makes sure storage is available in the event a node goes down and load balancing for performance. These two technologies combined provide a very stable, highly available and integral storage solution. ![]() To see how to set up GlusterFS replication on two nodes, see this article. In Gluster terminology, this is called replication. We now need to synchronise the storage across both physical machines. At this point, you should have two physical servers presenting exactly the same ZFS datasets. Configure the required ZFS datasets on each node, such as binaries, homes and backup in this example. Set up ZFS on both physical nodes with the same amount of storage, presented as a single ZFS storage pool. For the very latest ZFS binaries, you will need to use Solaris as the ZFS on Linux project is slightly behind the main release. See this post for setting up ZFS on Ubuntu. On top of this storage layer, GlusterFS will synchronise, or replicate, the two logical ZFS volumes to present one highly available storage volume. GlusterFS handles this synchronisation seamlessly in the background making sure both of the physical machines contain the same data at the same time.įor this storage architecture to work, two individual hardware nodes should have the same amount of local storage available presented as a ZFS pool. GlusterFS is then set up on top of these 3 volumes to provide replication to the second hardware node. The volume is then split into three sub volumes which can have various properties applied for example, compression and encryption. This provides redundant storage and allows recovery from a single disk failure with minor impact to service and zero downtime. Each node contains three disks which form a RAIDZ-1 virtual ZFS volume which is similar to RAID 5. The below diagram shows the high level layout of the storage set up. Clients can mount storage from one or more servers and employ caching to help with performance. Client – this is the software required by all machines which will access the GlusterFS storage volume.The server also handles client connections with it’s built in NFS service. Server – the server is used to perform all the replication between disks and machine nodes to provide a consistent set of data across all replicas.GlusterFS is a distributed file system which can be installed on multiple servers and clients to provide redundant storage. This gives your file storage added redundancy and load balancing and is where GlusterFS comes in. Distributed file systems can span multiple disks and multiple physical servers to produce one (or many) storage volume. ![]() The problem with ZFS is that it is not distributed. If you are using ZFS on Linux, you will need to use a 3rd party encryption method such as LUKS or EcryptFS. Please note, although ZFS on Solaris supports encryption, the current version of ZFS on Linux does not.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |