CentOS 8.x Moosefs 3.x Initial setup

<yambe:breadcrumb self="Initial setup">CentOS 8.x Moosefs 3.x|Moosefs 3.x</yambe:breadcrumb>

CentOS 8.x Moosefs 3.x Initial setup

Theory

To setup moosefs community edition you need following types of servers:

master

This is the master server which coordinates all activities within the distributed filesystem among various nodes. This maintains metadata about files (Which file / chunk is at which node).

metalogger

This server can keep track of changes to metadata. If master goes down we should be able to bring up another master using this server

chunk servers

These nodes store data and contribute to storage space / throughput. Chunk servers can specify a mounted filesystem or device that can be used by moosefs for storing data

clients

These nodes mount the distributed file-system. Multiple clients can mount the filesystem for rw concurrently without any issue.

Initialization

For moosefs to work above nodes (master, metalogger, chunkservers, etc.) should resolve each other by name. That can be done by DNS or by /etc/hosts file. For purpose of this document assume following names and IPs are present in /etc/hosts in all nodes:

172.31.1.183  mfsmaster
172.31.1.184  mfsmetalogger
172.31.1.185  mfschunk1
172.31.1.186  mfschunk2
172.31.1.187  mfschunk3
172.31.1.188  mfsclient1

Here name mfsmaster is critical as it is default in most configuration files. If you change this then please plan to change the name appropriately in most configuration files on most nodes.

Master setup

To setup master node use:

1. Configure moosefs repositories using:

   curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
   curl "http://ppa.moosefs.com/MooseFS-3-el8.repo" > /etc/yum.repos.d/MooseFS.repo

2. Install required packages for master

   dnf -y install moosefs-master moosefs-cgi moosefs-cgiserv moosefs-cli

3. Configure master server

   cd /etc/mfs
   cp mfsmaster.cfg.sample  mfsmaster.cfg
   cp mfsexports.cfg.sample  mfsexports.cfg

4. Edit /etc/mfs/mfsexport.cfg and append password=<secret> for both root (/) and meta(.) config lines
5. Create '/etc/default/moosefs-master' with:

   MFSMASTER_ENABLE=true

6. Start and enable moosefs-master service. Also check its running status:

   systemctl start moosefs-master
   systemctl enable moosefs-master
   systemctl status moosefs-master

7. Configure CGI server to start on boot by creating '/etc/default/moosefs-cgiserv' with

   MFSCGISERV_ENABLE=true

8. Start and enable moosefs-cgiserv service using:

   systemctl start moosefs-cgiserv
   systemctl enable moosefs-cgiserv
   systemctl status moosefs-cgiserv

9. Disable and stop firewalld

   systemctl disable firewalld
   systemctl stop firewalld

10. Now the CGI can be accessed at http://172.31.1.183:9425/ Most of the values would be blank as there are no other nodes in the setup at the moment.
11. We can also run mfscli commands such as:

    mfscli -SIN

    Use 'mfscli -h' to get help on using mfscli command.

Metalogger setup

As explained earlier in case of master failure, the data from metalogger can be used to build a new master. To setup metalogger server use:

1. Configure moosefs repositories using:

   curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
   curl "http://ppa.moosefs.com/MooseFS-3-el8.repo" > /etc/yum.repos.d/MooseFS.repo

2. Install required packages for metalogger using:

   dnf -y install moosefs-metalogger

3. Configure metalogger using:

   cd /etc/mfs
   cp mfsmetalogger.cfg.sample  mfsmetalogger.cfg

4. Configure metalogger to start automatically on boot by creating '/etc/default/moosefs-metalogger' with

   MFSMETALOGGER_ENABLE=true

5. Stop and disable firewalld using:

   systemctl stop firewalld
   systemctl disable firewalld

6. Start and enable metalogger service using:

   systemctl start moosefs-metalogger
   systemctl enable moosefs-metalogger
   systemctl status moosefs-metalogger

7. You can see the metalogger server on servers tab of master web CGI interface

Chunk server setup

1. Configure moosefs repositories using:

   curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
   curl "http://ppa.moosefs.com/MooseFS-3-el8.repo" > /etc/yum.repos.d/MooseFS.repo

2. Install required packages for chunk server

   dnf -y install moosefs-chunkserver

3. Configure chunk server using:

   cd /etc/mfs
   cp mfschunkserver.cfg.sample mfschunkserver.cfg
   cp mfshdd.cfg.sample mfshdd.cfg

4. It is possible for moosefs to use any folder for storing chunks. In case you want to use entire disks, it is best to format them and mount them appropriately eg /mnt/mfschunks1 etc. Example steps for formatting /dev/sdb and mounting it at /mnt/mfschunks1 are:
   1. Create a single partition for storing chunks

      parted  --align  optimal /dev/sdb
      (parted) mklabel  gpt
      (parted) mkpart  mfschunks1  0% 100%
      (parted) q

   2. Install xfs utilities

      dnf -y install xfsprogs

   3. Create required xfs filesystem using:

      mkfs.xfs /dev/sdb1

      If the drives have 4k sector size use 'mkfs.xfs -s size=4k /dev/sdb1' instead

   4. Create appropriate entires in /etc/fstab to mount the partition on /mnt/mfschunks1 folder

      mkdir /mnt/mfschunks1
      blkid
      #vim /etc/fstab
      #UUID="842950d7-89bd-4272-9532-24773a74d164"     /mnt/mfschunks1 xfs     defaults 0 0
      mount -a
      df -h

5. Set correct ownership and permission on folder to be used for mfs chunks

   chown  mfs:mfs /mnt/mfschunks1
   chmod  770 /mnt/mfschunks1

6. Specify chunkfolder in '/etc/mfs/mfshdd.cfg' as:

   /mnt/mfschunks1

7. Configure chunk server to start automatically on boot by creating '/etc/default/moosefs-chunkserver' with

   MFSCHUNKSERVER_ENABLE=true

8. Stop and disable firewalld using:

   systemctl stop firewalld
   systemctl disable firewalld

9. Start and enable metalogger service using:

   systemctl start moosefs-chunkserver
   systemctl enable moosefs-chunkserver
   systemctl status moosefs-chunkserver

10. You can see the chunk servers on servers tab of master web CGI interface

Configure moosefs-client

1. Configure moosefs repositories using:

   curl "https://ppa.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
   curl "http://ppa.moosefs.com/MooseFS-3-el8.repo" > /etc/yum.repos.d/MooseFS.repo

2. Install required packages for chunk server

   dnf -y install moosefs-client

3. Create folder for mounting moosefs data

   mkdir /mnt/mfs

4. Mount the folder using:

   mfsmount /mnt/mfs -H mfsmaster -p

   Specify the password given in mfsexport.cfg in mfsmaster

5. Check the space of mfs mount

   df -h

6. To mount using /etc/fstab instead of using mount command use syntax:

   mfsmount		/mnt/primary		fuse	defaults,mfspassword=<secret> 0 0

   where <secret> is the password given in mfsexport.cfg in mfsmaster

7. You can see the client information in mfsmaster CGI web UI.

Refer:

• https://moosefs.com/Content/Downloads/moosefs-installation.pdf

<yambe:breadcrumb self="Initial setup">CentOS 8.x Moosefs 3.x|Moosefs 3.x</yambe:breadcrumb>