FortiVM – Disk Error

So recently I have seeing lots of vm disk mount error on the Fortigate FortiVM’s. I attempted to research but nothing actually mentioning why. Further to this, I had applied some configuration for a client as always a good practice, I had done a backup before the new configurations and afterwards. I then did a reboot of the system and guess what? the configurations did not save.

Here is a screen shot of the error:

FortiVM disk error

Lucky enough I was able to re-upload the backup with the new configurations and reboot. Also was done after hours. I logged a call with Fortinet support. They were also not able to advise why this happened. From my experience, this may happen when there either of the following:

  1. The VM had a power reset on the Hypervisor level.
  2. The storage of the VM had an issue.
  3. The VM was migrated between hosts (Dirty method)

Another issue I have come across with the disk mount error is if you are running an HA solution. Firstly the changes do not sync from the primary device to the HA. Secondly, when you failover, due to the primary and secondary device is not in sync, the primary does not hand over sessions, connections and role to the HA. The only way to resolve this is reboot the primary and once it comes up, it should sync to the HA (It maybe temporarily).

Resolution

I would recommend running a disk scan. At first, I was skeptical and also the system tells you it could be several hours. However, I attempted running it. You can setup in the GUI to automatically scan the disk upon rebooting but I prefer controlling it. Go into the CLI session or directly onto the VM via your hypervisor. Runn the following commands:

execute disk list“- will provide a list of disk. If you are using the system and image as default, you should have one disk.

Disk Internal(boot) ref: xxGB type: SSD [ATA SanDisk SSD U100] dev: /dev/sda

partition ref: 3 xxGB, xxGB free mounted: Y label: xxxxxxxxxxxxx dev: /dev/sda3

You will use the ref number when you scan the disk

execute disk scan <ref>

it will prompt the following:

scan requested for: 3/Internal (device=/dev/sda3)

This action requires the unit to reboot.

Do you want to continue? (y/n)

Journal commit error

This error means it is not saving any info to the disk.

Once it reboots, monitor the system for about 15 minutes. if there is no disk mount errors, you should be good. Always backup. Also on your hypervisor, if you have multiple hosts, rather have the FortiVM, static and only move if the host is under strain or needs to be drain for maintenance.

Hope this helps.

2 Comments

  1. How big was the partition that you ran this disk check for? I want to get a rough estimate on how long the file system check will take for a VM_64 using 50GB partition. Thanks

Leave a Reply

Your email address will not be published.


*


This site uses Akismet to reduce spam. Learn how your comment data is processed.