Search This Blog

Showing posts with label Storage. Show all posts
Showing posts with label Storage. Show all posts

Monday, August 31, 2026

Expanding a virtual disk on a live Linux virtual machine does not require downtime, complex LVM setups, or reboots.

Managing disk space on Linux virtual machines often involves two recurring tasks: attaching a brand-new virtual disk (for example, offloading /var/log) and expanding an existing partition online when data grows. Both workflows can be performed on a running VM with zero downtime, no LVM overhead, and without requiring a reboot.


Part 1: Adding, Partitioning & Mounting a New Disk (e.g., /var/log)

If you have just attached a new virtual disk to your VM, you need to scan the SCSI bus, partition the drive, format the filesystem, and configure persistent mounting.

1. Scan All SCSI Hosts for the New Disk

Trigger a bus rescan across all SCSI hosts so the kernel discovers the newly attached block device without restarting:

for SCSI_HOST in /sys/class/scsi_host/*; do echo "- - -" > $SCSI_HOST/scan; done

2. Create the Partition

Use gdisk (or fdisk) to initialize the partition table on the new device (e.g., /dev/sdb):

gdisk /dev/sdb

(Create a new partition /dev/sdb1 using default parameters and write the changes.)

3. Format the Partition

Format the partition to ext4 and assign a filesystem label (e.g., VARLOG):

mkfs.ext4 -L VARLOG /dev/sdb1

4. Migrate Existing Data & Mount

Mount the new partition to a temporary location, move the existing log files, and mount the partition directly to /var/log:

mount /dev/sdb1 /mnt && mv /var/log/* /mnt && umount /mnt && mount /dev/sdb1 /var/log && df -h

5. Make the Mount Persistent

Open /etc/fstab in your editor:

vi /etc/fstab

Append the following entry:

/dev/sdb1                    /var/log          ext4    defaults        0      2

Part 2: Enlarge a Disk and Partition Online (No Reboot)

When the disk runs low on space, you can expand the virtual disk from your hypervisor and grow the filesystem dynamically.

Prerequisites & Limitations

  • No Active Snapshots: Ensure hypervisor snapshots are consolidated before resizing the disk.
  • Partition Position: Online expansion applies directly to the last or only partition on the block device.
  • Required Tool: Install growpart:
    sudo apt-get install -y cloud-utils        # Debian/Ubuntu
    sudo dnf install -y cloud-utils-growpart   # RHEL/CentOS/Rocky

Step-by-Step Expansion

  1. Increase Disk Size in Hypervisor: Expand the provisioned size in VMware, Proxmox, or cloud portal.
  2. Rescan the Specific Device:
    echo 1 > /sys/class/block/sdb/device/rescan

    Verify the new capacity in dmesg:

    dmesg | tail -n 10
    # Look for: "sdb: detected capacity change from X to Y"
  3. Grow the Partition Table:
    growpart /dev/sdb 1
  4. Resize the Filesystem:

    For EXT4:

    resize2fs /dev/sdb1

    For XFS:

    xfs_growfs /var/log

Automation via Cron

To automate live expansion on recurring data drives, you can schedule a script in /usr/local/bin/disk_resize.sh:

#!/usr/bin/env bash
set -euo pipefail

# Rescan device
echo 1 > /sys/class/block/sdb/device/rescan
sleep 2

# Grow partition and resize ext4
if growpart /dev/sdb 1; then
    resize2fs /dev/sdb1
    echo "[$(date)] Successfully expanded /dev/sdb1"
fi

Schedule it in /etc/crontab:

*/5 * * * * root /usr/local/bin/disk_resize.sh > /var/log/disk_resize.log 2>&1

Friday, November 25, 2022

Growing a ext4 disk on a cloud / VMware VM

Normal I use GParted CD boot a VM to growing a Linux ext4 disk partition. Or using LVM. It's way easier on Windows VM. I tried cloud-utils tool on a Debian 10 VM on VMware ESXi 7. Just a bare ext4 partition without LVM. It should works on any other Cloud plantform like: AWS, Oracle or Google. Install cloud-utils after increase the disk size from the Cloud Control Console:
sudo apt-get install -y cloud-utils
################### ### Before ########## ###################
df -h
	Filesystem      Size  Used Avail Use% Mounted on
	/dev/sda1        16G   11G  4.2G  72% /
	/dev/sdb1       295G  247G   33G  89% /data
################### ### grow /dev/sdb1 ### ###################
echo 1 > /sys/class/block/sdb/device/rescan
growpart /dev/sdb 1
	CHANGED: partition=1 start=2048 old: size=629143552 end=629145600 new: size=880801759,end=880803807

resize2fs /dev/sdb1
	resize2fs 1.44.5 (15-Dec-2018)
	Filesystem at /dev/sdb1 is mounted on /data; on-line resizing required
	old_desc_blocks = 38, new_desc_blocks = 53
	The filesystem on /dev/sdb1 is now 110100219 (4k) blocks long.
################### ### grow /dev/sda1 ### ###################
echo 1 > /sys/class/block/sda/device/rescan
growpart /dev/sda 1
	CHANGED: partition=1 start=2048 old: size=33550336 end=33552384 new: size=83883999,end=83886047

resize2fs /dev/sda1
	resize2fs 1.44.5 (15-Dec-2018)
	Filesystem at /dev/sda1 is mounted on /; on-line resizing required
	old_desc_blocks = 2, new_desc_blocks = 5
	The filesystem on /dev/sda1 is now 10485499 (4k) blocks long.
################### ### After ########### ###################
df -h
	Filesystem      Size  Used Avail Use% Mounted on
	/dev/sda1        40G   11G   27G  29% /
	/dev/sdb1       413G  247G  146G  63% /data

Friday, October 24, 2014

CentOS 7 add new SSD in LVM format XFS

Got sometime with CentOS 7 recently.

Try to add an New SSD into system.
I know to get the best performance should avoid LVM, but for easy extend I will using LVM here.

First make sure you are using newer fdisk (added SSD alignment support after 2.17), CentOS 7 came with 2.23.2.


# fdisk /dev/sdc
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0xd6b73b64.

Command (m for help): p

Disk /dev/sdc: 214.7 GB, 214748364800 bytes, 419430400 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xd6b73b64

Device Boot Start End Blocks Id System

Command (m for help): q


Create partition and change the flag to LVM (e8):


# fdisk -c -u /dev/sdc
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0xab948127.

Command (m for help): p

Disk /dev/sdc: 214.7 GB, 214748364800 bytes, 419430400 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xab948127

Device Boot Start End Blocks Id System

Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-419430399, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-419430399, default 419430399):
Using default value 419430399
Partition 1 of type Linux and of size 200 GiB is set

Command (m for help): p

Disk /dev/sdc: 214.7 GB, 214748364800 bytes, 419430400 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xab948127

Device Boot Start End Blocks Id System
/dev/sdc1 2048 419430399 209714176 83 Linux

Command (m for help): t
Selected partition 1
Hex code (type L to list all codes): 8e
Changed type of partition 'Linux' to 'Linux LVM'

Command (m for help): p

Disk /dev/sdc: 214.7 GB, 214748364800 bytes, 419430400 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xab948127

Device Boot Start End Blocks Id System
/dev/sdc1 2048 419430399 209714176 8e Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.



Create PV


# pvcreate /dev/sdc1
Physical volume "/dev/sdc1" successfully created


Display PV

# pvdisplay
--- Physical volume ---
PV Name /dev/sda2
VG Name centos_milliondollorserver-database
PV Size 15.51 GiB / not usable 3.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 3970
Free PE 0
Allocated PE 3970
PV UUID tY2Myw-m0hm-sscc-Qc8M-hOfJ-XNA6-fZUIxv

--- Physical volume ---
PV Name /dev/sdb1
VG Name VG_Backup
PV Size 100.00 GiB / not usable 3.00 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 25599
Free PE 0
Allocated PE 25599
PV UUID wNDnMs-vOAL-WwAX-0N6d-xzsJ-ap85-gDbnik

"/dev/sdc1" is a new physical volume of "200.00 GiB"
--- NEW Physical volume ---
PV Name /dev/sdc1
VG Name
PV Size 200.00 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID o9Hvzz-YDy0-p8pa-aOhJ-nPtA-nNUX-kX1WzF


Create VG

# vgcreate VG_SSD00 /dev/sdc1
Volume group "VG_SSD00" successfully created


Create LV using all free space available on the VG just created


# lvcreate -l 100%FREE -n LV_SSD_Database VG_SSD00
Logical volume "LV_SSD_Database" created


Format in XFS

# mkfs.xfs /dev/VG_SSD00/LV_SSD_Database
meta-data=/dev/VG_SSD00/LV_SSD_Database isize=256 agcount=4, agsize=13106944 blks
= sectsz=512 attr=2, projid32bit=1
= crc=0
data = bsize=4096 blocks=52427776, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=0
log =internal log bsize=4096 blocks=25599, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0


Found /dev/mapper for mounting:

# ls /dev/mapper/
centos_milliondollorserver--database-root control VG_SSD00-LV_SSD_Database
centos_milliondollorserver--database-swap VG_Backup-LV_Backup


Update /etc/fstab


# cat /etc/fstab
#
# /etc/fstab
# Created by anaconda on Wed Oct 22 20:44:00 2014
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/centos_milliondollorserver--database-root / xfs defaults 1 1
UUID=b6fc0e61-697f-42db-8be7-a6feabc8e85b /boot xfs defaults 1 2
/dev/mapper/centos_milliondollorserver--database-swap swap swap defaults 0 0
/dev/mapper/VG_Backup-LV_Backup /backup xfs defaults 0 0
/dev/mapper/VG_SSD00-LV_SSD_Database /var/ssd xfs defaults 0 0


Mount it:

# mount /var/ssd
# df -k


To check SSD alignment:
Looking at the Start Sector, should be 2048 not 63!


# fdisk -l -u /dev/sdc

Disk /dev/sdc: 214.7 GB, 214748364800 bytes, 419430400 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xab948127

Device Boot Start End Blocks Id System
/dev/sdc1 2048 419430399 209714176 8e Linux LVM

Thursday, January 30, 2014

Oracle Linux: Maximize Performance / Minimize Downtime

Oracle Linux: Maximize Performance / Minimize Downtime
http://medianetwork.oracle.com/video/player/2921003278001

Oracle’s Greg Marsden provides a large number of specific tips and tricks for getting the most from Oracle Linux.
Really helpful information about Large Memory Linux system.
Example for 1TB RAM system.

Key Note:
NUMA
Swap Tuning
Kernel UEK
kdump
Serial console
HugePages
...


Highly Recommend!

Sunday, March 21, 2010

xmarks:Seamlessly synchronize bookmarks across all your computers

Seamlessly synchronize bookmarks across all your computers
http://www.xmarks.com/

Can  synchronize bookmarks between FireFox, IE, Chrome and Safari.


Try it and you will love it.


Other Features:


Smarter Search And Account Password synchronize.

Best Online Backup Solution: mozy.com

Best Online Backup Solution Mozy hold By EMC

Every Database/System/Storage Engineer knows EMC.
And they bought Mozy.com for $76 million.

I would say online storage/ Cloud Storage is the future.
$4.95/month with unlimited space and upload/download coast, there is nothing better than this for my Home Server ( with about 2 TB personal files).

And with EMC's support, I will give it a 5 start from the secure side.

Only thing is there is no Linux client right now.
Hope it coming soon.