Posts tagged lvm

linux ssd and TRIM

posted on 2016-09-12 13:44

One of my setups is an encrypted LVM on top of a SSD.

The important part here is, should you want to TRIM the SSD, then TRIM has to be available on all layers:

  • SSD
  • dm-crypt
  • LVM

Now before you do anything, you might want to jump to the back of this post and to a test with fio for getting a baseline, so you got some numbers to compare for after you changed everything.

check storage media

hdparm, like always, is your friend:

root@ctr-014:~# hdparm -I /dev/sda | grep TRIM
    *   Data Set Management TRIM supported (limit 8 blocks)

If there is nothing to be found, TRIM just won't work for you.

dm-crypt

grep discard /etc/crypttab:

sjas@host:~/blog$ grep discard /etc/crypttab 
sda5_crypt UUID=985f3826-e502-4402-ad01-5c13b84e9141 none luks,discard

If discard is not present, add it there.

LVM

grep issue_discards /etc/lvm/lvm.conf:

root@ctr-014:~# grep issue_discards /etc/lvm/lvm.conf
    #issue_discards = 0
    issue_discards = 1

Set it to "1" if it is not already.

update initramfs

Debian:

update-initramfs -u

RedHat:

dracut -f

reboot

In case you changed anything above, to a restart so your settings become active.

TRIM !

for fs in $(lsblk -o MOUNTPOINT,DISC-MAX,FSTYPE | grep -E '^/.* [1-9]+.* ' | awk '{print $1}'); do fstrim "$fs"; done

Voila!

cron

You might want to put this into a file in /etc/cron.weekly with a #!/bin/sh shebang, to it will be run automatically.

test method

Test were done with fio:

fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=512M --readwrite=randrw --rwmixread=75

Keep in mind these are single-threaded thus somewhat aenemic, but the result should become visible.

after

read iops = 19947
write iops =  6671

before

read iops = 3374 
write iops =  1128

actual measurement sample

root@ctr-014:/home/sjas# fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=64 --size=512M --readwrite=randrw --rwmixread=75
test: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=libaio, iodepth=64
fio-2.1.11
Starting 1 process
test: Laying out IO file(s) (1 file(s) / 512MB)
Jobs: 1 (f=1): [m(1)] [100.0% done] [78716KB/26344KB/0KB /s] [19.7K/6586/0 iops] [eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=3245: Mon Sep 12 13:40:44 2016
  read : io=392888KB, bw=79790KB/s, iops=19947, runt=  4924msec
  write: io=131400KB, bw=26686KB/s, iops=6671, runt=  4924msec
  cpu          : usr=5.77%, sys=33.72%, ctx=71337, majf=0, minf=6
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.1%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued    : total=r=98222/w=32850/d=0, short=r=0/w=0/d=0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: io=392888KB, aggrb=79790KB/s, minb=79790KB/s, maxb=79790KB/s, mint=4924msec, maxt=4924msec
  WRITE: io=131400KB, aggrb=26685KB/s, minb=26685KB/s, maxb=26685KB/s, mint=4924msec, maxt=4924msec

Disk stats (read/write):
    dm-2: ios=95567/32093, merge=0/0, ticks=197872/140256, in_queue=355552, util=97.95%, aggrios=98222/33021, aggrmerge=0/0, aggrticks=203680/161852, aggrin_queue=366328, aggrutil=97.02%
    dm-0: ios=98222/33021, merge=0/0, ticks=203680/161852, in_queue=366328, util=97.02%, aggrios=96973/32744, aggrmerge=1249/277, aggrticks=197660/149212, aggrin_queue=347000, aggrutil=96.94%
  sda: ios=96973/32744, merge=1249/277, ticks=197660/149212, in_queue=347000, util=96.94%

linux: resize vm to full disk size

posted on 2016-06-11 10:38

fter resizing the the virtual harddisk of your virtual machine, several other steps are needed so you can utilize this additional space within the VM. This will only talk about increased sizing, which will usually just work. Unlike with downsizing, which are the same steps just in reverse order, but where you can easily kill your currently still running system. Handle downsizing with very, very much care.

This guide assumes you have a single partition, which is used by LVM, where in you have your filesystem(s) in different logical volumes.

resize the partition

I have a vm with hostname called 'test', which has a single disk (/dev/vda), with a single partition (/dev/vda1), which is used by lvm. LVM volume groups are usually called like the hostname (best approach I know of, so here test), and the logical volumes what they are used for (root, swap), or where they are mounted (i.e. var_lib_vz, not shown here).

root uses ext4 as file system.

Initially the disk size was 50G and was increased to 500G.

After the disk size was increased, you can see the available space on the device:

root@test:~# lsblk
NAME             MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0               11:0    1 1024M  0 rom
vda              253:0    0  500G  0 disk
+-vda1           253:1    0   50G  0 part
  +-test-root 252:0    0 14.3G  0 lvm  /
  +-test-swap 252:1    0  976M  0 lvm  [SWAP]

Use a partition manager of your choice (fdisk or cfdisk for disks with an MBR, gdisk or cgdisk for disks using a GPT, or parted if you know what you are doing.), delete your partition. Recreate it with the maximum size, reboot.

Then it should look like this, with adjusted partition size:

root@test:~# lsblk
NAME             MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0               11:0    1 1024M  0 rom
vda              253:0    0  500G  0 disk
+-vda1           253:1    0  500G  0 part
  +-test-root 252:0    0   49G  0 lvm  /
  +-test-swap 252:1    0  976M  0 lvm  [SWAP]

resize PV, LV, file system

First make LVM format the addition free space: (It will 'partition' it so it can work with it, effectively splitting it into junks of like 4MB if I recall correctly.)

root@test:~# pvresize /dev/vda1
  Physical volume "/dev/vda1" changed
  1 physical volume(s) resized / 0 physical volume(s) not resized

Since the PV was already a member of the VG, no need to extend the VG.

Now for the actual volume:

root@test:~# lvextend -L 499G /dev/test/root
  Size of logical volume test/root changed from 49.04 GiB (12555 extents) to 499.00 GiB (127744 extents).
  Logical volume root successfully resized.

Here I specified it to be resized to 499GB. If I wanted to just use all available space, I'd do:

lvextend -l +100%FREE /dev/mapper/test-root

root@test:~# lvextend -l +100%FREE /dev/mapper/test-root
  Size of logical volume test/root changed from 450.00 GiB (115200 extents) to 499.04 GiB (127755 extents).
  Logical volume root successfully resized.

The -L is just easier to remember.

Lastly, resize the used filesystem:

root@test:~# resize2fs -p /dev/mapper/test-root
resize2fs 1.42.13 (17-May-2015)
Dateisystem bei /dev/mapper/test-root ist auf / eingeh�ngt; Online-Gr��en�nderung ist
erforderlich
old_desc_blocks = 1, new_desc_blocks = 32
Das Dateisystem auf /dev/mapper/test-root is nun 130821120 (4k) Bl�cke lang.

Verify it:

root@test:~# df -h
Dateisystem              Groesse Benutzt Verf. Verw% Eingehaengt auf
udev                      983M       0  983M    0% /dev
tmpfs                     201M    3.2M  197M    2% /run
/dev/mapper/test-root  492G    2.3G  469G    1% /
tmpfs                    1001M       0 1001M    0% /dev/shm
tmpfs                     5.0M       0  5.0M    0% /run/lock
tmpfs                    1001M       0 1001M    0% /sys/fs/cgroup

Linux Software RAID: revisited

posted on 2015-05-13 14:55:51

Having done a linux install based on a software raid and LVM some time ago, with the help of the debian installer, I found out the hard way that booting from it can be possible, but only if the first disk works. Maybe I did something wrong, but I wasn't able to fix the install or find the point where I erred, a reinstall from hand will be a nice learning experience, so here we go.

get a livedisk

To partition the disks manually, you need a livedisk. There are many of these out there, google for the one of your choosing. I ended up using the kali live disk from last time, but have had to manually install mdadm everytime, like described in the last blog post. You need mdadm and the LVM tools for the following.

Usually you will get an .iso file, where dd will help you to put the ISO onto the stick. If for some reason a stick will not work, you might also try burning a CD and run it from there.

boot from the live disk

Depending on your BIOS setup (UEFI booting will no be covered here.), you might have to readjust the boot order, so your system will boot. After having a running OS, open a shell.

overview

TBD

This blog covers .csv, .htaccess, .pfx, .vmx, /etc/crypttab, /etc/network/interfaces, /etc/sudoers, /proc, 10.04, 14.04, AS, ASA, ControlPanel, DS1054Z, GPT, HWR, Hyper-V, IPSEC, KVM, LSI, LVM, LXC, MBR, MTU, MegaCli, PHP, PKI, R, RAID, S.M.A.R.T., SNMP, SSD, SSL, TLS, TRIM, VEEAM, VMware, VServer, VirtualBox, Virtuozzo, XenServer, acpi, adaptec, algorithm, ansible, apache, apachebench, apple, arcconf, arch, architecture, areca, arping, asa, asdm, autoconf, awk, backup, bandit, bar, bash, benchmarking, binding, bitrate, blackarmor, blockdev, blowfish, bochs, bond, bonding, booknotes, bootable, bsd, btrfs, buffer, c-states, cache, caching, ccl, centos, certificate, certtool, cgdisk, cheatsheet, chrome, chroot, cisco, clamav, cli, clp, clush, cluster, coleslaw, colorscheme, common lisp, console, container, containers, controller, cron, cryptsetup, csync2, cu, cups, cygwin, d-states, database, date, db2, dcfldd, dcim, dd, debian, debug, debugger, debugging, decimal, desktop, df, dhclient, dhcp, diff, dig, display manager, dm-crypt, dmesg, dmidecode, dns, docker, dos, drivers, dtrace, dtrace4linux, du, dynamictracing, e2fsck, eBPF, ebook, efi, egrep, emacs, encoding, env, error, ess, esx, esxcli, esxi, ethtool, evil, expect, exportfs, factory reset, factory_reset, factoryreset, fail2ban, fbsd, fdisk, fedora, file, filesystem, find, fio, firewall, firmware, fish, flashrom, forensics, free, freebsd, freedos, fritzbox, fsck, fstrim, ftp, ftps, g-states, gentoo, ghostscript, git, git-filter-branch, github, gitolite, global, gnutls, gradle, grep, grml, grub, grub2, guacamole, hardware, haskell, hdd, hdparm, hellowor, hex, hexdump, history, howto, htop, htpasswd, http, httpd, https, i3, icmp, ifenslave, iftop, iis, imagemagick, imap, imaps, init, innoDB, innodb, inodes, intel, ioncube, ios, iostat, ip, iperf, iphone, ipmi, ipmitool, iproute2, ipsec, iptables, ipv6, irc, irssi, iw, iwconfig, iwlist, iwlwifi, jailbreak, jails, java, javascript, javaws, js, juniper, junit, kali, kde, kemp, kernel, keyremap, kill, kpartx, krypton, lacp, lamp, languages, ldap, ldapsearch, less, leviathan, liero, lightning, links, linux, linuxin3months, lisp, list, livedisk, lmctfy, loadbalancing, locale, log, logrotate, looback, loopback, losetup, lsblk, lsi, lsof, lsusb, lsyncd, luks, lvextend, lvm, lvm2, lvreduce, lxc, lxde, macbook, macro, magento, mailclient, mailing, mailq, manpages, markdown, mbr, mdadm, megacli, micro sd, microsoft, minicom, mkfs, mktemp, mod_pagespeed, mod_proxy, modbus, modprobe, mount, mouse, movement, mpstat, multitasking, myISAM, mysql, mysql 5.7, mysql workbench, mysqlcheck, mysqldump, nagios, nas, nat, nc, netfilter, networking, nfs, nginx, nmap, nocaps, nodejs, numberingsystem, numbers, od, onyx, opcode-cache, openVZ, openlierox, openssl, openvpn, openvswitch, openwrt, oracle linux, org-mode, os, oscilloscope, overview, parallel, parameter expansion, parted, partitioning, passwd, patch, pdf, performance, pfsense, php, php7, phpmyadmin, pi, pidgin, pidstat, pins, pkill, plesk, plugin, posix, postfix, postfixadmin, postgres, postgresql, poudriere, powershell, preview, profiling, prompt, proxmox, ps, puppet, pv, pvecm, pvresize, python, qemu, qemu-img, qm, qmrestore, quicklisp, r, racktables, raid, raspberry pi, raspberrypi, raspbian, rbpi, rdp, redhat, redirect, registry, requirements, resize2fs, rewrite, rewrites, rhel, rigol, roccat, routing, rs0485, rs232, rsync, s-states, s_client, samba, sar, sata, sbcl, scite, scp, screen, scripting, seafile, seagate, security, sed, serial, serial port, setup, sftp, sg300, shell, shopware, shortcuts, showmount, signals, slattach, slip, slow-query-log, smbclient, snmpget, snmpwalk, software RAID, software raid, softwareraid, sophos, spacemacs, spam, specification, speedport, spi, sqlite, squid, ssd, ssh, ssh-add, sshd, ssl, stats, storage, strace, stronswan, su, submodules, subzone, sudo, sudoers, sup, swaks, swap, switch, switching, synaptics, synergy, sysfs, systemd, systemtap, tar, tcpdump, tcsh, tee, telnet, terminal, terminator, testdisk, testing, throughput, tmux, todo, tomcat, top, tput, trafficshaping, ttl, tuning, tunnel, tunneling, typo3, uboot, ubuntu, ubuntu 16.04, udev, uefi, ulimit, uname, unetbootin, unit testing, upstart, uptime, usb, usbstick, utf8, utm, utm 220, ux305, vcs, vgchange, vim, vimdiff, virtualbox, virtualization, visual studio code, vlan, vmstat, vmware, vnc, vncviewer, voltage, vpn, vsphere, vzdump, w, w701, wakeonlan, wargames, web, webdav, weechat, wget, whois, wicd, wifi, windowmanager, windows, wine, wireshark, wpa, wpa_passphrase, wpa_supplicant, x11vnc, x2x, xfce, xfreerdp, xmodem, xterm, xxd, yum, zones, zsh


Unless otherwise credited all material Creative Commons License by sjas