Posts tagged parted

highlevel overview how to change partition sizes

posted on 2016-11-18 18:54

These are some rough notes for colleague of mine, on how to make more swapspace available and resizing partitions in general. The workflow highly depends on the previously existing layout. Here's a shot on a manual on how to approach this.

disclaimer

This is mostly written from memory, so bear with me if you stumble upon errors. No guarantees for nothing below this line.

do you use XFS? or do you REALLY need another swap partition, when you don't have unpartitioned space?

In case you need to resize a partition, as you do not have unpartitioned space available, you cannot enlarge the swap partition or add a second one, if you cannot shrink the filesystem (i know that's the case with XFS) on the partition which you want to shrink. Shrinking partitions is more like deleting the currently available partition and recreating it, only smaller. (Linux lets you do that, even if you make the partition smaller then the filesystem that should be in there, rendering the system unbootable in case you do this. Don't worry, it can be fixed by recreating the old partition schema, so you better backup the information good.)

If the stuff above is the case, you need to create a swapfile and use that. Of course you need enough free space on the filesystem. There should not be a speed difference from what I heard (and honestly I am too lazy to test that), if you have enough free space in your filesystem, create the file with dd, do mkswap it and fix /etc/fstab.

Enough tutorials are on google, this approach is the easiest, hands down.

But let's go on.

how does the system boot: does it use BIOS or UEFI?

  • BIOS -> can work with either a MBR or a GPT
  • UEFI -> needs a GPT, using a MBR won't work

Also UEFI needs a bios boot partition. Basically:

  • first partition is like 300m in size
  • with a fat32 file system
  • has boot and esp flags (sometimes also called bios boot partition)
  • is mounted likely to /boot/efi in your linux installation

The rest is as usual, like you can have a separate boot partition housing the /boot mountpoint, or just using another large partition for / and everything else directly.

how is the partitioning info saved

  • MBR -> 4 primary partitions are maximum, or use the 4th one as an extended partition, which points to further partitioning info somewhere else.

That's also the reason why you might have /dev/sda1, /dev/sda2, /dev/sda5 after a fresh install.

  • sda1 = primary partition
  • sda2 = primary used as extended partition
  • sda5 = first logical partition

The MBR is located on the first sector of a harddisk and 512k in size. During the boot process the executed boot code from the BIOS scans all disk in hope of finding a MBR or GPT. Due to the MBR's structure it can only store the information for four partition entries. Information for partitions of type 'primary' is stored directly in the MBR. Partitions cannot be larger then 2t, if you need that you either have to use a GPT instead or build a logical volume via LVM out of several MBR partitions. (Ok, in that case go for GPT...)

An extended partition points to another partitioning table in a VBR. That's like a MBR, but without boot code and located in the first sector of a partition depicted in the MBR.

  • GPT -> all partitions are created equal (haha), but you need a bios boot partition (see above) so it can work.

You can delete the partitions as you please, and it's autmatically backupped to the end of the disk. Its 33 logical blocks in size (like 33 * 512b or 33 * 4k in disk size, depending on block size), and uses the first 33 and the last 33 blocks of the disk. (In comparison to the MBR, which uses only the first block on a disk.)

Maximum size are about 8 zebibytes or 9 zettabytes, which should do rather fine for the storage needs you have with five nines of probability.

Keep that in mind when you want to use a sofware raid and the raid superblock shall be stored at the end of the disk, depending on the version of the software raid metadata.

backup your partitioning info!!!

Resizing partitions is more or less just deleting a partition and recreating it with a different size. This can fail, rendering the system unbootable when the partition is smaller then the filesystem it shall contain. This can be fixed by deleting the partitioning info for the partition in question, and recreating it bigger again.

Nothing is destroyed here, unless you start recreating filesystems on your newly created partitions, keep that in mind prior to panicking.

Partition info's are just pointers to start and end of a partition, so the kernel knows where to look for filesystems relative to its start.

Also the absolute sizes are important. Best in sectors, bytes do work, too.

Copy the output of the commands below into a text editor and save it somewhere (when working over ssh) or use your smartphone camera to make a picture. Of course pen and paper work, too, but don't do anything without this information backed up. SERIOUSLY!

These will give the partition boundaries in sectors or bytes. I prefer sectors.

parted /dev/sdX u s p

and

parted /dev/sdX u b p

Don't read on unless you did this. If you still do and fuck up, you can try testdisk, but this will not work with more complex setups. From my experience, testdisk only works with like a 60% chance.

highlevel overview for the general approach, the shrink and resize operations

You can only use continuous space for creating new partitions.

I.e. if you have like a 1g swap partition which you want to enlarge, followed by a 100g root partition, you can shrink the root partition, but the new unallocated space will be located at the end of the disk.

If you cannot do that, you need to use LVM.

  • Create new partition in the unallocated space.
  • Create physical volumes on the first partition and the newly-created one.
  • Add both to a volume group.
  • Create a new logical volume using the fully available space in that volume group.
  • Use the new LV as swap.

shrink

This would be the work without LVM being used:

  • Reset, and boot from a livedisk like grml
  • To shrink, start with the innermost part, the filesystem.
  • Shrink filesystem via resize2fs. Either to a particular size, or with the -m flag to the minimum size. This may take time.
  • Delete partition of filesystem you re sized.
  • Recreate partition, but larger than the filesystem. To be on the safe side, create it like 1g bigger than the filesystem, calculating that is annoying due to 1000 vs. 1024 base discrepancies.
  • You may also delete partitions you still don't need anymore. If you do that, fix the /etc/fstab, else no boot for you.
  • Reboot and see if the system still works as you need it to.
  • If it doesn't, look up your backup information from above, and recreate the boot and/or root partitions properly.
  • If it does, create your other partition(s)/logical volume(s) and work on.

In case you have LVM in use:

  • From inner parts to outside, too.
  • First shrink filesystem.
  • Then shrink the logical volume where the filesystem lays on, but not smaller than the filesystem was.
  • Resize the physical volume, too, in case you want to create a new volume group for whatever reason.
  • Adjust partition size if you need to for your desired layout.

Remember, if your system does not boot because you made partition(s) or logical volume(s) too small, that is fixable. But only as long as you did not kill any data on disk, i.e. by creating file systems.

enlarge

  • From outside to inside, basically the reverse from above.
  • Enlarge existing partition.
  • Enlarge physical volume if lvm was used, so also the volume group gets bigger. (pvresize /dev/sdXy will use all available space.)
  • Enlarge logical volumn, if lvm was used. (lvextend -l +100%FREE /dev/mapper/<vg-lv-name> is what you want to use all available space.)
  • Enlarge filesystem. (resize2fs /path/to/device, so either /dev/sdXy without lvm, or /dev/mapper/<vg-lv-name>, to use all available space.)

changing partitions via parted

For editing partitions parted does work quite good, both for MBR/GPT partition tables. fdisk/gdisk also still do exist, if you want something with a fancy curses gui go with cfdisk/cgdisk. Also there are are sfdisk/sgdisk for the hacker types, according to the manpage.

  • "f" -> edit MBR's
  • "g" -> edit GPT's

parted commands cause immediate changes, whereas the others let you view your changes, but won't change anything until you write the changes to disk.

I really prefer using parted non-interactively nowadays, though I cannot explain why.

All commands in as short as possible:

# show help
parted /dev/sdX h
# show help on particular command, may help greatly
parted /dev/sda h <parted_command>

# drop partition info
parted /dev/sdX u s p       # "unit sector print"
parted /dev/sdX u s p free  # "unit sector print free"
parted /dev/sdX u b p       # "unit byte print"

# create new disklabel, read: MBR or GPT.
# if you do this you basically delete the complete partitioning table
# do only if you need to, and backupped the 'print' output above!
parted /dev/sdX mkl msdos  ## create MBR
parted /dev/sdX mkl gpt    ## create GPT

# delete partition
parted /dev/sdX rm <ID>  # 1 or 2 or 3, depending which partition you want to edit

# create partition
# -a opt can be used with all commands listed here, but only has impact here
# units can be mixedly used, like 2048s, 10GiB, 10GB, 100%
parted -a opt /dev/sdX mkp  # mkpart, -a opt is essential for optimal alignment!

# show options
parted /dev/sdX h set
# enable/disable options (like boot flag)
parted /dev/sdX set <ID> <OPTION> on   # enable
parted /dev/sdX set <ID> <OPTION> off  # disable

swap

If you can still boot, and have a shiny new partition (or logical volume) which you can use, finish:

  • mkswap /path/to/device
  • fix /etc/fstab, i.e. create an entry so the system knows about the swapspace

This should be everything one may encounter. Good luck.

luks and lvm and partitioning and filesystem from the shell

posted on 2016-08-24 20:58

Don't overwrite your devices via cp. But we've all been there, done that.

If you don't want to reinstall 'just because', an idea might be to use testdisk depending on what you did.

Getting nice partition layout I tend to use parted (see below), for creating partitions cgdisk (for GPT stuff) or cfdisk (for MBR creation only IIRC) are decent choices.

Back on topic.

preparation

Partitions were still present in my cause, so no need create them anew.

If you have to, do parted /dev/sda p and parted /dev/sdX u b p and use your phone to make photos, in case you have to redo something.

luks

Create and open the cryptocontainer to hold the complete partition, wherein the LVM and your filesystems will lie.

cryptsetup --cipher=aes-xts-plain64 luksFormat /dev/sdXN --force-password
cryptsetup open /dev/sdXN sdXN_crypt

Did you really type an uppercased YES when you were promted? The password you were prompted for is the one you will have to enter in the future.

In case you did something wrong:

cryptsetup close
cryptsetup erase /dev/sdaX

Then start by recreating the container. Did you really type an uppercased YES when you were promted?

lvm2

After the crypto device was opened, you can reference it through the device mapper. Now create the physical volume (PV), volume group (VG) and logical volumes (LV's) where your system will be installed later on:

pvcreate /dev/mapper/sdXN_crypt
vgcreate `hostname` /dev/mapper/sdXN_crypt
lvcreate -L 2G -n swap `hostname`
lvcreate -l 100%FREE -n root `hostname`

Here is a catch: I did not have to recreate a separate /boot partition, as I already had one. If you don't create one first. It has to be located outside the crypto container, else you won't be able too boot after your installation.

If something went wrong, here's how to delete things, too. Choose what you need in particular:

pvremove /dev/sdXN_crypt
vgremove `hostname`
lvremove /dev/`hostname`/<LVname>

filesystems and swap

Create swap:

mkswap /dev/mapper/`hostname`-swap

Create root filesystem:

mkfs -t ext4 /dev/mapper/`hostname`-root

This is pretty much it. From here on you can chroot or do whatever else you want.

Maybe you only want the container for data but for installing a system on there. In that case not calling the LV 'root' and omitting the swap partition up there would have been a wise choice.

Linux Software RAID: revisited

posted on 2015-05-13 14:55:51

Having done a linux install based on a software raid and LVM some time ago, with the help of the debian installer, I found out the hard way that booting from it can be possible, but only if the first disk works. Maybe I did something wrong, but I wasn't able to fix the install or find the point where I erred, a reinstall from hand will be a nice learning experience, so here we go.

get a livedisk

To partition the disks manually, you need a livedisk. There are many of these out there, google for the one of your choosing. I ended up using the kali live disk from last time, but have had to manually install mdadm everytime, like described in the last blog post. You need mdadm and the LVM tools for the following.

Usually you will get an .iso file, where dd will help you to put the ISO onto the stick. If for some reason a stick will not work, you might also try burning a CD and run it from there.

boot from the live disk

Depending on your BIOS setup (UEFI booting will no be covered here.), you might have to readjust the boot order, so your system will boot. After having a running OS, open a shell.

overview

TBD

parted

posted on 2014-10-11 15:27:56

Sometimes you have to partition disks by hand. For some people, parted is the weapon of choice.

If so, keep in mind you only use it for partitioning. Filesystems are not implemented and parted may tell you so, too. Besides you won't get filesystems created by it. Just use the mkfs.xxx tools at hand. It's just sad that half finished programs are distributed... banana software - it matures in the works of the customer. :(

Back to the posts intention: There are two ways to use parted. Either via the interactive parted-shell, or directly from the commandline.

Most needed commands in the shell might be:

a   = align-check = check alignment to sectors (min or opt as params)
p   = print       = show info for chosen disk
sel = select      = choose disk (i.e. /dev/sdb)
u   = unit        = measuring of sizes (i.e. %,MB,GB,...)
mkl = mklabel     = create disk label (i.e. gpt) (mktable = mkt = the same)
mkp = mkpart      = create partitions
rm  = remove      = delete a partition
q   = quit        = exit parted

An overview on the possible sizes you'd most likely use:

MiB  Mebibyte (1048576 bytes)
GiB  Gibibyte (1073741824 bytes)
MB   Megabyte (1000000 bytes)
GB   Gigabyte (1000000000 bytes)
%    procentual (between 0% and 100%) 
s    sectors (logical sector size)

So usually you would want MiB and GiB, I guess.

Sadly, I haven't found out yet how to optimally align the partition onto the harddisks (so it matches it's sectors best). For this I actually used to use parted from the regular bash shell:

parted /dev/sdb mklabel gpt
parted -a optimal /dev/sdb unit gb mkpart primary ext4 2 100%

Nowadays I mostly use parted interactive with procentual parameters.

But for when I don't, here two example workflows

# create a single MBR  partition with an ext4 on the usb stick /dev/sdb
lsblk -f                ## make sure the stick is really on /dev/sdb
parted /dev/sdb p           ## check whats already on the stick
parted /dev/sdb mkl ms y        ## create MBR not GPT
parted /dev/sdb mkp p 0% 100%   ## single primary partition, percentages are used for proper alignment
mkfs.ext4 /dev/sdb1         ## create filesystem at last
parted /dev/sdb a opt 1     ## check alignment to be optimal for partition 1

# create the same, but with gpt
lsblk -f
parted /dev/sdb p           
parted /dev/sdb mkl g y     ## create GPT
parted /dev/sdb mkp asdf 0% 100%    ## creates partition with name 'asdf', a name must be specified, ext4 already on it

It seems that currently creating the filesystem only works when using GPT's, and only ext4's can be created. But somewhere I heard that it was officially said that you should use mkfs anyway, and not let parted create filesystems.

This blog covers .csv, .htaccess, .pfx, .vmx, /etc/crypttab, /etc/network/interfaces, /etc/sudoers, /proc, 10.04, 14.04, AS, ASA, ControlPanel, DS1054Z, GPT, HWR, Hyper-V, IPSEC, KVM, LSI, LVM, LXC, MBR, MTU, MegaCli, PHP, PKI, R, RAID, S.M.A.R.T., SNMP, SSD, SSL, TLS, TRIM, VEEAM, VMware, VServer, VirtualBox, Virtuozzo, XenServer, acpi, adaptec, algorithm, ansible, apache, apachebench, apple, arcconf, arch, architecture, areca, arping, asa, asdm, autoconf, awk, backup, bandit, bar, bash, benchmarking, binding, bitrate, blackarmor, blockdev, blowfish, bochs, bond, bonding, booknotes, bootable, bsd, btrfs, buffer, c-states, cache, caching, ccl, centos, certificate, certtool, cgdisk, cheatsheet, chrome, chroot, cisco, clamav, cli, clp, clush, cluster, coleslaw, colorscheme, common lisp, console, container, containers, controller, cron, cryptsetup, csync2, cu, cups, cygwin, d-states, database, date, db2, dcfldd, dcim, dd, debian, debug, debugger, debugging, decimal, desktop, df, dhclient, dhcp, diff, dig, display manager, dm-crypt, dmesg, dmidecode, dns, docker, dos, drivers, dtrace, dtrace4linux, du, dynamictracing, e2fsck, eBPF, ebook, efi, egrep, emacs, encoding, env, error, ess, esx, esxcli, esxi, ethtool, evil, expect, exportfs, factory reset, factory_reset, factoryreset, fail2ban, fbsd, fdisk, fedora, file, filesystem, find, fio, firewall, firmware, fish, flashrom, forensics, free, freebsd, freedos, fritzbox, fsck, fstrim, ftp, ftps, g-states, gentoo, ghostscript, git, git-filter-branch, github, gitolite, global, gnutls, gradle, grep, grml, grub, grub2, guacamole, hardware, haskell, hdd, hdparm, hellowor, hex, hexdump, history, howto, htop, htpasswd, http, httpd, https, i3, icmp, ifenslave, iftop, iis, imagemagick, imap, imaps, init, innoDB, innodb, inodes, intel, ioncube, ios, iostat, ip, iperf, iphone, ipmi, ipmitool, iproute2, ipsec, iptables, ipv6, irc, irssi, iw, iwconfig, iwlist, iwlwifi, jailbreak, jails, java, javascript, javaws, js, juniper, junit, kali, kde, kemp, kernel, keyremap, kill, kpartx, krypton, lacp, lamp, languages, ldap, ldapsearch, less, leviathan, liero, lightning, links, linux, linuxin3months, lisp, list, livedisk, lmctfy, loadbalancing, locale, log, logrotate, looback, loopback, losetup, lsblk, lsi, lsof, lsusb, lsyncd, luks, lvextend, lvm, lvm2, lvreduce, lxc, lxde, macbook, macro, magento, mailclient, mailing, mailq, manpages, markdown, mbr, mdadm, megacli, micro sd, microsoft, minicom, mkfs, mktemp, mod_pagespeed, mod_proxy, modbus, modprobe, mount, mouse, movement, mpstat, multitasking, myISAM, mysql, mysql 5.7, mysql workbench, mysqlcheck, mysqldump, nagios, nas, nat, nc, netfilter, networking, nfs, nginx, nmap, nocaps, nodejs, numberingsystem, numbers, od, onyx, opcode-cache, openVZ, openlierox, openssl, openvpn, openvswitch, openwrt, oracle linux, org-mode, os, oscilloscope, overview, parallel, parameter expansion, parted, partitioning, passwd, patch, pdf, performance, pfsense, php, php7, phpmyadmin, pi, pidgin, pidstat, pins, pkill, plesk, plugin, posix, postfix, postfixadmin, postgres, postgresql, poudriere, powershell, preview, profiling, prompt, proxmox, ps, puppet, pv, pvecm, pvresize, python, qemu, qemu-img, qm, qmrestore, quicklisp, r, racktables, raid, raspberry pi, raspberrypi, raspbian, rbpi, rdp, redhat, redirect, registry, requirements, resize2fs, rewrite, rewrites, rhel, rigol, roccat, routing, rs0485, rs232, rsync, s-states, s_client, samba, sar, sata, sbcl, scite, scp, screen, scripting, seafile, seagate, security, sed, serial, serial port, setup, sftp, sg300, shell, shopware, shortcuts, showmount, signals, slattach, slip, slow-query-log, smbclient, snmpget, snmpwalk, software RAID, software raid, softwareraid, sophos, spacemacs, spam, specification, speedport, spi, sqlite, squid, ssd, ssh, ssh-add, sshd, ssl, stats, storage, strace, stronswan, su, submodules, subzone, sudo, sudoers, sup, swaks, swap, switch, switching, synaptics, synergy, sysfs, systemd, systemtap, tar, tcpdump, tcsh, tee, telnet, terminal, terminator, testdisk, testing, throughput, tmux, todo, tomcat, top, tput, trafficshaping, ttl, tuning, tunnel, tunneling, typo3, uboot, ubuntu, ubuntu 16.04, udev, uefi, ulimit, uname, unetbootin, unit testing, upstart, uptime, usb, usbstick, utf8, utm, utm 220, ux305, vcs, vgchange, vim, vimdiff, virtualbox, virtualization, visual studio code, vlan, vmstat, vmware, vnc, vncviewer, voltage, vpn, vsphere, vzdump, w, w701, wakeonlan, wargames, web, webdav, weechat, wget, whois, wicd, wifi, windowmanager, windows, wine, wireshark, wpa, wpa_passphrase, wpa_supplicant, x11vnc, x2x, xfce, xfreerdp, xmodem, xterm, xxd, yum, zones, zsh


Unless otherwise credited all material Creative Commons License by sjas