Recent Posts

On blockdevices and their blocksizes

posted on 2017-04-27 06:00

Disclaimer: This is a draft and waiting to be verified once I get around plugging all old disks in. As of now, it mainly tries to specify the problem of disk sector size being report differently by linux on OS level and from hdparm via the firmware, and wether dd can really specify arbitrary blocksizes.

Getting it from the kernel via /sys/dev/, lsblk, or fdisk did show different sizes from hdparm when tested.

It's the only seemingly correct way to determine the sector sizes, since it queries the firmware on disk directly. Getting it from the kernel via /sys/dev/ (lsblk or fdisk should use that same info) did show different sizes from hdparm -I when tested.

hdparm sports -i and -I:

-i     Display the identification info which the kernel drivers (IDE, libata) have stored from boot/configā€
       uration time.  This may differ from the current  information  obtainable  directly  from  the  drive
       itself with the -I option.  The data returned may or may not be current, depending on activity since
       booting the system.  For a more detailed interpretation of the  identification  info,  refer  to  AT
       Attachment  Interface  for Disk Drives, ANSI ASC X3T9.2 working draft, revision 4a, April 19/93, and
       later editions.

- I     Request identification info directly from the drive, which is displayed in  a  new  expanded  format
        with considerably more detail than with the older -i option.

An overview on the information hdparm gets, can be found here. I did not find the actual block size in there, so my guess is the sizes are calculated from the information found in the drives registers.

-i shows pretty much a dump of the firware registers it seems, while -I seems to do additional calculations to present more information in human-friendly form.

To overwrite the last 1mb of a disk via dd, on a disk with 512b sectors this will do:

dd bs=512 if=/dev/zero of=/dev/sda count=2048 seek=$((`blockdev --getsz /dev/sda` - 2048))

bs is in bytes, count and seek are blockcounts.

Questions for the future are:

  • does this work reliably with devices having 4k sectors?
  • what does blockdev --getbsz print in these cases?
  • does it use the size reported by the kernel?
  • do mismatches come from different physical/logical blocksizes reported by the disk?
  • were the mismatches I witnessed really in between disk and OS, or just within the firmware?

Mind you, the numbers I saw earlier (2 years?) were with a dozen directly connected IDE and SATA drives, not with a raid controller in between, on disks of differenent manufacturers and sizes I had left over, and some showed in-kernel 512b even though they were 4k ones.

This really intrigues me, since the linux tools all seem to 'just work'. But using dd in the past I ran into problems were I simply could not manually find data during forensic work on disks where its offset was calculated to be.

gnu global

posted on 2017-04-26 20:56

Get the current version from here. Alternatively, use the use the anonymous access to CVS, but no Idea wether the patches would work there.

The steps to version 6.5.6 running on a current gentoo install were, including the patches follow. I hope I don't forget something, this write was done a week later.

needed packages

emerge -atv these:

  • dev-python/pygments
  • dev-util/ctags
  • sys-devel/automake
  • sys-devel/m4

getting and compiling

mkdir asdf
cd asdf
curl -O http://tamacom.com/global/global-6.5.6.tar.gz
tar xzvf global-6.5.6.tar.gz
cd global-6.5.6

applying these two patches:

diff --git a/configure.ac b/configure.ac
index cda52e4..d7d1a73 100644
--- a/configure.ac
+++ b/configure.ac
@@ -24,6 +24,7 @@ dnl
 AC_INIT([GNU GLOBAL],[6.5.6])
 AC_DEFINE(COPYRIGHT_YEAR,["1996-2016"],[Copyright Year])
 AC_CONFIG_SRCDIR(global/global.c)
+AC_CONFIG_MACRO_DIRS([m4])
 AC_CONFIG_HEADERS([config.h:config-h.in])
 AC_PREREQ(2.59)
 AM_INIT_AUTOMAKE([1.9.3 gnu subdir-objects])


diff --git a/Makefile.am b/Makefile.am
index 43585a6..f6ab5c5 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -29,6 +29,8 @@ gtags_DATA = AUTHORS COPYING COPYING.LIB ChangeLog FAQ INSTALL LICENSE NEWS READ

 EXTRA_DIST = ${gtags_DATA} reconf.sh convert.pl mainpage.dox head.in

+ACLOCAL_AMFLAGS = -I m4
+
 doxygen:
    GTAGSCONF=':htags_options=--insert-header=head.in --suggest:' doxygen

Then back to the shell:

autoreconf -fvi
./configure --with-universal-ctags=`which ctags`
make
make check
sudo make install

setup and test

sudo updatedb
sudo locate gtags.conf
cp -va /usr/local/share/gtags/gtags.conf ~/.globalrc
gtags --version
global --version
cd ../..
rm -rf asdf

usage

Now onto actual usage:

CLI:

gtags -v        # generate tag database (initially, rerun with -i everytime code is changed!)

global FUNC     # finds file where FUNC is defined, regexp's work, too
global -r FUNC      # find files where FUNC is used in
global -x FUNC      # == ctags -x, name+line+path+line_content

Of course you only want to look at source, not generated code. If git is used, doing this from the project root will do the trick:

git ls-files > gtags.files

There's more to it, but it should be sufficiant for now. Usually global should be integrated into your dev environment of choice.

Out of the box, less should work. vim and emacs need plugins to work, they won't be covered at this time here.

less:

export LESSGLOBALTAGS=global  ## this is needed so less knows it can work with global
less -t FUNC        # opens file where FUNC is defined

installing quicklisp

posted on 2017-04-22 20:48

Each lisp install can be considered as a variable image. To add quicklisp for proper library management, there are some simple steps needed.

This is done on a steel bank common lisp install.

curl -O https://beta.quicklisp.org/quicklisp.lisp
sbcl --load quicklisp.lisp

* (quicklisp-quickstart:install)
* (ql:add-to-init-file)
* (sb-ext:quit)

rm quicklisp.lisp

openvpn dns pushed on linux 2

posted on 2017-04-15 15:08

Last time I got this to work on debian (see here), now it was time to port this stuff to gentoo (there was more to it, this just to document that openvpn part).

needed packages

emerge -atv net-vpn/openvpn
emerge -atv net-dns/openresolv

resolvconf packages are deprecated it seems, openresolv is the new hotness.

openvpn config

These to lines need to be present in your .ovpn file at the end:

up  /etc/openvpn/up.sh
down    /etc/openvpn/down.sh

This differs from /etc/openvpn/update-resolv-conf in debian.

openvpn cannot find /dev/net/tun

Simple solution, just create the device if its not already present:

mkdir /dev/net
mknod /dev/net/tun c 10 200

Afterwards everything should work as expected, including the updated dns settings. Use watch -n1 -d cat /etc/resolv.conf to see things changing upon establishing and terminating the tunnel.

make it stick

After every reboot you'd have to recreate the /dev/net folder and device node. To let this happen automatically, create a file called /etc/local.d/create_tun_device.start containing:

#!/bin/bash
[ ! -d /dev/net ] && mkdir /dev/net
[ ! -e /dev/net/tun ] && mknod /dev/net/tun c 10 200

mysql increase redo log size

posted on 2017-03-03 18:46

For tuning mysql innodb-wise it helps to allocate enough space in ram for the tables via the buffer pool. For this innodb_buffer_pool_size is to be increased as much as possible and needed, depending on your database size. mysqltuner may help discerning the minimum size, along with other hints.

Besides that the redo log has to be adjusted to about 1/6 of the buffer pool via innodb_log_file_size.

To fix this, there are more steps needed, else the mysql instance will not start properly after the next restart.

  1. start mysql client
  2. SET GLOBAL innodb_fast_shutdown=0
  3. \q
  4. service mysql stop
  5. fix both mentioned config vars, best in /etc/mysql/conf.d/zz_yourconfig.cnf
  6. rm /var/lib/mysql/ib_logfile*
  7. service mysql start

gmrl boot issues

posted on 2017-02-17 07:52

From time to time, depending on your hardware, grml might have startup issues.

If it hangs on udev discovery, it tells you about the noudev boot flag. If you use that and it hangs later on, try mgag200.modeset=0.

Sometimes nomodeset (disabling the framebuffer) helps, too.

If it boots almost completely through, you might try pressing d or u, depending on the language you want to use, followed by x so it boots the xserver. This sometimes works even if the text console does not.

kemp loadmaster password and factory reset

posted on 2017-02-15 10:46

Connect via serial console, and login:

user: pwreset
pass: 1pwreset

The password will be reset to 1fourall.

Note this is only a temporary password, which will be reset to the old password after the next reboot. Use the ncurses menu to set the password to a proper new one.

The factory reset is slightly hidden and can be found under 'software update'.

make scite show all files in open dialogue

posted on 2017-01-27 16:38

By default ScITE has the annoying setting, that it will only show 'all source files' in the open file dialog.

To change this behaviour, open scite as root user (or use sudo), then:

Options >> Open Global Options File
ctrl-f
button bottom right 'match whole words only'

to find something like this:

# Each platform has a different idea of the most important filters
if PLAT_WIN
        all.files=All Files (*.*)|*.*|
        top.filters=All Source|$(source.files)|$(all.files)
if PLAT_GTK
        all.files=All Files (*)|*|Hidden Files (.*)|.*|
        top.filters=All Source|$(source.files)|$(all.files)
# As OS X only greys out filtered files, show all as default
if PLAT_MAC
        all.files=All Files (*.*)|*.*|
        top.filters=$(all.files)All Source|$(source.files)|

Change to:

# Each platform has a different idea of the most important filters
if PLAT_WIN
        all.files=All Files (*.*)|*.*|
        top.filters=$(all.files)
if PLAT_GTK
        all.files=All Files (*)|*|Hidden Files (.*)|.*|
        top.filters=$(all.files)
# As OS X only greys out filtered files, show all as default
if PLAT_MAC
        all.files=All Files (*.*)|*.*|
        top.filters=$(all.files)

Save, quit, restart, works.

powershell ip reverse resolution

posted on 2017-01-27 15:22

A quick script to do reverse resolution of a textfile containing ips called ips.txt located in the same directory as the the file you put this content into:

$erroractionpreference = 'silentlycontinue'
get-content .\ips.txt | foreach-object {
        $resolvedip = [System.Net.Dns]::gethostentry($_).hostname
        echo "$_        : $resolvedip"
}

Save and execute. Might have some rough edges, this did not get much testing.

apt cheatsheet

posted on 2017-01-23 21:31

As short as possible:

apt-cache search = search for package (old)
apt-cache show = show package information
dpkg -l = show installed packages
dpkg -L = show package contents
dpkg -S = search packages for file
apt-get install = install package (old)
apt-get remove = uninstall package, leave configuration on disk (old)
apt-get purge = uninstall package, delete configs (old)

apt search = (new)
apt install = (new)
apt remove = (new)
apt purge = (new)

This blog covers .csv, .htaccess, .pfx, .vmx, /etc/crypttab, /etc/network/interfaces, /etc/sudoers, /proc, 10.04, 14.04, AS, ASA, ControlPanel, DS1054Z, GPT, HWR, Hyper-V, IPSEC, KVM, LSI, LVM, LXC, MBR, MTU, MegaCli, PHP, PKI, R, RAID, S.M.A.R.T., SNMP, SSD, SSL, TLS, TRIM, VEEAM, VMware, VServer, VirtualBox, Virtuozzo, XenServer, acpi, adaptec, algorithm, ansible, apache, apachebench, apple, arcconf, arch, architecture, areca, arping, asa, asdm, autoconf, awk, backup, bandit, bar, bash, benchmarking, binding, bitrate, blackarmor, blockdev, blowfish, bochs, bond, bonding, booknotes, bootable, bsd, btrfs, buffer, c-states, cache, caching, ccl, centos, certificate, certtool, cgdisk, cheatsheet, chrome, chroot, cisco, clamav, cli, clp, clush, cluster, coleslaw, colorscheme, common lisp, console, container, containers, controller, cron, cryptsetup, csync2, cu, cups, cygwin, d-states, database, date, db2, dcfldd, dcim, dd, debian, debug, debugger, debugging, decimal, desktop, df, dhclient, dhcp, diff, dig, display manager, dm-crypt, dmesg, dmidecode, dns, docker, dos, drivers, dtrace, dtrace4linux, du, dynamictracing, e2fsck, eBPF, ebook, efi, egrep, emacs, encoding, env, error, ess, esx, esxcli, esxi, ethtool, evil, expect, exportfs, factory reset, factory_reset, factoryreset, fail2ban, fbsd, fdisk, fedora, file, filesystem, find, fio, firewall, firmware, fish, flashrom, forensics, free, freebsd, freedos, fritzbox, fsck, fstrim, ftp, ftps, g-states, gentoo, ghostscript, git, git-filter-branch, github, gitolite, global, gnutls, gradle, grep, grml, grub, grub2, guacamole, hardware, haskell, hdd, hdparm, hellowor, hex, hexdump, history, howto, htop, htpasswd, http, httpd, https, i3, icmp, ifenslave, iftop, iis, imagemagick, imap, imaps, init, innoDB, innodb, inodes, intel, ioncube, ios, iostat, ip, iperf, iphone, ipmi, ipmitool, iproute2, ipsec, iptables, ipv6, irc, irssi, iw, iwconfig, iwlist, iwlwifi, jailbreak, jails, java, javascript, javaws, js, juniper, junit, kali, kde, kemp, kernel, keyremap, kill, kpartx, krypton, lacp, lamp, languages, ldap, ldapsearch, less, leviathan, liero, lightning, links, linux, linuxin3months, lisp, list, livedisk, lmctfy, loadbalancing, locale, log, logrotate, looback, loopback, losetup, lsblk, lsi, lsof, lsusb, lsyncd, luks, lvextend, lvm, lvm2, lvreduce, lxc, lxde, macbook, macro, magento, mailclient, mailing, mailq, manpages, markdown, mbr, mdadm, megacli, micro sd, microsoft, minicom, mkfs, mktemp, mod_pagespeed, mod_proxy, modbus, modprobe, mount, mouse, movement, mpstat, multitasking, myISAM, mysql, mysql 5.7, mysql workbench, mysqlcheck, mysqldump, nagios, nas, nat, nc, netfilter, networking, nfs, nginx, nmap, nocaps, nodejs, numberingsystem, numbers, od, onyx, opcode-cache, openVZ, openlierox, openssl, openvpn, openvswitch, openwrt, oracle linux, org-mode, os, oscilloscope, overview, parallel, parameter expansion, parted, partitioning, passwd, patch, pdf, performance, pfsense, php, php7, phpmyadmin, pi, pidgin, pidstat, pins, pkill, plesk, plugin, posix, postfix, postfixadmin, postgres, postgresql, poudriere, powershell, preview, profiling, prompt, proxmox, ps, puppet, pv, pvecm, pvresize, python, qemu, qemu-img, qm, qmrestore, quicklisp, r, racktables, raid, raspberry pi, raspberrypi, raspbian, rbpi, rdp, redhat, redirect, registry, requirements, resize2fs, rewrite, rewrites, rhel, rigol, roccat, routing, rs0485, rs232, rsync, s-states, s_client, samba, sar, sata, sbcl, scite, scp, screen, scripting, seafile, seagate, security, sed, serial, serial port, setup, sftp, sg300, shell, shopware, shortcuts, showmount, signals, slattach, slip, slow-query-log, smbclient, snmpget, snmpwalk, software RAID, software raid, softwareraid, sophos, spacemacs, spam, specification, speedport, spi, sqlite, squid, ssd, ssh, ssh-add, sshd, ssl, stats, storage, strace, stronswan, su, submodules, subzone, sudo, sudoers, sup, swaks, swap, switch, switching, synaptics, synergy, sysfs, systemd, systemtap, tar, tcpdump, tcsh, tee, telnet, terminal, terminator, testdisk, testing, throughput, tmux, todo, tomcat, top, tput, trafficshaping, ttl, tuning, tunnel, tunneling, typo3, uboot, ubuntu, ubuntu 16.04, udev, uefi, ulimit, uname, unetbootin, unit testing, upstart, uptime, usb, usbstick, utf8, utm, utm 220, ux305, vcs, vgchange, vim, vimdiff, virtualbox, virtualization, visual studio code, vlan, vmstat, vmware, vnc, vncviewer, voltage, vpn, vsphere, vzdump, w, w701, wakeonlan, wargames, web, webdav, weechat, wget, whois, wicd, wifi, windowmanager, windows, wine, wireshark, wpa, wpa_passphrase, wpa_supplicant, x11vnc, x2x, xfce, xfreerdp, xmodem, xterm, xxd, yum, zones, zsh


Unless otherwise credited all material Creative Commons License by sjas