Posts from 2015-08

Linux: create bootable DOS on USB stick

posted on 2015-08-31 23:29:13

create

When needing a bootable DOS installation (which is something you should rarely, if ever, need), do you need windows to create it?

unetbootin will help.

  1. wipe the stick, and create a single partition with a FAT filesystem on it.
  2. install unetbootin
  3. install FreeDos by using it.

use

Once you put the stick into your computer and reboot, you should be provided with a menu where you can choose different DOS options.

When one is chosen, you should end up with a A:\> prompt and be ready to go.

If you need additional files, just throw them to the file root of the stick.

troubleshoot

If things do not work as planned, here are some hints as why they might refrain from working properly:

  1. Mainboard is set to UEFI or is UEFI-only: you cannot boot a bios stick then. Go to setup and fix it.
  2. Boot order is wrong: Either choose the boot menu option (if possible), or fix the boot order in BIOS options.
  3. No files? - Try using B: or C: to change the drive. For me, files were to be found at C:\>.

Plesk 12 database overview

posted on 2015-08-11 18:40:06

Ever felt the need to dig deep into plesk data model?

Starting with 12.5, there will be some official documentation on this, until it's done see here.

Battling a sophos cluster and on appliances in general

posted on 2015-08-09 11:56:27

An anectdote on appliances...

Recently a sophos UTM cluster of a client died, consisting of two of these. Simply the web interface stopped working, and nothing was reachable anymore. Of course he only had clustered systems, to prevent exactly this scenario. The cause?

A NIC on one of the boxes died. Sadly the one for clustering both devices, the direct connection. The symptoms?

  • Downtime.
  • No accessible web interface.
  • Looks like both appliances play ping-pong and have a split-brain situation where they cannot settle who gets to be the master.

Getting a single application to boot and run, an all works in order again. But in the post-mortem analysis of the device nothing could be found in the web interface. All NIC's were present.

In the shell (accessed with SSH) an interface, eth3 to be exact, can be seen missing... What could that just mean?

So the vendor told 'to flash the device with current firmware release'. This did work. Rebuilding the cluster did work, too. When trying to set up link aggregation (bonding 2 NIC's, so one port can die) for future prevention, the cluster did not work reliably again.

Removed the second box again. Flashing it did not work.

RMA again, exchange it with a new box from the vendor, it works.

Usually this hardware system ist set up for the most convenient use, configuring these is a breeze (setup the first box, connect clusterconnection on eth3, wait until sync is finished, plug the other ethernet cables on), but link aggregation has to be configured by hand from scratch, which is where you will need your own solid knowledge again.

This has nothing to do with especially sophos appliances, the problems are the same for everything else, in different grades. Standards work, rest often doesn't.

And as sophisticated these things are, as inexperienced is sometimes the support technician you have on the other end of the phone. Be it from the vendor directly or from a so-called 'gold partner'. All this does not help you anything at all.

The problems come with core firewalls (F5), vpn appliances (I am looking at you, Cisco ASA cluster!!!), switches (Hi, Juniper!).

Often you have no real symptoms besides the to hardest issues to ever debug:

  • it does not work reliably
  • it is slow

The problems and the root causes for them are all understandable and fine, easily so.

But as I said, it does not help you at all with your day to day work.

And it makes you question the use and return of investment of some of these purchases, especially when you are told to wait for the next hotfix, as 'we are working on your problem' and your network goes down sporadically until then.

Whitebox hardware is the latest craze, why not build the smaller stuff yourself?

NAS boxes or firewalls are often just linux based, junipers run on a FreeBSD variant. Ponder a little on these facts, and stop pulling your hair out when things break.

How to learn linux in three months - 1

posted on 2015-08-09 10:08:14

how to learn linux in three months

It just so happened that a colleague need to get serious linux skills for a new job... fast. Timeline is like three months.

What to do about that?

I have used linux for years, what have things were the most important during all this time?

This is the problem at hand, and there might be a series of posts as a rant and some kind of exploratory research on what I'd learn in what order with all my knowledge today, if I had to relearn things from scratch.

The underlying theme is to get the basics right and literally all the rest will follow.

Oh, and forget what university told you, this is practice, baby. University is worth jack, you know less than an apprentice at an ISP once you are finished, no matter what your master's average was.

a little more differentiation

What are essential skills in 2015? How about these as a braindump, importance-wise roughly sorted in that order:

  • linux and operating systems knowledge WRT booting the system up, process and user management
  • storage knowledge essentials, from hardware over partitioning to filesystems, even covering RAID's and (logical) volume management
  • ipv4 and networks, especially switching / routing, dns, vpn
  • server virtualization technologies

This is what I'd consider essential as of this moment, likely this is not perfect. :)

To learn things, knowing how to use a virtualization technology like virtual box is key, but from the start you will not know what you are doing anyway. Get a grip on the other things first, you will see it is way easier after the basics.

All in all this will give a solid base to start from, and give a proper perspective for working with 'this so-called internet'.

the ideas in more depth

linux, operating systems, running things

For real learning purposes and the best bang-for-buck, don't bother with windows.

Seriously. Why?

unix systems are everywhere

Windows may be a neccessity as you will have to have to work with it, but in the long run you will get much more out of unix-based knowledge. It is used in appliances, switches, stable servers, 'real' clusters. There are things you just cannot build with microsoft stuff directly, from what my windows colleagues explained to me.

Read that? You. Simply. Cannot.

If you doubt that, build a real multinode cluster without paying your ass off for some crappy VMWare stuff where the licensing fees will eat your lunch, revenue-wise.

Using linux as a base to get this knowledge is the easiest way, just know there are the BSD's and other unices out there, too.

linux vs. ENTERPRISE competitioners

Even if you have the money for VMware licenses, it's 'enterprise'.

"ENTERPRISE, FUCK YEAH."

This translates to 'build a product, get market share, earn $$$ for crappy software and use your vendor-lock-in to quench as much money out of your loyal customers as you can'. All the while using these customers as guinea pigs, as software development is expensive and rigorous testing cannot be covered by your budget.

Using open source software I get the absolute same results without paying shitloads of money. This is why redhat thrives on being successful with just providing support, even when CentOS serves the same functionality as RHEL.

appliances vs. self-built systems

Also having an accessible operating system at hand for your hypervisor or 'appliance' ('appliance' is a swearword for me, TBH) instead of some crappy busybox is immensely useful. Don't let you tell otherwise from the naysayers.

But uptime requirements become more harsh nowadays, and cluster solutions by most vendors are just not there. (I have bled a lot with a SOPHOS cluster in the past. Discussions with their support were absolute crap, too. But really all vendors are the same, some more, some less.)

Once you have an 'appliance' (which is just a firewall+IDS/IPS external and in front of your production systems), you are covered for the 80% of use cases. Which is nice, if you just need a single box for securing a small network.

Not wanting to run a custom-built OS on generic hardware, but buying a dedicated box?
Don't bother, it costs a lot of time, will not work reliably, is just expensive in every way.

I was going to write more here on some real-life examples with examples, but this will be stuff for another posting.

knowledge can easily be adapted to windows

Once having an idea how things work, this knowledge can be adapted easily to windows systems from my experience, whereas windows users often have really hard times with the CLI. But often CLI is all you have, and I dimly remember microsoft's hyper-v has soon (if not yet already) an install mode where you only have a console, and no gui anymore. This in combination of the advent of the windows powershell just screams, 'get comfortable on the commandline'. Powershell is just bloated, unintuitive and complex from my perspective, but at least you can automate things an order of magnitude better with it. Also SSH finds it way into windows.

Man might see a pattern here. :)

there are many flavours of unixoid OS's out there

Out of neccessity, a word from the wise:

To get some perspective for the linux fanboys out there: Even linux is not the pinnacle of everything. Lots of doubled functionality and applications doing the same things and you have to know like three applications where a good single one would cut it, too. Open-source driven without a real single paying customer behind it, development-wise it is like PHP. There sure are companies investing money in the kernel and userland development, but there are just to many directions at once. It is not like a single concentrated effort behind everything, spreading the development power thin.

Do you think it is just a coincidence Netflix and Whatsapp run on BSD systems?

But for things beside the standard use cases, BSD's are not the wisest choice, too. If you need esoteric stuff, linux is faster up and running, has the better drivers (or has at least even drivers at all) and thus runs on much more hardware more easily (NetBSD? whatever.). At the end of the day there exist simply way more userland tools on which more work is done. It is not just about the core system, you BSD guys. Just compare linux' top with bsd' top, functionality wise.

I know, that this just a question of manpower. But how about shifting the BSD focus from servers to the becoming THE operating system for the internet of things? Gaming consoles are already more likely bsd-based, for example.

storage knowledge essentials

There are several layers, in short:

  • hardware

Starting with hardware or software raid setups, giving you redundancy. Over the different discs (HDD vs. SSD and the available types, block size stuff, and interfaces like SATA vs. SAS). To network technologies, but these are not of interest in a 'basic course', just know you can access storage via networks, too.

  • raids

Software vs. fake raids vs. hardware raids. What you usually use, what types exist with which tradeoffs. How much you can rely upon them really.

What happens when things break? What issues can arise, and how can these show up?

This is basic essential stuff. It's needed for redundancy, and HARDWARE DOES FAIL. Period.

You need it, and you need to know what you are doing.

  • partitioning

There are BIOS based systems, but the transition it currently to UEFI. The hardware brain of the computer is only indirectly linked to you partitioning your disks, but there are reasons MBR and GPT's exist. Also the knowledge from how your system boots from the last section comes handy here.

Also you have to know about the almighty LVM, which lets you do things you would not be able to do otherwise.

  • filesystems

How the actual files are stored on disk, which is just a stream of magnetic (or other techniques) information, what are the differences. How you can use these, what are advantages of one over the other.

ipv4 and networking

V(X)LANs, LACP, QoS, traffic shaping are rather less important concepts but you should have heard at least what they are.

IP protocol is everywhere.

From the datacenter at work to your router at home and in the 'internets of things' (tm) within the next 5 to 10 years.

IPv6 is not there yet

Sure, the networks are depleted, but things will keep on for a little longer while, NAT will help until larger adoption. IP v6 will not see huge adoption rates unless the carriers and telcos agree on a switch and consumer hardware sees nationwide rollouts. As long as the old consumer routers do not speak ipv6, there is no point in doing a grand scale switch. Adoption rates in every country of the world were usually way below 10 percent the last time I checked.

It is nice to know that ipv6 exists and getting a dual stack setup up and running is nice, but not something to learn if you have only three months.

NAT, PAT, subnetting, bridging, VLAN

Concepts are universal, and you can try them out easily at home. (Exception here are the VLAN's, you need a switch being able to do these as well as working NIC's.)

No need to go complete lowlevel, but you should know what the difference between network devices is and what a how a broadcast domain is different from a collision domain. No need to know what multiplexing really is, just know ethernet exists and this is what switching is about, whereas routing is the "ip stuff".

Linux will help there, as the kernel can do a lot of things so you can play with networks.

DNS

In 30% of all cases when things break, it's DNS stuff. (At least that how it feels for me, the guess may be off.)

It's easy, it's simple. And sometimes people running a web agency for over 10 years are too stupid to set up an A and a PTR record properly?

You got to be fucking kidding me, but I am not making this up. It's easy, just noone bothers ever to tell new people how to do things right.

VPN

Virtual private networking.

Three words, endless hours of unfruitful troubleshooting and disconnectivity, if you are lacking your network basics. Still essential in everyday business work.

When it is just so simple, if you have roughly an idea what you are doing with networks. For openVPN sprinkle in a 'little' PKI / SSL/TLS certificates knowledge.

But cert knowledge is an absolute MUST in the long run, no matter what you do. You have to blindly know how to use openssl, how certificate files can and have to look, how to they are actually created.

server virtualization technologies

There exist several layers of virtualization, and there is no really good differentiation out there between some of them. But they exist and are important as they help you a lot with your work, they let you try things without having to reinstall servers completely, are just faster then playing with regular hardware and thus enable way faster feedback loops.

Using snapshots is just damn easy:
Need to try an update, have no test environment at hand?
If it goes wrong you are in deep shit?
Operating system virtualization has got you covered.

There also exists storage virtualization, like DRBD, which is essential for budget clustering without a dedicated shared storage. It's basically a RAID 1 setup over a network connection.

Of course you can get an EMC or dothill storage or whatever. But that is spending $$$ again, and often you do not need the extra performance through premium hardware (except for virtualization cluster environments) or just cannot afford it.. These SATA 6g platters don't pay themselves, and waiting weeks for a new harddisk due to delivery issues does not help your damaged RAID or your nerves. And when not using original hardware you may void your warranty, and are just as bad off if you built the box yourself in the first place..............

summing everything above up

Hand in hand with virtualization go storages and storage technologies.
Be it local or network storage.
And non-local storages need network connections.
And network connections need to be setup on the operating system.

This is the full circle.

Once you need to get an understanding on clusters, simply build on all of the foundations above and be amazed how easy all this falls into place.

And be left wondering why others seem to have such a hard time with it, or cannot seem to know where to start fixing when things break down.

This was written in a single session, lets see how it holds up over the next three months.

Speedport w701 rs232 pin location

posted on 2015-08-09 00:17:33

For documentation purposes:

        Top right of PCB
_________________________
                         |
F     R A M              |
L                WIFI    |
A                CHIP    |
S        SoC             |
H        CPU  XTal       |
                 4  GND  |
                 3  TX   |
                 2  RX   |
                 1  VCC  |
                         |
              ADM6996    |
                         |
                         |
                         |
                         |
                         |
                         |
                         |
                         |
                         |
                         |
                         |
                         |
                         |
_________________________|

An rs232 bus is used there: GND, TX, RX is what you searched for. Found here.

LVM: shrink volume

posted on 2015-08-07 18:34:25

To shrink a LVM partition, there are several steps to be reproduced:

  • the volume has to be unmounted
  • activate the LVM volumes, so linux can handle them
  • check that the filesystem is error free
  • shrink filesystem, a little more than needed
  • shrink LVM partition
  • expand filesystem to full LVM partition size
  • fsck again, if you are anxious :)

If the volume is mounted, you will not be able to filesystem-check it, or even shrink it. So you can not simply shrink the root partition of your running live system. For this you will need a live disk (google for 'grml linux') and boot from this to make the changes.

So here something to copy paste from:

vgchange -a y
e2fsck -f /dev/<volume_group>/<logical_volume>
resize2fs /dev/<volume_group>/<logical_volume> <size-in-GB-MINUS-1GB>G
lvreduce -L <size-in-GB>G /dev/<volume_group>/<logical_volume>
resize2fs /dev/<volume_group>/<logical_volume>
e2fsck -f /dev/<volume_group>/<logical_volume>

Voila.

Usually you'd want to do this in order to create another volume / partition, but this is stuff for another blogpost.

RIGOL DS1054z cheatsheet

posted on 2015-08-06 18:51:11

I own a RIGOL oscilloscope, which is like the best bang-for-buck in the low end price sector, it seems.

PRO:

  • 4 channels
  • 50 Mhz (can be hacked to 100Mhz)
  • can analyze SPI, RS232 etc (free via trialperiod at first, but this can be hacked to persist, too)
  • CHEAP (other comparable products cost three times as much, easily so)
  • 400 euros (you can get an older one, like a 2-channel DS1052E for like 250 to 300 Euros already!)
  • digital scope, not analogue
  • big memory (maximum of 24M can also be hacked)

CONTRA:

  • menus are somewhat muddled and intricate
  • auto/normal/single trigger functioniality duplicated 3 times (2 times via button, plus menu)
  • menus are... oh well

To mitigate the menu problem, here all shorcuts for regular use-cases will be described.

TBD: group this and expand where needed

  • set x10 etc
  • enable roll mode
  • measure everything
  • show frequency
  • measure more than current channel
  • show history graph
  • show history table
  • chose bus type (RS232 or whatever)
  • delete extra graphs
  • delete all extra graphs
  • scroll extra graphs, if possible
  • reset graph positions
  • cursor handling
  • proper trigger handling
  • show version
  • show enabled modules
  • enter license key
  • change persistence time
  • change grid type
  • store to usbstick in all variations
  • use rj45 uplink
  • use usb uplink
  • auto vs. normal vs. single
  • maximums of possible measurements
  • change fineness degree when scaling the graph
  • limits of the device (list things that cannot be shown)
  • math
  • reference handling!
  • trigger out and pass/fail
  • overlay on/off

iptables: sole config

posted on 2015-08-03 17:21:27

DISCLAIMER: This is almost a complete ripoff of this answer here.

Usually when ending a iptables rule with something like -j LOG --log-prefix "dropped:", this information will go straight to the general syslog file. This creates quite some clutter, depending on the rules your firewall has in place.

/etc/rsyslog.d/10-iptables:

if ( $msg contains 'IN=' and $msg contains 'OUT=' ) 
then { 
    /var/log/10-iptables.log
    stop
}

& ~ is deprecated in the new rsyslog, you should use stop instead.

/etc/logrotate.d/iptables:

/var/log/iptables.log
{
        rotate 30
        daily
        missingok
        notifempty
        delaycompress

        postrotate
                service rsyslog rotate > /dev/null
        endscript
}

Note: The prefix is set to 10- to catch it before it reach the default rules (i.e. named 50-defaults).

Lighnting connector pin usage

posted on 2015-08-03 00:52:15

For documentary purposes, the used pins on a regular apple USB-to-Lightning cable are like this:

(Shown here are the receptacles, not the plugs themselves!)

LIGHNTNING:

          A  D  D
          C  A  A  V
          C  T  T  b           G
          I  A  A  u           N
          D  1  2  s           D
       + -------------------------+
       |  1  2  3  4  5  6  7  8  |
       |                          |
       |  1  2  3  4  5  6  7  8  |
       + -------------------------+
          G  D  D  A  V
          N  A  A  C  L
          D  T  T  C  u
             A  A  I  s
             1  2  D


USB:

           +--------------+
           |              |
           |  1  2  3  4  |
           +--------------+
              V  D  D  G
              b  A  A  N
              u  T  T  D
              s  A  A
                 1  2

When chargin the phone, these voltages can be measured:

V_bus         = 5.1V
DATA1 / DATA2 = 2.1 to 2.7V
GND           = 0V

ACC_ID        = 0.1V

The measurement difference could be due to my cheap multimeter, I do not know. On the lower side of the lightning connector the values were quite higher for the data channels.

The Accessory ID info is from here.

This blog covers .csv, .htaccess, .pfx, .vmx, /etc/crypttab, /etc/network/interfaces, /etc/sudoers, /proc, 10.04, 14.04, AS, ASA, ControlPanel, DS1054Z, GPT, HWR, Hyper-V, IPSEC, KVM, LSI, LVM, LXC, MBR, MTU, MegaCli, PHP, PKI, R, RAID, S.M.A.R.T., SNMP, SSD, SSL, TLS, TRIM, VEEAM, VMware, VServer, VirtualBox, Virtuozzo, XenServer, acpi, adaptec, algorithm, ansible, apache, apachebench, apple, applet, arcconf, arch, architecture, areca, arping, asa, asdm, autoconf, awk, backup, bandit, bar, bash, benchmarking, binding, bitrate, blackarmor, blockdev, blowfish, bochs, bond, bonding, booknotes, bootable, bsd, btrfs, buffer, c-states, cache, caching, ccl, centos, certificate, certtool, cgdisk, cheatsheet, chrome, chroot, cisco, clamav, cli, clp, clush, cluster, coleslaw, colorscheme, common lisp, configuration management, console, container, containers, controller, cron, cryptsetup, csync2, cu, cups, cygwin, d-states, database, date, db2, dcfldd, dcim, dd, debian, debug, debugger, debugging, decimal, desktop, df, dhclient, dhcp, diff, dig, display manager, dm-crypt, dmesg, dmidecode, dns, docker, dos, drivers, dtrace, dtrace4linux, du, dynamictracing, e2fsck, eBPF, ebook, efi, egrep, emacs, encoding, env, error, ess, esx, esxcli, esxi, ethtool, evil, expect, exportfs, factory reset, factory_reset, factoryreset, fail2ban, fbsd, fdisk, fedora, file, filesystem, find, fio, firewall, firmware, fish, flashrom, forensics, free, freebsd, freedos, fritzbox, fsck, fstrim, ftp, ftps, g-states, gentoo, ghostscript, git, git-filter-branch, github, gitolite, global, gnutls, gradle, grep, grml, grub, grub2, guacamole, hardware, haskell, hdd, hdparm, hellowor, hex, hexdump, history, howto, htop, htpasswd, http, httpd, https, i3, icmp, ifenslave, iftop, iis, imagemagick, imap, imaps, init, innoDB, innodb, inodes, intel, ioncube, ios, iostat, ip, iperf, iphone, ipmi, ipmitool, iproute2, ipsec, iptables, ipv6, irc, irssi, iw, iwconfig, iwlist, iwlwifi, jailbreak, jails, java, javascript, javaws, js, juniper, junit, kali, kde, kemp, kernel, keyremap, kill, kpartx, krypton, lacp, lamp, languages, ldap, ldapsearch, less, leviathan, liero, lightning, links, linux, linuxin3months, lisp, list, livedisk, lmctfy, loadbalancing, locale, log, logrotate, looback, loopback, losetup, lsblk, lsi, lsof, lsusb, lsyncd, luks, lvextend, lvm, lvm2, lvreduce, lxc, lxde, macbook, macro, magento, mailclient, mailing, mailq, manpages, markdown, mbr, mdadm, megacli, micro sd, microsoft, minicom, mkfs, mktemp, mod_pagespeed, mod_proxy, modbus, modprobe, mount, mouse, movement, mpstat, multitasking, myISAM, mysql, mysql 5.7, mysql workbench, mysqlcheck, mysqldump, nagios, nas, nat, nc, netfilter, networking, nfs, nginx, nmap, nocaps, nodejs, numberingsystem, numbers, od, onyx, opcode-cache, openVZ, openlierox, openssl, openvpn, openvswitch, openwrt, oracle linux, org-mode, os, oscilloscope, overview, parallel, parameter expansion, parted, partitioning, passwd, patch, pct, pdf, performance, pfsense, php, php7, phpmyadmin, pi, pidgin, pidstat, pins, pkill, plasma, plesk, plugin, posix, postfix, postfixadmin, postgres, postgresql, poudriere, powershell, preview, profiling, prompt, proxmox, ps, puppet, pv, pveam, pvecm, pvesm, pvresize, python, qemu, qemu-img, qm, qmrestore, quicklisp, quickshare, r, racktables, raid, raspberry pi, raspberrypi, raspbian, rbpi, rdp, redhat, redirect, registry, requirements, resize2fs, rewrite, rewrites, rhel, rigol, roccat, routing, rs0485, rs232, rsync, s-states, s_client, samba, sar, sata, sbcl, scite, scp, screen, scripting, seafile, seagate, security, sed, serial, serial port, setup, sftp, sg300, shell, shopware, shortcuts, showmount, signals, slattach, slip, slow-query-log, smbclient, snmpget, snmpwalk, software RAID, software raid, softwareraid, sophos, spacemacs, spam, specification, speedport, spi, sqlite, squid, ssd, ssh, ssh-add, sshd, ssl, stats, storage, strace, stronswan, su, submodules, subzone, sudo, sudoers, sup, swaks, swap, switch, switching, synaptics, synergy, sysfs, systemd, systemtap, tar, tcpdump, tcsh, tee, telnet, terminal, terminator, testdisk, testing, throughput, tmux, todo, tomcat, top, tput, trafficshaping, ttl, tuning, tunnel, tunneling, typo3, uboot, ubuntu, ubuntu 16.04, udev, uefi, ulimit, uname, unetbootin, unit testing, upstart, uptime, usb, usbstick, utf8, utm, utm 220, ux305, vcs, vgchange, vim, vimdiff, virtualbox, virtualization, visual studio code, vlan, vmstat, vmware, vnc, vncviewer, voltage, vpn, vsphere, vzdump, w, w701, wakeonlan, wargames, web, webdav, weechat, wget, whois, wicd, wifi, windowmanager, windows, wine, wireshark, wpa, wpa_passphrase, wpa_supplicant, x11vnc, x2x, xfce, xfreerdp, xmodem, xterm, xxd, yum, zones, zsh


Unless otherwise credited all material Creative Commons License by sjas