Posts from 2016-12

apache proxy ssl directives

posted on 2016-12-29 18:01

Likely you'd need these:

SSLEngine on
SSLProxyEngine on
SSLProxyVerify none 
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off
SSLCertificateFile      CERTFILE
SSLCertificateKeyFile   PRIVKEY
SSLCACertificateFile    CACERTFILE

clustershell

posted on 2016-12-29 13:26

When needing to run commands on several servers over ssh, there's always that for-loop for you.

But you could also try running clustershell:

sjas@ws:~$ clush -w server-[01,02,05,11,12] -b hostname -f
---------------
server-01
---------------
server-01.some-domain.com
---------------
server-02
---------------
server-02.some-domain.com
---------------
server-05
---------------
server-05.some-domain.com
---------------
server-11
---------------
server-11.some-domain.com
---------------
server-12
---------------
server-12.some-domain.com

-b to use it non-interactively and to get the shown aggregated results (the hosts are colored), -w to specify the hosts. Use [ ] instead of { } like you would in bash.

-B also includes STDERR.

A problem you may run into, is when you try to run commands with pipes.

Further you can also predefine hostgroups and copy files from/to remote hosts. This is a rather nice tool.

notes on using vimdiff

posted on 2016-12-29 11:55

Bare minimum to do some work with it:

do - Get changes from other window into the current window.
dp - Put the changes from current window into the other window.

]c - Jump to the next change.
[c - Jump to the previous change.

Ctrl W + Ctrl W - Switch to the other split window.

If you load up two files in splits (:vs or :sp), you can do :diffthis on each window and achieve a diff of files that were already loaded in buffers. :diffoff can be used to turn off the diff mode.

Also helpful: :help copy-diffs, and this link here.

apache webdav configuration

posted on 2016-12-29 10:27

notes up front

  • don't use suexec. just don't.
  • you should be able to configure a vhost on your own, else the apache config will not be of use to you
  • we'll use ssl certifcates, too

setup

Load the apache modules:

a2enmod dav
a2enmod dav_fs

Create certificate:

cd /etc/apache2/ssl
openssl genrsa -out mydomain.de 1024  ## create private key
openssl req -new -key mydomain.de.key -out mydomain.de.csr  ## create certificate signing request
openssl x509 -in mydomain.de.csr  -out mydomain.de.crt -req -signkey mydomain.de.key -days 3650  ## create certificate
rm mydomain.de.csr 

Create a vhost config for your apache, and enable it:

<VirtualHost *:80>

        ServerName mydomain.de
        ServerAlias www.mydomain.de

        DocumentRoot /var/www/mydomain.de/htdocs

        <Directory /var/www/mydomain.de/htdocs/>
                Options Indexes MultiViews
                AllowOverride None
                Order allow,deny
                allow from all

                DAV on

                AuthType Basic
                AuthName DAV
                AuthUserFile /var/www/mydomain.de/.htpasswd
                Require valid-user
        </Directory>

        ErrorLog /var/www/mydomain.de/logs/error.log
        LogLevel warn
        CustomLog /var/www/mydomain.de/logs/access.log combined

</VirtualHost>

<VirtualHost *:443>

        ServerName mydomain.de
        ServerAlias www.mydomain.de

        DocumentRoot /var/www/mydomain.de/htdocs

        <Directory /var/www/mydomain.de/htdocs/>
                Options Indexes MultiViews
                AllowOverride None
                Order allow,deny
                allow from all

                DAV on

                AuthType Basic
                AuthName DAV
                AuthUserFile /var/www/mydomain.de/.htpasswd
                Require valid-user
        </Directory>

        ErrorLog /var/www/mydomain.de/logs/error.log
        LogLevel warn
        CustomLog /var/www/mydomain.de/logs/access.log combined

        sslengine on
        sslcertificatefile      /etc/apache2/ssl/mydomain.de.crt
        sslcertificatekeyfile   /etc/apache2/ssl/mydomain.de.key

</VirtualHost>

Create a htpasswd file:

htpasswd -c /var/www/mydomain.de/.htpasswd USERNAME

USERNAME and the password you'll enter will be your access credentials.

testing (on linux)

apt install -y cadaver
cadaver https://mydomain.de

Then your are promted for entering the user credentials. ls and help should help you onwards then.

using it with windows

Since this is a setup with SSL (doensn't make much sense to use plain http from my POV), you'll need to import the certifate (mydomain.de.crt) in windows.

Else you will get an error along the lines of Mutual Authentication Failed. The Server's Password Is Out Of Date At The Domain Controller.

It needs to go there: (in windows)

  • win + r
  • certmgr.msc
  • enter
  • trusted root certificates
  • certificates

ubuntu 10.04 change ulimit

posted on 2016-12-27 16:31

When trying to change the ulimit setting for open files this did not work system-wide by changing the /etc/security/limits.conf:

root@server:~# grep -v ^# /etc/security/limits.conf 


* soft nofile 4096
* hard nofile 10240

The only solution was to change the init script of the service needing more open files.

In my case it was a tomcat:

...

case "$1" in
  start)
        ulimit -n 10240

...

That way both the hard and the soft limit gets set to 10240, instead of setting them separately via -Hn and -Sn.

Of course (haha) you need to have enough capability to allow that many files systemwide, either put it into /etc/sysctl.conf and do sysctl -p or just do:

sysctl -w fs.file-max=1000000

Related bug report here.

proxmox delete and recreate cluster

posted on 2016-12-21 22:48

In case you have the questionable idea of renaming a hypervisor of your proxmox cluster, you are going to feel some pain. (It won't work and you will get scared wether you fucked your system landscape up or not. Been there, done that.)

The only viable and reproducible approach I found was removing all cluster configurations from all HV's, rebooting them, then recreating the cluster on one HV and readding all the others again.

read this, or continue at your own peril

To sum it up again, some notes before:

  • tested with proxmox 4.4
  • do all the next steps on all hosts
  • rebooting is neccessary aftwards, of all hv's. maybe not, but at least the first node had to be rebooted
  • the sqlite output, if any is shown, should only appear once
  • working ssh between your hv's is neccessary, errors or warning will prevent you from readding nodes after the cluster recreation
  • backup /etc/pve before doing anything, as you will lose all your vm configurations in the process, but these can be copied back afterwards.
  • no guarantee that this post will cover everything

howto remove all clusterconfigs

Let's go: (this is completely paste-able)

# backup
cp -va /etc/pve /root

systemctl stop pvestatd.service
systemctl stop pvedaemon.service
systemctl stop pve-cluster.service
systemctl stop corosync
systemctl stop pve-cluster
pmxcfs -l
rm /etc/pve/corosync.conf
rm /etc/corosync/*
rm /var/lib/corosync/*
rm -rf /etc/pve/nodes/*
sqlite3 /var/lib/pve-cluster/config.db "select * from tree where name='corosync.conf'"
sqlite3 /var/lib/pve-cluster/config.db "delete from tree where name='corosync.conf'"
sqlite3 /var/lib/pve-cluster/config.db "select * from tree where name='corosync.conf'"

Check for error messages, then:

reboot

Recreate the cluster on the first HV: (or whichever one you see fit)

pvecm create CLUSTER-NAME

Then readd all other HVs to your newly created cluster. From each of them, do:

#test ssh
ssh IP-OF-FIRST-HV

if that does work, add, else see below how to troubleshoot
pvecm add IP-OF-FIRST-HV

troubleshooting SSH issues

Adding nodes works best with keyauth (Don't know wether I ever tried it without, to be honest, but I doubt it works.). In case you have reinstalled a node or something, try connecting via ssh from the host in question to your 'first' hv.

Read the error message closely, as known hosts are stored in /etc/ssh/ssh_known_hosts, not ~/.ssh/known_hosts:

# in case you have trouble on a certain host
> /root/.ssh/known_hosts
> /etc/ssh/ssh_known_hosts
ssh-copy-id FIRST_HV

As said before, ssh errors or warnings won't let you add vm's to a cluster.

browser not working

Once you have completed the stuff above, close all browsertabs you had opened to access your cluster. Simply refreshing them does not seem to work.

finishing touches (fix your vms before you become stressed out)

When looking at the webgui, you might become scared, as all your virtual hosts seem to be missing. This happens with VM's, but I guess the same happens with Containers, too.

In fact, we worked on proxmox cluster filesystem where it stores a lot of its settings, which gets mounted at /etc/pve aftwards. Which happens to be stored completely under /var/lib/pve-cluster/config.db as a sqlite3 database.

There all file contents (the actual character that get written into the config file(s)), the inode of the file that shall be created, along with the folder structure etc. etc. .

Once your cluster is running, try diff / colordiff to spot the exact differences. (I.e. colordiff /root/pve /etc/pve to see the file contents) Or a simple find /root/pve -iname "*conf" might also do.

Copy the configs back to their original locations, and everything should be fine.

spacemacs org-mode in very short

posted on 2016-12-14 23:34

Some notes to get up to speed with orgmode AFAP:

  • Header to make emacs recognize org files regardless of file extension: -*- mode: org -*-
  • create bullet point: m-enter
  • bullet-point indenting: tab
  • change indentation of non-bulleted text: tab
  • insert bullet-points without m-enter: start a new line with one or several *'s followed by a space.
  • move bullet point up/down: m-up and m-down
  • change bullet point indentation: m-left and m-right
  • change DONE / TODO for bulletpoint: s-left and s-right
  • show TODO items: spc m T
  • TODO tags can be defined up to in file: #+TODO: TODO IN-PROGRESS WAITING DONE
  • unordered lists (not bulletpointed) start with a - (+ will work, too)
  • exit into normal mode and use o to insert a new list-line
  • descriptions use not just a single colon, but two with leading space: ::
  • change list type with s-left and s-right
  • opening/closing folds of current item: c-tab
  • insert source code block, in insert mode, do: <sTAB (<, s, tab)

There's more, but these are the very basics. A lot of these plus some other stuff can be found here.

On logging time:

  • start timer for current item: spc m I
  • stop timer for current item: spc m O
  • expand LOGBOOK: c-tab
  • insert deadline: spc m d

plesk onyx phpioncube install

posted on 2016-12-13 14:35

install php7 in plesk

plesk sbin autoinstaller --select-product-id plesk --select-release-current  --install-component php7.0

get files

http://www.ioncube.com/loaders.php

Copy to the server, und tar xzvf it.

copy file

cd ioncube
cp ioncube_loader_lin_7.0.so /opt/plesk/php/7.0/lib/php/modules/

link it with php

Into /opt/plesk/php/7.0/etc/php.ini put this:

zend_extension=ioncube_loader_lin_7.0.so

(Somewhere to the other zend options.)

reload php

  plesk bin php_handler --reread

test

/opt/plesk/php/7.0/bin/php -v

Will show you ioncube php loader (enabled) ... so it actually works.

Now don't use the OS php version (i.e. if you already have php 7 available from ubuntu 16.04), but the plesk one from the dropdown menu in the php settings of your hosting.

bonus

If you cannot upload zip files, install php-zip from your OS's package management. (apt install php-zip -y)

postfixadmin update and php7

posted on 2016-12-09 08:52

After a dist upgrade from ubuntu 14.04 to 16.04 and updating to php7 the postfixadmin in version 2.3.7 stopped working due to php5 modules naturally being amiss. These were the steps I took so the upgrade to postfixadmin 3.0 worked. The major roadblocks were the changed php modules and updating the mysql tables during the update.

  • mkdir /root/postfixadmin-backup; cd /root/postfixadmin-backup
  • mysqldump <POSTFIX-DB-NAME> > <POSTFIX-DB-NAME>.sql so I'd have a database backup.
  • Copied the old htdocs to /root/htdocs, to have a webdata backup in case I'd fuck up renaming the htdocs later.
  • cd /path/to/postfixadmin/webroot
  • The apache setup was fine so already, so I renamed the old htdocs (docroot), created a new one and extracted the newly downloaded postfixadmin there.
  • Copied the old config.inc.php over into the new docroot, named config.local.php.disabled so it would not get read upon opening the postfixadmin webinterface.
  • Copied the new config.inc.php to config.local.php.
  • diff config.local.php.disabled config.inc.php to show me the differences to the new installation.
  • Adjust database settings and the other settings I wanted to conserve.
  • Browser: https://domain.of.postfixamin/setup.php

Then it showed:

...

DEBUG INFORMATION:
Invalid query: Invalid default value for 'created'

...

Upon using https://domain.of.postfixadmin/setup.php?debug=1 I found out that that was related to the vacation table in the database. To be exact, this line shown by describe vacation at the mysql client prompt:

| created       | datetime     | NO   |     | 0000-00-00 00:00:00 |       |

The ALTER TABLE statement could not run the change due to mysql's strict mode being active, and a date of 0000-00-00 00:00:00` being forbidden.

At first I tried to alter the table to something like 1970-01-01 01:01:01 but that wouldn't work.

mysql's strict mode is controlled by the variable sql_mode:

mysql> show variables like 'sql_mode'\G
*************************** 1. row ***************************
Variable_name: sql_mode
        Value: ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
1 row in set (0.00 sec)

To set this variable upon mysql's starting process, I edited /etc/mysql/my.cnf and added under the [mysqld] section:

sql_mode = ''

Then service mysql stop, service mysql start and upon reopening https://domain.of.postfixamin/setup.php the setup went through and all was well again in postfix-land.

I removed the sql_mode parameter from /etc/mysql/my.cnf again, service mysql stop, service mysql start and I was free to go off to other ventures again.

proxmox magic fix script

posted on 2016-12-05 14:52

From here, this link often is handed out in ##proxmox on FreeNode:

#!/bin/bash

# on all nodes
magicfix() {
        service pve-cluster stop
        service pvedaemon stop
        service cman stop

        service pve-cluster start
        service cman start
        service pvedaemon start

        # this one could possibly restart VMs in 4.x (but doesn't in 3.x), so disable unless you think you need it
        #service pve-manager restart

        service pvestatd restart
        service pveproxy restart
        service pve-firewall restart
        service pvefw-logger restart
}
magicfix

# again after above was done on all nodes (makes /etc/pve rw)
service pve-cluster restart

strongswan ipsec vpn site to site

posted on 2016-12-02 09:42

This guide was written for debian 8.

network layout

local/left       lan: 192.168.0.0/16
local/left   gateway: 10.0.0.2
remote/right gateway: 10.0.0.3
remote/right     lan: 172.16.0.0/16

Our network, expressed differently:

192.168.0.0/16 --- unencrypted --- 10.0.0.2 === vpn === 10.0.0.3 --- unencrypted --- 172.16.0.0/16

In strongswan it doesn't matter which side is defined in either left or right, but this convention helps:

  • local = left
  • rremote = right

ipsec settings for the tunnel

These may be somewhat arbitrarily, but we got to use something:

phase1:
ikev1 / aes256 / sha2 / dh5 / 86400s (24h statt 8h)

phase2:
esp / aes256 / sha2 / dh5 / 3600s

( protocol / encryption / hashing / DH group or PFS if present / lifetime )

install

apt-get install strongswan libcharon-extra-plugins

define PSK

Add to /etc/ipsec.secrets:

10.0.0.2 10.0.0.3 : PSK "thatsmydamnsecretPSKwhichreallyshouldbearandomsting"

setup tunnel

/etc/ipsec.conf:

config setup

conn %default
    keyexchange=ikev1
    keyingtries=%forever
    leftauth=psk
    rightauth=psk
    auto=start

conn myconfig-main
    left=10.0.0.2
    ike=aes256-sha256-modp1536
    ikelifetime=86400s
    esp=aes256-sha256-modp1536
    lifetime=3600s

conn myconfig1
    right=10.0.0.3
    leftsubnet=192.168.0.0/24
    rightsubnet=172.16.0.0/16
    also=myconfig-main

include /var/lib/strongswan/ipsec.conf.inc

That way you can add additional phase2 entries analoguous to conn myconfig1.

%default is valid for everything, myconfig-main is included via auto=myconfig-main into other connection definitions.

test

service ipsec restart

These might help:

tail -f /var/log/syslog
watch -n1 -d ipsec statusall

Ping from withing your lan a host inside the remote lan.

For watching the pings, the ones you want to see will be colored:

tcpdump -D # discern the interface you need to have a look at, usually eth0 / 1
tcpdump -nli 1 icmp | grep -color -e $ -e 192.168.

routing rules are automatically added by strongswan, do service ipsec restart while watching:

watch -n1 -d "ip ru; ip r l t 220"

This blog covers .csv, .htaccess, .pfx, .vmx, /etc/crypttab, /etc/network/interfaces, /etc/sudoers, /proc, 10.04, 14.04, AS, ASA, ControlPanel, DS1054Z, GPT, HWR, Hyper-V, IPSEC, KVM, LSI, LVM, LXC, MBR, MTU, MegaCli, PHP, PKI, R, RAID, S.M.A.R.T., SNMP, SSD, SSL, TLS, TRIM, VEEAM, VMware, VServer, VirtualBox, Virtuozzo, XenServer, acpi, adaptec, algorithm, ansible, apache, apachebench, apple, applet, arcconf, arch, architecture, areca, arping, asa, asdm, autoconf, awk, backup, bandit, bar, bash, benchmarking, binding, bitrate, blackarmor, blockdev, blowfish, bochs, bond, bonding, booknotes, bootable, bsd, btrfs, buffer, c-states, cache, caching, ccl, centos, certificate, certtool, cgdisk, cheatsheet, chrome, chroot, cisco, clamav, cli, clp, clush, cluster, coleslaw, colorscheme, common lisp, configuration management, console, container, containers, controller, cron, cryptsetup, csync2, cu, cups, cygwin, d-states, database, date, db2, dcfldd, dcim, dd, debian, debug, debugger, debugging, decimal, desktop, df, dhclient, dhcp, diff, dig, display manager, dm-crypt, dmesg, dmidecode, dns, docker, dos, drivers, dtrace, dtrace4linux, du, dynamictracing, e2fsck, eBPF, ebook, efi, egrep, emacs, encoding, env, error, ess, esx, esxcli, esxi, ethtool, evil, expect, exportfs, factory reset, factory_reset, factoryreset, fail2ban, fbsd, fdisk, fedora, file, filesystem, find, fio, firewall, firmware, fish, flashrom, forensics, free, freebsd, freedos, fritzbox, fsck, fstrim, ftp, ftps, g-states, gentoo, ghostscript, git, git-filter-branch, github, gitolite, global, gnutls, gradle, grep, grml, grub, grub2, guacamole, hardware, haskell, hdd, hdparm, hellowor, hex, hexdump, history, howto, htop, htpasswd, http, httpd, https, i3, icmp, ifenslave, iftop, iis, imagemagick, imap, imaps, init, innoDB, innodb, inodes, intel, ioncube, ios, iostat, ip, iperf, iphone, ipmi, ipmitool, iproute2, ipsec, iptables, ipv6, irc, irssi, iw, iwconfig, iwlist, iwlwifi, jailbreak, jails, java, javascript, javaws, js, juniper, junit, kali, kde, kemp, kernel, keyremap, kill, kpartx, krypton, lacp, lamp, languages, ldap, ldapsearch, less, leviathan, liero, lightning, links, linux, linuxin3months, lisp, list, livedisk, lmctfy, loadbalancing, locale, log, logrotate, looback, loopback, losetup, lsblk, lsi, lsof, lsusb, lsyncd, luks, lvextend, lvm, lvm2, lvreduce, lxc, lxde, macbook, macro, magento, mailclient, mailing, mailq, manpages, markdown, mbr, mdadm, megacli, micro sd, microsoft, minicom, mkfs, mktemp, mod_pagespeed, mod_proxy, modbus, modprobe, mount, mouse, movement, mpstat, multitasking, myISAM, mysql, mysql 5.7, mysql workbench, mysqlcheck, mysqldump, nagios, nas, nat, nc, netfilter, networking, nfs, nginx, nmap, nocaps, nodejs, numberingsystem, numbers, od, onyx, opcode-cache, openVZ, openlierox, openssl, openvpn, openvswitch, openwrt, oracle linux, org-mode, os, oscilloscope, overview, parallel, parameter expansion, parted, partitioning, passwd, patch, pct, pdf, performance, pfsense, php, php7, phpmyadmin, pi, pidgin, pidstat, pins, pkill, plasma, plesk, plugin, posix, postfix, postfixadmin, postgres, postgresql, poudriere, powershell, preview, profiling, prompt, proxmox, ps, puppet, pv, pveam, pvecm, pvesm, pvresize, python, qemu, qemu-img, qm, qmrestore, quicklisp, quickshare, r, racktables, raid, raspberry pi, raspberrypi, raspbian, rbpi, rdp, redhat, redirect, registry, requirements, resize2fs, rewrite, rewrites, rhel, rigol, roccat, routing, rs0485, rs232, rsync, s-states, s_client, samba, sar, sata, sbcl, scite, scp, screen, scripting, seafile, seagate, security, sed, serial, serial port, setup, sftp, sg300, shell, shopware, shortcuts, showmount, signals, slattach, slip, slow-query-log, smbclient, snmpget, snmpwalk, software RAID, software raid, softwareraid, sophos, spacemacs, spam, specification, speedport, spi, sqlite, squid, ssd, ssh, ssh-add, sshd, ssl, stats, storage, strace, stronswan, su, submodules, subzone, sudo, sudoers, sup, swaks, swap, switch, switching, synaptics, synergy, sysfs, systemd, systemtap, tar, tcpdump, tcsh, tee, telnet, terminal, terminator, testdisk, testing, throughput, tmux, todo, tomcat, top, tput, trafficshaping, ttl, tuning, tunnel, tunneling, typo3, uboot, ubuntu, ubuntu 16.04, udev, uefi, ulimit, uname, unetbootin, unit testing, upstart, uptime, usb, usbstick, utf8, utm, utm 220, ux305, vcs, vgchange, vim, vimdiff, virtualbox, virtualization, visual studio code, vlan, vmstat, vmware, vnc, vncviewer, voltage, vpn, vsphere, vzdump, w, w701, wakeonlan, wargames, web, webdav, weechat, wget, whois, wicd, wifi, windowmanager, windows, wine, wireshark, wpa, wpa_passphrase, wpa_supplicant, x11vnc, x2x, xfce, xfreerdp, xmodem, xterm, xxd, yum, zones, zsh


Unless otherwise credited all material Creative Commons License by sjas