Some notes about people and mistakes

2 minute read

I have been criticize many times in my past for mistakes that were done. However, they just pointed the finger on what’s wrong. “You need to do this better”. That’s not helpful. Somehow this sounds normal for many, but it’s bad.

Don’t get me wrong, criticism is essential for everyone. We need it to become better in things we do. The questions is often how it’s done. One example that comes to my mind is grammar. I’m not good with grammar.

Especially when I wrote documentation. There is a always something I get wrong.

But you can tell someone that he made mistake in his writing or you can tell them that there are bad because of such mistakes. One points at a flaw, which is nice. The other is making you feel bad. Even worse you’re being punished for a mistake. But this way you’ll not get better with it.

Pushing for mistakes is a common patter I have seen over the past years. But it’s like a poison and not productive. Sure sometimes it can help. But most of the times it’s not.

What’s the point I want to make: Try not to do this. Try to add value to the things. Sometimes just the attempt is of value. It’s much about how we judge others and it’s hard to not fall for this pit. Point out where a mistake lies. Tell them how to detect them. Maybe how to prevent them for the further. Don’t make them feel bad about the mistake, help to feel good doing so and that they learning about it.

best regards Akendo

Converting a netmask to CIDR with vanilla python

1 minute read

On a quick note:

I wasn’t able to find a simple solution to convert a classicly formated netmask to a CIDR format in python. So I wrote this line:

sum([ bin(int(bits)).count("1") for bits in m_netmask.split(".") ])

Place into a function:

def netmask_to_cidr(m_netmask):
  return(sum([ bin(int(bits)).count("1") for bits in m_netmask.split(".") ]))

It takes a netmask as a string (for example 255.255.255.0) and will convert it into a binary representation, then it will count the ones in it. Subsequently it computes the sum of the count result. Very simple.

Some examples:

In [79]: netmask_to_cidr('255.255.255.0')
Out[79]: 24

In [80]:                                                                                                                                                      
                                                                                                                                                              
In [80]: netmask_to_cidr('255.255.255.255')                                                                                                                   
Out[80]: 32                                                                                                                                                   
                                                                                                                                                              
In [81]: netmask_to_cidr('255.255.255.128')                                                                                                                   
Out[81]: 25 

Please be aware it assume you have a valid netmask as input.

You can also see it here

have fun!

Postgres data migration

1 minute read

Quick note how to upgrade a postgresql from version 9.3 to 9.5. You’ll need to start both services. In my case postgresql 9.3 was listen on port 5432 (defaul port) and postgresql on port 5433.

Now you need to run:

pg_dumpall |psql -d postgres -p 5433

Depending on the size of your dbs it will take some time. When this went well you can connect to the postgresql-9.5 to check that the databases are correctly migrated. Next is to stop postgresql-9.3 and reconfigure postgresql-9.5 to listen on the default port. Restart postgres-9.5 and here we go!

best regards akendo

Task Warrior - Preventing duplicated recurrent tasks

1 minute read

Task Warrior Logo To keep track of my tasks I use a tool called Task Warrior. It’s a command line tools. Therefor a perfect fit for me.

Overview of task warrior

One issue that did arise: Recurrent tasks. Basically every task that need to re-run over a regular period of time.

The problem is when this type of task is synced onto different workstations. With taskd you can share all tasks betweens hosts.

So when the same tasks is recurrent, it will be created on each of the different workstation. All new task have a unique UUID. With this UUID taskwarriror ensures that tasks with the name can exists and don’t collide. This means that n many workstations will create n times the same recurrent task.

There is no way to create a single task over the n workstations. Task Warrior has no fix for this in the moment. Only a workaround: By setting following parameter in the .taskrc you can control the behavir of taskwarriror.

recurrence=off

So you need to set the recurrence parameter to off on n - 1 workstations. Only a single remains allowed to create the recurrence tasks. This way you can make sure nothing gets messed up.

I got this workaround from Paul Beckingham’s Comment in the Jira of taskwarriror.

Anyway best regards Akendo

Using OpenWRT as Router

6 minute read

cheaper Router as a general problem

Some years ago,I had a very cheap router. A TP-Link DIR 600. The router made my very unhappy. For several reasons. I had this 'unsecure' Router and was unhappy about it.

First, the software that was operating on the Router were very limited. How limited? I was able only to set a few firewall rules (DNAT for example) on my own. When I recall correctly, i wasn’t able to redirect any port.

Second, the little insight to the devices I had. You just got a very basic logger on the devices that would only give the minimal amount of information. This is especially a problem when you tried to debug connections. My ISP at that time made it even harder, because they did Carrier-grade NAT. Means they was not really giving me any real public IP, but a private IP. The router on the other side of the DLS Modem them NAT them back. Not nice for forwarding traffic.

Last, the software had some flaws. To replace this devices was one of the better decision [0], when the router router apocalypse came I was prepared. The TP-LINK DIR 600 was affect very much. However back at the time this wasn’t much of an issue. The software had just some limitation and that drove me crazy.

OpenWrt

is a Linux distribution, that is intended to support small router(CPE). It’s focus in general is on WiFi. Another possible software to run on such small router is for example ddwrt.

However the old router (DIR 600) of mine didn’t supported any other Software. No drivers for some important part of the device. So a new devices was at need.

I choose OpenWRT over ddwrt for a simple reason. I know some people that work with it, they do freifunk. They heavily rely on OpenWrt and modified the software to they needs. Most of them did work with the TP-Link WDR3600.

Why such small devices after all? I mean it might be more reasonable to get better hardware to use it as router. The answer is simple. The router is very cheap. They don’t draw much power and I don’t have to play around with the hardware.

In addition to this, I liked to play with embedded hardware around. This way I would have touched something else than an x86 system. The TP-Link WDR3600 based on a MIPS CPU Architecture. Something completely different.

OpenWrt as router

So I got one like this.

The installation process is simple. You need to download the generic-tl-wdr3600-v1-squashfs-factory.bin. Drop this file in the update firmware page of the original firmware and you good to go! But you can find more details here

Next is to login via telnet to the box.

Please note: I had some upgrade issue I documented a while ago. One problem of OpenWRT is that you don’t have any type of auto update. At least I haven’t found them yet.

Install software on a openwrt router

Here some basic things. OpenWRT does provide a software like debians apt called opkg. information.

Installation openvpn

There are some remarks to the openvpn installation. They are using different libraries for SSL by default. You’ll get something like polarssl. I installed openvpn with openssl support. That makes my configuration quite alike and does not create snowflakes configuration files.

opkg install openvpn-openssl

Besides, you shouldn’t trust any SSL implementation that hasn’t been audit. openssl is bad, broken and fuck up. But I would bet it’s broken the least. Especially since the hearthbleed incident they infrastructure gain a big amount of attenchen and money. All what raises the changes that the software stop being such waste. Another possible ssl implementation to use would be libressl, but this getting of topic.

Installation openssh

While OpenWRT provides a dropbear ssh server, i always like to use OpenSSH instead. So you can install and setup openssh via:

opkg install openssh-server

Please note you need to disable the dropbear in favour of the ssh.

Issues

My experience so far: Very well. By now I do have a SSH Server on it with well set crypto settings in place (yay). WiFi is someway okish. Some of the hardware support for Wifi isn’t working. Nothing to bad.

IPv6 is pain

One exception is there: IPv6.

For some reason the IPv6 Stack is broken. It’s not broken in it’s function. I’m able to send and receive data. But any linux client does not get a correct routing information. Windows clients are not affected.

This made this strange.

I have a router in ISP provided modem. It connects me to my cable IPS. I have a Dual Stack Lite (DS-Lite) on this modem. By the way, don’t get starting about the security on this device…. DS-Lite means: I have fully working IPv6 address and again does my provided do a Carrier-grade NAT for IPv4. That’s acceptable.

So what this modem does, it announce it’s IPv6 Network Mask to the network. My OpenWRT is the next router that does the forwarding. It acts as gateway and protects me from the everything. I don’t trust the ISP modem. I do NAT for the IPv4.

But because this a nativ IPv6 modem it tries to auto configure IPv6 to all clients in the network. For this we’ll need Router Advertisement.

However the linux client seems to get the IPv6 route wrong. Instead of replacing the IPv6 with the correct IPv6 address of the OpenWRT Router, it keeps in using the IPv6 address of the modem. Windows however does get the correct IPv6 route…. I does to the OpenWRT gateway and that forwards all the IPv6 taffic to the modem. My linux however is miss the hope and can’t reach out to the web correctly with IPv6.

It seems to be a kernel bug in OpenWRT. But I might have to dig into this in sometime… My fix for the moment is to just have the IPv6 Route place manual on my linux clients…and don’t let me get starting on newer android devices….

Security aftermath

Two years ago(around 2015) there were something that was called ‘plastic router apocalypse’. tldr; Poor products with bad design flaws became victims to bad persons.

Long Story: Most router are left unattended and unmaintained. Because they are produces very cheap and on mass. They’re place only once after you purchases a DSL connection. No one brother about this type of devices.

At some point, some guys started to take a look onto them and looking for some security flaws. What they found was bad. Many flaws. This was partial bad because most of this plastic router are exposed directly to the internet. So they were victim to bad guys.

So replacing it

Resources

[0] http://www.s3cur1ty.de/m1adv2013-003

€dit: It took me 3.2 years to get this from a draft to a finished document….so things are moving!

News about the HipChat breach that confused me

1 minute read

This morning I got a info about a security breach at Atlassian. To be precise, HipChat. The “HipChat security notice”. The news page I was reading additionally pointed out a security vulnerability in Atlassians confluence. I got a chill in this moment. Somehow I got this wrong. I understood there would be a new vulnerability.

Because of a bad Internet connection I wasn’t able to the entry article(just got the header). So I rushed to shutdown my confluence instances. When a better Internet connection was available I continued to read. It turned out to be an already fixed vulnerability that was not related to the data breach of HipChat.

When you’re going to write an article about a security incidents, don’t point out to the most recent (and already) fixed vulnerability. People gets confused about this…. or at me.

best regards akendo

Notes about Hugo

2 minute read

Hugo allows you to get very easy a new theme. Sometimes this new theme have some option you don’t want. For example:In this theme there is by default two additional menus. One for tags, for for categories. That’s something nice.

However, both menu had a very long list, that than needed scrolling bar. The scrolling bar broken than the design of the theme.

My fix for this was disable the two menu for categories and tags. For this I created custom layout pages to overwrite the themes ones. In Hugo there is an order how template documents are rendered.

/layouts/section/SECTION.html
/layouts/_default/section.html
/layouts/_default/list.html
/themes/THEME/layouts/section/SECTION.html
/themes/THEME/layouts/_default/section.html
/themes/THEME/layouts/_default/list.html

As we can see the local folder in /layouts/ are checked first for any template pages to render. Followed by the _default type of a page. Only then comes the template pages of the theme.

So I copied the layout file of the theme to the local /layouts folder. In there I altered the template to not render the menus for categories and tags.

cp themes/code-editor/layouts/partials/menu.html layouts/partials/

The file looks like this now:

<nav class="col-md-3">
    <h3 class="home-link"><a href="/">{{ ( index $.Site.Data.translations $.Site.Params.locale ).root }}</a></h3>
    <div id="last-posts" class="open">
        <h3 data-open="last-posts">{{ .Site.Title }} - {{ ( index $.Site.Data.translations $.Site.Params.locale ).mostrecentposts }}</h3>
        <ul>
            {{ range first 15 .Site.Pages }}
            <li><a href="{{ .Permalink }}">{{ .Title }}</a></li>
            {{ end }}
        </ul>
    </div>
</nav>

Furthermore, I increased the amount of items that are listed from 10 to 15.

so far akendo

grafana

5 minute read

I attended the 33c3 in Hamburg. A awesome event, as always. One of the slogan: “use more bandwidth!”. To display the current amount of data that has been sent a dashboard (Note: Down to this time of the year) was created. I want to have something similar. So I started digging into the dash board while be on the congress. Which is running on grafana.

Grafana

What is grafana?

Grafana is a open source metric analytics & visualization suite. It is most commonly used for visualizing time series data for infrastructure and application analytics but many use it in other domains including industrial sensors, home automation, weather, and process control.

Quote from the docs

grafana A similar look, but with more details on bandwidth, this is the dashboard I created.

Setup

For testing it, I create a VM, a script that captures my traffic on a interface (wlan0) and a Database. You can download grafana with a fitting binary from the project page. My example VM was ubuntu 14.04, create with vagrant.

wget https://grafanarel.s3.amazonaws.com/builds/grafana_4.0.2-1481203731_amd64.deb
sudo apt-get install -y adduser libfontconfig
sudo dpkg -i grafana_4.0.2-1481203731_amd64.deb

Note: only use this type of installation for testing, for production use please add the repository!

Next step is to read the getting started. You’ll recognize that a datasource is necessary. Using Plaintext is not a option.

Data sources / InfluxDB

For the sake of simplicity, I used influxDB, as described here.

What’s that? Again a quote from they docs

InfluxDB is a time series database built from the ground up to handle high write and query loads. It is the second piece of the TICK stack. InfluxDB is meant to be used as a backing store for any use case involving large amounts of timestamped data, including DevOps monitoring, application metrics, IoT sensor data, and real-time analytics.

I has a neat rest api you can sent your data to.

Installation

Same simple tick like before for the installation:

wget https://dl.influxdata.com/influxdb/releases/influxdb_1.1.1_amd64.deb
sudo dpkg -i influxdb_1.1.1_amd64.deb

Note: only use this type of installation for testing, for production use please add the repository!

enable remote access for InfluxDB

Because I didn’t create the data on the VM itself, a remote host needs access to the db. InfluxDB is configure to allow access ONLY from localhost. Yet without any further authentication, how great this is seems debatable. But for another node you need to change within the /etc/influxdb/influxdb.conf the auth-enabled from true to false.

Note: This is only done of the sake of testing!

InfluxDB create a db

Just start the influx-shell and create a db. Note: The official way, as documented on github is broken….

influx -precision rfc3339
CREATE DATABASE mydb

Get network data, the hacky way

There are many great tools out for collecting network statics, I prefer for example vnstat. But for getting straightforward data it’s quite a pain. So instead I use python to do this job.

psutil

There is useful python module, called psutil. It can read data from the /proc/ filesystem in an easy fashion. To get the data send per second, you’ll need to read the /proc/net/dev. In there is a static for network interfaces.

Inter-|   Receive                                                |  Transmit
 face |bytes    packets errs drop fifo frame compressed multicast|bytes    packets errs drop fifo colls carrier compressed
  eth0:  217872    1985    0    0    0     0          0         0   202460    1690    0    0    0     0       0          0
  eth1:  237343     500    0    0    0     0          0         0   852594     594    0    0    0     0       0          0
    lo:  262258     570    0    0    0     0          0         0   262258     570    0    0    0     0       0          0
docker0:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0

_ content of a /proc/net/dev file _

Next is to select the interface of your desire and create the delta of the receive/transmitted byte and/or packets.

Here’s the script to get the traffic in one second on an interface:

#!/usr/bin/env python3
"""
Interface static 

Usage:
  interface-statistic.py help | --help | -h
  interface-statistic.py version| --version | -v
  interface-statistic.py <interface>

Akendo 2016
Apache 2.0
"""


# https://pypi.python.org/pypi/psutil/
import psutil # for access to the procfs
from docopt import docopt
from time import sleep

class interface:
  def __init__(self, interface_name):
      # test for /sys/class/net/{interface} is, else this is not existing interface
      # test for /sys/class/net/{interface}/link_mode is 1
      if interface_name not in psutil.net_if_stats():
        raise Exception('invalid interace name!')
      if not psutil.net_if_stats()[interface_name].isup:
        raise Exception('interace is down!')
      self.interface_name = interface_name
      self.data = []

  def stats(self, interface_name):
    # 0, bytes_sent
    # 1, bytes_recv 
    sent_a = psutil.net_io_counters(pernic=True)[self.interface_name][0]
    recv_a = psutil.net_io_counters(pernic=True)[self.interface_name][1]
    sleep(1)
    sent_b = psutil.net_io_counters(pernic=True)[self.interface_name][0]
    recv_b = psutil.net_io_counters(pernic=True)[self.interface_name][1]
    print('TX/s:{0} RX/s:{1}'.format(sent_b - sent_a, recv_b - recv_a))


if __name__ == '__main__':
  arguments = docopt(__doc__, version='Interface static  0.0a')
  if '<interface>' in  arguments:
    iface = arguments['<interface>']
    net = interface(iface)
    net.stats(iface)
  else: 
    print(arguments)

When the code is execute with an valid interface, that’s up and running, you’ll get the traffic from the last second.

python interface-statistic.py enp0s25
TX/s:102 RX/s:102

Putting everything together

Next is to send the data to the influxdb and configure grafana to display this dataset. Here’s a hacky bash script to send it to the db. I just was to lacy at this point to do this proper in python.

#!/bin/env bash

while /bin/true;
do
out=$(python interface-statistic.py wlp3s0)
curl -XPOST 'http://192.168.56.200:8086/write?db=mydb' -d "traffic,host=x240 up=$(echo ${out}|cut -d ':' -f 2 |sed 's/[^0-9]*//g'),down=$(echo ${out}|cut -d ':' -f 3 )"
done

This will create a measurement in influxdb, with the tag host with value x240, with two field-keys. Their contains the value up and down. Let check this:

influx -precision rfc3339
Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring.
Connected to http://localhost:8086 version 1.1.1
InfluxDB shell version: 1.1.1
> use mydb
Using database mydb
SELECT "down", "up" FROM "traffic" 
...
16-12-30T17:05:50.226740631Z  1268            816
2016-12-30T17:05:51.319601571Z  3815            1836
2016-12-30T17:05:52.401584693Z  4467            3186
2016-12-30T17:05:53.512383018Z  1137            758
2016-12-30T17:05:54.600852827Z  2857            1200
2016-12-30T17:05:55.698312396Z  2814            1164
2016-12-30T17:05:56.81280255Z   2898            372
2016-12-30T17:05:57.885670347Z  3000            1284
2016-12-30T17:05:59.040079442Z  2264            792
2016-12-30T17:06:00.134257182Z  1522            552
2016-12-30T17:06:07.686102817Z  2210            416
...

Here we go! Now we can utilize this in the dashboard!

create a dashboard in grafana

Last step is to add a dashboard and let select the right data set. The result can be seen above.

recap

grafana is a nice tool, it also includes an alert feature that will allow you to notify in case a thrash value is hit. It’s a tool I’m going to use in the further more.

best regards Akendo