Using Keycloak APIs: "RESTEASY004655: Unable to invoke request"

Following exception was thrown while executing multiple calls to Keycloak API.

Caused by: javax.ws.rs.ProcessingException: RESTEASY004655: Unable to invoke request
at org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.invoke(ApacheHttpClient4Engine.java:287)
at org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.invoke(ClientInvocation.java:436)
at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientInvoker.invoke(ClientInvoker.java:102)
at org.jboss.resteasy.client.jaxrs.internal.proxy.ClientProxy.invoke(ClientProxy.java:64)
at com.sun.proxy.$Proxy276.findAll(Unknown Source)
at org.keycloak.admin.client.resource.ClientsResource$findAll.call(Unknown Source)
Caused by: java.lang.IllegalStateException: Invalid use of BasicClientConnManager: connection still allocated.
Make sure to release the connection before allocating another one.
at org.apache.http.util.Asserts.check(Asserts.java:34)
at org.apache.http.impl.conn.BasicClientConnectionManager.getConnection(BasicClientConnectionManager.java:160)
at org.apache.http.impl.conn.BasicClientConnectionManager$1.getConnection(BasicClientConnectionManager.java:142)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:423)
at org.apache.http.impl.client.AbstractHttpClient.doExecute(AbstractHttpClient.java:863)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
at org.jboss.resteasy.client.jaxrs.engines.ApacheHttpClient4Engine.invoke(ApacheHttpClient4Engine.java:283)

I was calling

keycloak.realm(realm).clients().create(representation) 

and did not read anything from response. Simple fix was

            def response = keycloak.realm(realm).clients().create(representation)
            response.close()

Problem with Kubernetes SkyDNS healtz

I had some problems when trying to get DNS working on Kubernetes. I followed instructions from https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/dns. Everything seemed to be working well, but the pod got restarted after 30 seconds. The log for healthz -container had following entries:

2016/03/19 04:25:25 Client ip 10.0.96.1:50326 requesting /healthz probe servicing cmd sleep 10 && nslookup kubernetes.default.svc.kube.local localhost >/dev/null

2016/03/19 04:25:25 Healthz probe error: Result of last exec: nslookup: can't resolve 'kubernetes.default.svc.kube.local': Name does not resolve, at 2016-03-19 04:25:23.967737423 +0000 UTC, error exit status 1

After trying a lot of things, I found a bug report for Alpine Linux. Basically, the nslookup does not respect the server parameter, if /etc/resolv.conf  has entries. Comment on that issues recommends using dig or drill for querying.

So I made a simple image and pushed it into docker hub, https://hub.docker.com/r/jyrki/arm-kubernetes-healthz-drill/. Nothing fancy, just added drill (https://github.com/jyrkiput/arm-kubernetes-healthz-drill/blob/master/Dockerfile). I used the existing image as base as I wanted to have the exechealtz available.

Then I had to change the healtz command to

drill -q kubernetes.default.svc.kube.local @localhost

Kubernetes 1.2.0 beta-1 not starting on Raspbian 8.0

While trying to start kubernetes v1.2.0 on Raspbian 8.0 I ran into problems. Only k8s-master and k8s-master-proxy containers were started so the system was not getting up properly. Logs for k8s-master were telling following:

7215 kubelet.go:2365] skipping pod synchronization - [Failed to start ContainerManager system validation failed - Following Cgroup subsystem not mounted: [memory]]

Cgroup memory subsystem is not enabled by default. You can enable it by adding

cgroup_enable=memory

into /boot/cmdline. Reboot is needed after this.

You can check if the memory subsystem is enabled by listing /sys/fs/cgroup/ which should the have directory called "memory" among others

blkio  cpu  cpuacct  cpu,cpuacct  cpuset  devices  freezer  memory  net_cls  systemd

 

Reveal.js with backgound image and company logo, all in css theme

I wanted to use reveal.js for my presentations. In our company, we have (as usual) a standard template for presentations. The problem was that the background had two parts: A small triangle in top left corner, and company logo in bottom right corner.

CSS3 supports having multiple backgrounds, so after a little tinkering I came up with following css snippets.

body {
  background-image: url('theme_images/corner.svg'), url('theme_images/sysart.svg');
  background-repeat: no-repeat;
  background-position: top left, bottom right;
  background-size: auto 30%, 20% 20%;
  }

The frontpage was some what different.

html.title body {
  background:url("theme_images/front.svg"), url('theme_images/sysart.svg');
  background-repeat: no-repeat;
  background-position: top left, bottom right;
  background-size: 100% auto, 20% 20%;
}

The whole css can be seen in https://github.com/sysart/reveal.js/blob/sysart-theme/sysart.css#L10

Getting project version from Maven project in Jenkins

Execute system Groovy Script with following script

import hudson.FilePath
import hudson.remoting.VirtualChannel

def pomFile = build.getParent().getWorkspace().child('pom.xml').readToString();
def project = new XmlSlurper().parseText(pomFile);      
def param = new hudson.model.StringParameterValue("MAVEN_VERSION", project.version.toString());
def paramAction = new hudson.model.ParametersAction(param);
build.addAction(paramAction);

Now you can use the "MAVEN_VERSION" in build, for example pass it on with "Trigger parameterized build on other projects" post build action by adding predefined parameters:

project_version=${MAVEN_VERSION}

Or in some shell commands

build.sh ${MAVEN_VERSION}

I've found this to be useful when one project deploys artefacts into repository, and another project wants to use those artefacts with exact version number.

Bash script for resolving Docker ports

Docker containers can expose ports to outer world when needed. This is done by giving "-P" flag to docker run -command. This will publish all exposed ports to "a random high port from the range 49000 to 49900" (from Docker userguide). Even though the user guide doesn't explicitly say that the docker daemon will track what ports are published, I would guess that it does.

Publishing to random ports is useful as then you can have multiple containers running at the same. We need this for our CI -setup. But there's also requirement for accessing container from outside, so we need to resolve the port published by -P in build scripts.

"docker inspect" is a command which can be used for this. It takes "--format=template" parameter, which can be used to output information about container.

So following bash script resolves public port for given container name and exposed port.

#!/bin/bash
set -o nounset
set -o errexit

function resolvePort() {
  local container=$1
  local exposedPort=$2
  local port=$(docker inspect --format='{{range $p, $conf := .NetworkSettings.Ports}}{{if eq $p "'$exposedPort'/tcp"}}{{(index $conf 0).HostPort}}{{end}} {{end}}' $container)
  echo $port
}

 The magic happens in

{{range $p, $conf := .NetworkSettings.Ports}}{{if eq $p "'$exposedPort'/tcp"}}{{(index $conf 0).HostPort}}{{end}} {{end}}

In easier to read format:

{{range $p, $conf := .NetworkSettings.Ports}}
    {{if eq $p "'$exposedPort'/tcp"}}
        {{(index $conf 0).HostPort}}
    {{end}}
{{end}}

The "{{range $p, $conf := .NetworkSettings.Ports}}" iterates over ports configuration. It is like map, and one key-value -pair looks something like this

"80/tcp": [
  {
      "HostIp": "0.0.0.0",
      "HostPort": "49101"
  }
] 

$p is the key and $conf is the value. $p is the exposed port and its value is something like "80/tcp".

Then there's {{if eq $p "'$exposedPort'/tcp"}}, which is simple comparison.

The value of $conf is a array, and in this use case, we just need the first value (also there is just one). So in

{{(index $conf 0).HostPort}}

(index $conf 0) gives just that, and then (index $conf 0).HostPort returns 49101.

This can be then used

readonly publicPort=$(resolvePort containerName 80)
curl localhost:$publicPort

We might have been able to avoid this by using another container for tests and doing container linking, but this seems to work okay. And I wanted to learn about docker inspect --format :)

Using HTTP Basic Authentication with YUM

HTTP Basic Authentication is supported by yum. You just have to add username and password into repository configuration

username=username
password=password

But, at least on the older versions of Centos, this does not work. Problem is that yum relies on python library called "urlgrabber" for connection. The version of this library that is available on repositories doesn't seem to working. You can see the packages in the repository with "yum info", "yum search" but you cannot install them.

I resolved this problem by installing urlgrabber from sources:

git clone git://yum.baseurl.org/urlgrabber.git/
cd urlgrabber
python setup.py install

That got it working.