Problem and Solution: Bundle install failed with fatal: Could not parse object

If you specify a gem with a github url and branch in your Gemfile, you can occasionally run into the following problem:


Fetching https://github.com/DavyJonesLocker/client_side_validations
fatal: Could not parse object '261964fdec8051e5d55f85e9074ed77be555e8a5'.
Git error: command `git reset --hard 261964fdec8051e5d55f85e9074ed77be555e8a5` in directory
/.../vendor/bundle/ruby/2.3.0/bundler/gems/client_side_validations-261964fdec80
has failed.
If this error persists you could try removing the cache directory
'/.../vendor/bundle/ruby/2.3.0/cache/bundler/git/client_side_validations-e290eb7b61ac375e1849a12ab45a9444d029fd93'

You will scratch your head because the branch is definitely available on github, so what gives?

Removing the cache directory doesn’t change the outcome either.

Solution: The root of the problem is that Bundler saves the last commit ID of the branch in Gemfile.lock, and next time you try to run bundler install it will try to pull the same commit ID.

IF the repo owner has removed the last used commit, say by merging it, git won’t be able to pull it any more, and gives up, blaming the local cache directory instead of the local Gemfile.lock

To solve this, run ‘bundle update’ which will ignore the contents of Gemfile.lock and refreshes it.

j j j

Diversity Hiring and the Startup Manager

If you are reading this, you are most likely in a managerial position at a small company, and you are thinking about diversity and what you can do about it. You are already doing the work of multiple people and you are realizing you are the bottleneck in the company. You decide to hire someone.

You are smart, scrappy and capable and you know a lot of people who are a lot like you – you speak the same language, you have similar backgrounds and experiences. You are also trying to make a risky business succeed so you are trying to make very safe decisions in hiring.

In fact, you only know a few token people in your industry that are visibly different from you – women, minorities, people with disabilities. They are very smart, successful and you have no way of hiring them – they are so good they are out of your league.

Your conscience dictates to try supporting diversity so you set out to cast a wider net for your next candidate – advertise on Monster and Dice and Indeed and LinkedIn and resumes start pouring in.

Interestingly the candidates that have minority-sounding names are not the most qualified candidates – and you can’t risk the success of your company on someone who cannot hit the ground running.

The ones that pass muster look sufficiently different from you that you are afraid to crack a joke in front of them – and then you reject them based on “culture fit” or “potential communication issues”.

At the end you lament the fact that there are no minorities worth hiring in the pipeline and hire the people who resemble you again.

Congratulations – you are the reason there is no diversity in the workplace.

If you are willing to accept this, here is how you can address your issues:

1. YOUR FEAR OF INSUFFICIENT KNOWLEDGE: Think about your own attributes instead of your accomplishments, then try to find the attributes in your next employee. You know you have a good base understanding of technology, and you could pick up new technologies fast – so look for people who demonstrate a good understanding of the basics and who show that they can understand new concepts, instead of past accomplishments.

2. YOUR FEAR OF INSUFFICIENT SELF-LEADERSHIP: Regardless how how egalitarian you think you are, remember that you have almost complete control over your employees livelihood – you get to decide if they can afford to pay rent next month or not. They are not going to have the same “ownership” in the company you do. But they will try their best to meet your expectations – if your expectations are very clear.

3. YOUR FEAR OF WASTED TIME: You, the hiring manager, need to put in the time to support and build an employee. Even if you hire your exact copy, they need communication, feedback, support, guidance, direction, without which they will flounder. So dedicate serious time to support your employees so they can flourish under you.

4. YOUR FEAR OF SPECIAL TREATMENT: Don’t try to make everybody equal. Different people have different needs, and situations change – children, illness, family issues. Commit to supporting special needs as they arise, and your employees will also support your changing needs.

5. YOUR FEAR OF CONFLICTS: Conflicts are normal and common in human life. Competing priorities, missed communication, time constraints should not be considered surprises. Discuss hard things in private, and focus on getting a workable outcome. We are all people first.

j j j

Tutorial: How to create the smallest possible Ubuntu Docker image with apache, nginx, python, php, java or anything else you want in it

You must have read the tutorials that start with “docker pull ubuntu:14.04”, continue with apt-get update, and after a couple of apt-get installs end up with a docker image larger than a gigabyte.

There is a much leaner way – using Ubuntu as the host operating system and pulling in only the binaries and libraries that you will use.

The results are impressive: my last apache/python image was 1.6GB, but using the following method I ended up with a 0.3GB image.

The trick is that you can copy the executables into a docker image as long as you also copy the system libraries they depend on. To find out what libraries an executable depends on can be queried with the ldd command:

$ ldd /usr/sbin/apache2

linux-vdso.so.1
libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3
libaprutil-1.so.0 => /usr/lib/x86_64-linux-gnu/libaprutil-1.so.0
libapr-1.so.0 => /usr/lib/x86_64-linux-gnu/libapr-1.so.0
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1
libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1
libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
/lib64/ld-linux-x86-64.so.2

Apart from linux-vdso.so.1, a virtual library that the kernel emulates, all other libraries are actual files that can be copied into the Docker image. linux-vdso.1 will be taken care of by the kernel.

Instructions

You have to build your image on an Ubuntu host. I still run Ubuntu 14.0.4 LTS. Make sure you keep it up to date with apt-get update and apt-get upgrade!

Install your apache/mod_wsgi/python server on your host, if it works correctly on its own it will most likely work correctly in your container as well.

NOTE: remember to stop your webserver on your host before starting up your docker container – you don’t want the host’s webserver to conflict with the container’s webserver!

We will use a 5MB docker image as base that supports Ubuntu glibc and conveniently has busybox and an entire init system built in: busybox:glibc

Then we will create a directory and copy all our files and dependent libraries into it from our host system.

Finally we run docker build to create our new image.

1. Create the build directory and build config

Create a directory for your new build where your Dockerfile and all other files will reside.

touch Dockerfile
mkdir root

Your Dockerfile will be simple:

FROM busybox:glibc
COPY root /
// these are from /etc/apache2/envvars
ENV LANG C
ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
ENV APACHE_LOG_DIR /var/log/apache2
ENV APACHE_LOCK_DIR /var/run/apache2
ENV APACHE_PID_FILE=/var/run/apache2/apache2.pid
//
CMD /usr/sbin/apache2 -D FOREGROUND

2. Copy the executables and config files into build directory

First the executable:

mkdir -p root/usr/sbin
cp -a /usr/sbin/apache2 root/usr/sbin/

Then the loadable modules:

mkdir -p root/usr/lib/apache2
cp -a /usr/lib/apache2/modules root/usr/lib/apache2/

Then the configuration files:

mkdir -p root/etc
cp -a /etc/mime.types root/etc/
cp -a /etc/apache2 root/etc/

Then the html directory:

mkdir -p root/var/www
cp -a /var/www/html root/var/www/

3. Copy the library dependencies into build directory

This is the section that makes people stay away from hand-building Docker images: library files look arcane at first, but they are pretty straightforward.

First a simple one-line script to find and copy the dependencies of all executables in the build directory:

    mkdir -p root/lib
    for i in `find root -type f -executable | xargs ldd | grep -v "linux-vdso" | grep "=>" | awk ' { print $3 } '`; do
        cp -a $i* root/lib/
    done

Apache has some loadable modules that are not executable but they still pull in other libraries. A slight modification of the above script pulls those libraries in as well:

    for i in `find root/usr/lib/apache2/modules/ -type f | xargs ldd | grep -v "linux-vdso" | grep "=>" | awk ' { print $3 } '`; do
        cp -a $i* root/lib/
    done

Then make sure we copied the actual libraries not just the symlinks pointing to them:

    for i in `find root/lib -type l`; do
        if [ ! -e "$i" ]; then
           missing=`readlink $i`
           cp `find /lib -name $missing` root/lib/
        fi
    done

And finally the one missing library that somehow still got out:

    cp -a /lib/x86_64-linux-gnu/libgcc_s.so.1 root/lib/

4. Add a few missing directories

mkdir -p root/var/log/apache2
mkdir -p root/var/run/apache2

5. Build the image

This is the easiest part:

    docker build --rm --no-cache -t tiny-apache:latest .

6. Test the image

We run the image interactively to see all error messages and use net=host to skip having to specify port mapping. Of course you can specify port mapping if you prefer.

    docker run -ti --net=host -P tiny-apache:latest

The resulting apache Docker image is 21 megabytes. The equivalent ubuntu image is 233 megabytes.

Where to go from here

I use these instructions to build and debug mysql, nginx, redis, elasticsearch and other docker images.

I prefer to combine programs that depend on each other in the same container, for example I run nginx, gunicorn, celery and cron in one container. For this I use the busybox runit init system and I start runsvdir as the main command that starts everything else.

For logging I simply map my host’s syslog socket /dev/log into /dev/log inside the container as a volume: -v /dev/log:/dev/log

If you have any questions, ping me on twitter: imreFitos

j j j

Node NPM install fails with “Error: Cannot find module” – Solution

There are 187 thousand results on Google about this npm install error “Error: Cannot find module” and pretty much all responses say the say the same “delete your entire node installation.”

You might have an error like this:

> node install.js

module.js:328
throw err;
^

Error: Cannot find module 'readable-stream'
at Function.Module._resolveFilename (module.js:326:15)
at Function.Module._load (module.js:277:25)
at Module.require (module.js:354:17)
at require (internal/module.js:12:17)
at Object. (/usr/lib/node_modules/phantomjs-prebuilt/node_modules/extract-zip/node_modules/concat-stream/index.js:1:78)
at Module._compile (module.js:410:26)
at Object.Module._extensions..js (module.js:417:10)
at Module.load (module.js:344:32)
at Function.Module._load (module.js:301:12)
at Module.require (module.js:354:17)
npm ERR! Linux 3.13.0-32-generic
npm ERR! argv "/usr/bin/nodejs" "/usr/bin/npm" "-g" "install" "[email protected]"
npm ERR! node v4.3.0
npm ERR! npm v2.14.12
npm ERR! code ELIFECYCLE

npm ERR! [email protected] install: `node install.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] install script 'node install.js'.
npm ERR! This is most likely a problem with the phantomjs-prebuilt package,
npm ERR! not with npm itself.

I took the time to actually troubleshoot the error and found that it comes to file and directory permissions – npm can install the dependent modules as root, change the permissions and then unable to open them again!

Solution: You can fix the issue by changing the directories and files in /usr/lib/node_modules to be allowed to be read by everybody on your system:

find /usr/lib/node_modules -type d | xargs chmod go+rx
find /usr/lib/node_modules -type f | xargs chmod go+r

j j j

Docker build could not resolve ‘archive.ubuntu.com’ when using custom bridge

If you change your docker bridge IP range, Docker’s network configuration description says deleting your bridge and creating a new one will take care of resetting the IP addresses.

Unfortunately docker won’t update the outgoing network traffic masquerade setting if it sees an existing entry in the iptables nat table.

You can verify this by using the ‘iptables -L -t nat -n’ command which will show the default IP range still configured.

You will have to manually flush the PREROUTING table contents using the ‘iptables -t nat -F POSTROUTING’ command. After running this, restart docker and it will initialize the nat table with the new IP range.

j j j

Simple syslog setup for Docker

There are a lot of complex posts about logging from Docker containers – but there is a much simpler way.

On Linux, your syslog daemon, be it rsyslog, syslog-ng or the original bsd syslog, accepts messages via the /dev/log socket file.

If you want your app running inside your container to deliver log messages to the syslog daemon running on your host, just share /dev/log as a volume:

docker run -v /dev/log:/dev/log

And all messages sent to syslog inside your container will be sent to your main syslog daemon.

j j j

List of Hexadecimal Baby Names

For prospetive geek parents, here is the list of baby names that only contain the letters A B C D E F:

Ada
Becca
Bea
Eda
Fae
Cadee
Abcde
Dea
Febe
Bae
Edda
Adea
Dae
Bebe
Cece
Ebba
Edee
Cade
Ace
Abe
Dade
Efe
Dace
Dee
Cabe
Cace
Ab
Abba
Ed
Ac
Acea
Abed
Bae
Ebaad
Ebba

There are at least five kids born in the USA in 2013 who have these names, according to the SSA: http://www.ssa.gov/oact/babynames/limits.html

If you want leet-speak names, there are 120 of them:

Ad4
Abb13
Add13
A1d4
Ed13
Ac4c14
B3cc4
A01f3
B0bb13
C0c0
D14
B0bb1
Ac4d14
Ad14
Eff13
Abb1
A1d3
Ad1
C414
D3bb13
B0
C0d1
Add1
B34
Ed4
F14
C41
C0b13
C0d13
F43
C4d33
Ab1
Af14
Abcd3
C4d13
A1d33
C1c1
D34
Ab14
A01
D4c14
D4c13
A1
F3b3
B43
Edd4
Ad34
B1b1
B03
C3c14
C0b1
D43
A4d1
Ab1d4
Ad41
Ad1b4
B3b3
C3c3
D044
Ac13
Ad13
A14
C0d4
C0d33
D4c1
Ebb4
Ed33
Ed1
C4d3
Ac3
Edd13
B0
B0d13
B0d3
A4d1
Ab3
C41
F4b10
B03
C0d4
C410
D4d3
Ad1
B0d33
C0b3
Abd1
B0b
B40
B0d1
Ef3
C0d1
F4d1
D4c3
C0d13
D33
C0b1
C0b13
D0c
Ab1d
C4b3
C4c3
Ab
Abb4
C41d3
D30
Ed
Ac
Ac34
Ac13
Ad1b
C1d
C03
D10
Ed1
Ab3d
B43
C01
D4c0d4
Eb44d
Ebb4

j j j

Docker search images and tags in private registry

Here is how you can get a list of all docker images and all tags for each docker image using curl.

Docker as of 1.1.2 hasn’t yet implemented searching in private registries, although docker-registry supports it.

To list all images in a registry:

curl https://your.docker.server/v1/search

To list all tags for a given image in a registry:

curl https://your.docker.server/v1/repositories/YOUR_IMAGE/tags

H/T to sontags: private registry search

j j j

Docker private registry Error: certificate signed by unknown authority

Docker supports private registries and there are a few writeups on how to setup a private Docker registry.

You can switch docker to use your local registry with the “docker login” command:

docker login -u httpuser -p httppassword -e [email protected] https://docker.yourcompany.com

Since you run a private registry you most likely use a self-signed certificate. Docker insists on checking your certificate against a Certificate Authority.

If you are used to OpenSSL and put your CA certificate in /etc/ssl/certs and created a hash link and it still doesn’t work, here is the solution:

Docker is written in go, go looks up the CA certificates in the following files:

    /etc/ssl/certs/ca-certificates.crt
    /etc/pki/tls/certs/ca-bundle.crt
    /etc/ssl/ca-bundle.pem
    /etc/ssl/cert.pem
    /usr/local/share/certs/ca-root-nss.crt

Go crypto source reference

You have to attach your CA cert to one of those files as well.

H/T to Jérôme Petazzoni

j j j

Time does change

Time does change. Our common (civil) time is defined as a full rotation of our planet relative to our sun, but Earth wobbles a bit so every day is a little bit longer or shorter, and we are generally slowing down, ever so slowly.
The International Earth Rotation and Reference Systems Service publishes the Earth’s daily rotation speeds, and they decide when we should have leap seconds snuck into the official clocks of the world to account for the wobbliness.
If you want to learn more about leap seconds, there is a great article about them here: http://queue.acm.org/detail.cfm?id=1967009
j j j