Apr 07
Cloudera Manager is fairly opinionated. In its defence, it pretty much needs to be given that it needs to wrangle multiple underlying Open Source projects. Each of these, in turn, have their own quirks and opinions. The following is a description of how to recover a Cloudera Manager cluster post disaster, assuming that you have …
Mar 30
I’ve been rewriting a cleanroom version of the hadoop-in-a-box — just about finished. And, truth be told, the code, all in all, is a bit tighter than the original encumbered version. However, I ran into an interesting feature of Volumes — I had thought perhaps to optimize things a bit, but it caused some unexpected …
Mar 17
TL;DR: Java and cgroups/Docker memory constraints don’t always behave as you might expect. Always explicitly specify JVM heap sizes. Also be aware that kernel features may not be enabled. And Linux… lies. I’ve recently discovered an interesting “quirk” in potential interactions between Java, cgroups, Docker, and the kernel which can cause some surprising results. Unless …
Mar 15
TL;DR — When using AUFS in a memory constrained environment, Java can spawn (lots!) of Zombies. A workaround is to change the storage driver to the device mapper. In working on the Hadoop in a box CDH cluster with Cloudera Manager, I’ve discovered a few interesting things about AUFS. These experiences are with Ubuntu 14.04 …
Mar 10
While learning how the configuration worked — in particular which arguments to pass in order to set non-default values, I discovered that I could lose changes by following these steps: Use the GUI to set a value and save it. This is just so that you can find the variable. Keep the GUI open. Dump …
Feb 11
I had an instance where the docker filesystem ran out of space. After cleaning up with docker rm and docker rmi, I got:
|
1 2 |
Could not find container for entity id |
I followed iwidanalage‘s suggestion from “Creating fail with Could not find container for entity id after upgrading to 1.9.0 · Issue #17691 · docker/docker”
|
1 2 3 4 5 6 7 |
cd /var/lib/docker sudo sqlite3 linkgraph.db update edge set name='some-random-container-name' where name='failing-container-name'; .quit |
After this, it worked.
Jan 28
If you ever see a message like:
|
1 2 |
could not open session |
on startup and the Cloudera manager won’t start, it’s because you did not start the container in privileged mode:
|
1 2 |
docker run -d --hostname=master --privileged=true --name=master cdh-master |
Oct 09
I’ve been working with disk images from the excellent crew from Hypriot and there are times when I wish to mount the filesystems within to inspect and or make alterations to them. Since there are multiple filesystems, it’s not like mounting an ISO image — there’s more work which needs to be done. Being a …
Oct 08
I was asked an interesting question — how to create a regular expression which would allow a user to specify an arbitrary case insensitive series of strings, all of which must be on a line, as well as a series of strings which must not be on a line in order to filter logs. In …
Oct 06
Ruby’s Enumerable module provides a couple of methods for stepping through an enumerable with the index of the element as well. However, a couple are missing — there is no inject_with_index or map_with_index. There are a couple of ways of remedying this.
© 2008-2016 by Matthew K. Williams
Ramblings by Matt Williams is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Based on a work at http://matthewkwilliams.com.
Powered by WordPress and the Graphene Theme.
