Oct 29

First do no harm

If you can, help others; if you cannot do that, at least do not harm them. — Dalai Lama

Over the past couple of years I’ve been trying to put into words my thoughts regarding the type of company where I’d like to work. There are companies I would not consider working for — their ethics and/or business model are radically different from what I consider good or right.

“Evil begins when you begin to treat people as things.” — Terry Pratchett, I Shall Wear Midnight (Discworld, #38)

  • I don’t want to work for a company which takes advantage of others.
  • I don’t want to work for predatory companies or ones which don’t treat their employees well.
  • I don’t want to go home at the end of the day feeling slimy or otherwise compromised.
  • I don’t want to work for a company which treats people as things.

I think technology really increased human ability. But technology cannot produce compassion. — Dalai Lama

I’ve had the fortune to work at a not-for-profit for the last nine years. Not being profit-driven means we’re not out to gouge people. Not that there’s anything wrong with seeking a profit, but seeking profit for the sake of profit leads to evil.

At the same time, without the relentless drive for profits it’s easier to focus on customers and service.

  • I don’t want to be associated with a company that views profits more valuable than people.

Don’t be evil. We believe strongly that in the long term, we will be better served — as shareholders and in all other ways — by a company that does good things for the world even if we forgo some short term gains. This is an important aspect of our culture and is broadly shared within the company. — Larry Page, Sergey Brin, Google IPO

A lot of people have scoffed at Google’s ideals in recent years. Nevertheless, it’s an admirable goal. I wish that more people or companies chose not to be evil in their day to day interactions with others.

The Agile Manifesto has brought a lot of change to Software Development. Other fields have recognized its inherent value and rewritten it to fit their circumstances.

I’d like to present a first draft of a manifesto for business.

A Business Manifesto

We are uncovering better ways of running a business and helping others do it.

Through this work we have come to value:

  • People and interactions over profits and prestige
  • Quality service over quantity of service
  • Customer relationships over contract negotiation
  • Flexibility over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

In a nutshell I want to work for a company which values people — both inside and out of the company. I want to work where people strive to do things right.

Trust yourself. Create the kind of self that you will be happy to live with all your life. Make the most of yourself by fanning the tiny, inner sparks of possibility into flames of achievement. — Golda Meire

At the end of the day, I want to work for a company which isn’t evil. When I go home, I want to be able to look in the face of my daughter and not have to make excuses for the work that I do and the effect it has on others. And I want to be able to look myself in the mirror, too.

Be excellent to each other and Party On Dudes — Abraham Lincoln in Bill and Ted’s Excellent Adventure

Sep 28

Go executables are statically linked, except when they are not.

Generally GO executables are advertised as statically linked.

And they are, mostly, except for those times where they aren’t.

This is about one of those times, and what I discovered in the process.

Read the rest of this entry »

Sep 16

Docker Volumes Quirk

I discovered something interesting today regarding Docker and volumes — there’s a problem below with the call to run the private docker registry container:

docker run -d -v /opt/registry-cache/:/tmp/registry -p 5000:5000 registry

If you run that and curl -s http://localhost:5000, nothing is returned.

As it turns out, the trailing slash in /opt/registry-cache/ causes issues — the web proxy starts up, but the actual registry doesn’t run. In order to get it to work, the following needs to be done:

docker run -d -v /opt/registry-cache:/tmp/registry -p 5000:5000 registry

It’s amazing the difference a single character can make. Remove the extra '/' and it works as expected.

Jul 27

Using Openstack/Devstack Floating IPs from outside

I hope that this is of use to someone. After many hours of tearing down, building up, programming routers, etc., I’ve figured out why my devstack wasn’t allowing access from floating ips. I knew that the address was resolving (arp -a); I just couldn’t ssh to it (or do anything else with it)

It was the security rules.

So, I went in to the security rules in horizon and added the following rules to the default:

  1. Allow all CIDR ICMP traffic
  2. Allow all CIDR TCP traffic
  3. Allow all CIDR UDP traffic

Generally I wouldn’t advise opening it up totally — I’d open up applicable ports.  However, I’m happy to have it working!

Aug 01

Cloudy Update

I’ve been working with Docker a good bit and have updated my list of tools.  Here’s a quick dump of where I am in the design of the infrastructure.

  • collectd will be used to monitor cgroup statistics.  This necessitates compiling all or part of collectd — the current packages do not contain the cgroup plugin. This will run in the hosts which run the containers.  Additionally, information about the hosts will be collected.  Thresholds will be used to send alerts, scale up or down services, etc..  Graphite or some other tool will provide graphs for the dashboards.
  • A log aggregation tool will capture logs from the various containers.  I’m considering logstash due to the large number of inputs which are already defined.  OpenTSDB is another option — it looks like it may be more poweful in some senses, but more difficult to configure in others.  My main concern with both is that they’re java based and in the case of logstash it requires a java collector running in each container and even though it’s a default sized jvm, assume it will require ~128 mb for each container.  That adds up quickly and I’d rather have something lighter.  I’ve not done enough research on OpenTSDB to speak to what it uses.
  • Dynamic DNS will be used to register the various services.  At present I am leaning toward Power DNS using PyPdnsRedis as a pipe backend  and redis to store the dynamic data.
  • Nginx will sit in front of hipache which is a dynamic proxy/load balancer which uses redis. I have not decided whether to use the node flavour of hipache or the one embedded in nginx — that needs to be tested. Calls to the services will be routed through the proxy.
  • An image repository will be available for local hosting of images.
  • A web-based front end to configure hosts, services, and provide a dashboard view.  Given the many other pieces which are using redis, I am seriously contemplating using it to store the data used to define services and hosts.  I’m not 100% sure of this yet, however.
  • Services consist of a particular process, such as a restful service running in a web server, or a jvm, or …. In their definition, the following information will be stored:
    • The name of the service
    • Ports which need to be exposed
    • Whether the service is active
    • The image
    • Dependencies – particularly services which need to be running prior to the start of this service
    • The minimum and maximum number of instances of the service
    • Threshholds
    • System requirements (cpu, memory)
    • Heartbeat – this is in addition to the thresholds
    • There may be a sort of inheritance to help cut down on duplicate information.
  • Hosts run the docker daemon and host containers.  Their information consists of:
    • Name of the host
    • IP address (may be dynamic)
    • Is the host available
    • Does the host need to be started (is it out in the public cloud)
    • Any cloud information needed to start it.
    • Priority – this determines the order in which containers and services are started – I anticipate that private hosts will have priority over public cloud hosts — due to expense; it makes sense to have overflow go to the public cloud.
    • Server specs
    • Currently available resources — this can be grabbed, in part from collectd.
  • There may be a discovery process, akin to the old Sun Jini project whereby a service can advertise itself and other services can utilize that service.  I could see this being used, for instance, if a service needs a cache.  Databases likely would not be as useful.

A picture will follow soon.  However, a good bit of the work’s already been done — where possible I’m integrating existing tools and projects.  Obviously the front end will need to be written.

I’ve decided that I’m going to repurpose my nimblestratus project on github — it’s not like I’d really done much of anything with it.  Unfortunately someone registered nimblestratus.com two weeks ago, but .org is still available.

I’m pretty excited, though — the pieces/parts are coming together in my head and I believe that this is do-able in fairly short order.  I really, really want to have a simple proof of concept done in the next couple of weeks, or by the end of August — I’m going to Cloud Develop and would love to have something for a “show-and-tell”.

Jul 23

On demand containers

I’ve been interested in Linux Containers for quite a while; I think that they have their sweet spot where they are better than virtual instances — in particular they require less resources.

I am working on a framework to use linux containers for on-demand computing — increasing or decreasing instances of applications as needed.  I envision it being used for things such as:

  • JBoss or other application servers
  • Internet Applications
    • Web
    • Rails
    • node.js
  • Databases
  • Caches
  • And more

At this point I’m planning to use the following:

  • Docker
  • HAProxy
  • TBD Monitor
  • TBD Management (probably written by my self)

At this point, I think the key is in being able to dynamically assign units to HAProxy.  HAProxy: Reloading Your Config with Minimal Service Impact describes a method for activating changes to HAProxy.  Unfortunately there isn’t a good API provided by HAProxy to add/remove hosts as necessary.  I’m wanting to be able to support both public and private clouds, with the idea being to have the ability to add resources from public clouds when private cloud resources are depleted.

Why am I doing it?  For one, well, it’s interesting to me. For another I haven’t heard of any open source tools/frameworks which do this.  Additionally I’ve seen many instances where virtual instances have been over provisioned or sit idle wasting resources, particularly when jvm’s are running inside a virtual instance.

There’s still a number of unknowns, but I think it’s workable.

Jun 22

cygwin and torquebox and rvm, oh my!

rvm, despite being wonderful, doesn’t play very well with torquebox under cygwin. In particular, the gem paths are not working. So, in order to fix this, simply do:

rvm use system

And then it will work right. Once you’re done with the torquebox work, you can go back to using rvm.

Nov 04

JBoss Client Jars for Messaging

Prior to JBoss 5, the jboss-all-client.jar was pretty much all you needed. However, the JBoss 5 Getting Started Guide states the following:

The client/jbossall-client.jar library that used to bundle the majority of jboss client libraries, is now referencing them instead through the Class-Path manifest entry. This allows swapping included libraries (e.g. jboss-javaee.jar) without having to re-package jbossall-client.jar. On the other hand, it requires that you have jbossall-client.jar together with the other client/*.jar libraries, so they can be found.

In order to access JBoss Messaging from a remote client, you need the following jars in the client’s CLASSPATH:

  • $JBOSS_HOME/client/jnp-client.jar
  • $JBOSS_HOME/client/jboss-javaee.jar
  • $JBOSS_HOME/client/jboss-messaging.jar
  • $JBOSS_HOME/client/jboss-remoting.jar
  • $JBOSS_HOME/client/jboss-serialization.jar
  • $JBOSS_HOME/client/javassist.jar
  • $JBOSS_HOME/client/jboss-aop-client.jar
  • $JBOSS_HOME/client/trove.jar
  • $JBOSS_HOME/client/log4j.jar
  • $JBOSS_HOME/client/jboss-logging-spi.jar
  • $JBOSS_HOME/client/jboss-logging-log4j.jar
  • $JBOSS_HOME/client/jboss-common-core.jar
  • $JBOSS_HOME/client/jboss-mdr.jar
  • $JBOSS_HOME/client/concurrent.jar

Aug 04

rsh hates nohup

rsh does not like nohup — rather than returning, as one would expect, it just hangs.

This has to do with stdin, stdout and stderr.

The short of it is….

By preference use ssh.

If you must use rsh, make sure that stdin, stdout, and stderr are all redirected properly.

Jun 09

Torquebox and Cygwin: Take I

Torquebox and Cygwin don’t work as nicely together as one might hope.

That said, here are a couple of findings:

  1. In order to deploy, you need to set the $JBOSS_HOME with the Windows path.  You can do this via export JBOSS_PATH=cygpath -w PATH_TO_JBOSS.
  2. Additionally, the JRUBY_HOME needs to be a windows path as well.   Otherwise you will see:

    org.jruby.exceptions.RaiseException: no such file to load — date

That’s it so far, more to follow as discovered….

Older posts «