Featured Post

Tmux with Native Copy and Paste

This is a great tip and productivity enhancer.  You can now do native copy and paste with Tmux and OSX starting with Tmux 1.8

The guys over at thoughbot put this together and it's a good tip.

You'll use the new copy-pipe command to setup defaults in your tmux config. See the details in their post.

How do I install Elixir on a Mac?

How do I install Elixir on a Mac?

The easiest was is to use brew

$ brew install elixir 

How do I start Elixir?


$ iex

Rsync - a great tool

If you haven't used rsync yet you should. It's a great tool for mirroring

Vi paging up and paging down

In vi to page up and down type
Ctrl-b aka (^b)
Ctrl-f aka (^f)

End of line characters

How do I type the end of file character

The end of file character is different on different operating systems so the end of line aka eof is:
Control-D on Unix
Control-Z on Windows

How do I get UTC time in seconds in oracle sql

Timestamps can be confusing and difficult to work with in oracle. It
took me a little time to figure out but the following SQL will give
TO_DATE('19700101', 'YYYYMMDD')) * 86400 seconds , current_timestamp
FROM dual the UTC time in seconds from the epoch, as a floating point number. 60 * 60 * 24 for 60 seconds * 60 minutes * 24 hours = 86400

Perl loop through days code

Here is some code for looping through days

#!/bin/env perl
use DateTime;
my $start_dt = DateTime->new(year => 2009, month => 4, day => 27);
my $end_dt = DateTime->new(year => 2009, month => 5, day => 18);

my $weeks = 0;
for (my $dt = $start_dt->clone();
$dt <= $end_dt;
$dt->add(days => 1) ) {
print $dt->date('')."\n";

Vi paging up and paging down

In vi to page up and down type
Ctrl-b (^b)
Ctrl-f (^f)

How to escape single quotes bash

Just use my web page that I setup to escape single quotes in bash

sbcl lisp access command line arguments

$ sbcl --eval '(progn (print *posix-argv*)(quit))' two three

merge multple json files as object values with jq

There are a number of example show you how to merge objects or combine arrays with jq.  I haven't found any examples of merging json files in a single object.  Here is how you would do that on the command line with jq.

jq -n '{one:$variablename1, two:$variablename2}' --slurpfile variablename1 filename_one.json --slurpfile variablename2 filename_two.json

Install a single node Yarn / HBase Cluster with a single command

Setting up a cluster can be time consuming especially with all the services: (Hbase, Zookeeper, Yarn, Hive, Oozie, Ambari, HCat, HDFS, WebHCat)

Use this script and set it up with a single command.

Setup a Single Node Hadoop 2 Cluster with a Single Command

Setup a hadoop 2 cluster with a single command:
curl -sSL | bash -s -- -r For development purposes we wanted a easy way to setup an environment that's been tested everywhere we work:

VagrantAWSDigitalOcean This makes things much easier: Feel free to help add compatibility to other linux distros

Good read on HDFS small file compaction:

With that decided, we then looked for options to aggregate and compact small files on Hadoop, identifying three possible solutions: filecrush - a highly configurable tool by Edward Capriolo to “crush” small files on HDFS. It supports a rich set of configuration arguments and is available as a jarfile (download it here) ready to run on your cluster. It’s a sophisticated tool - for example, by default it won’t bother crushing a file which is within 75% of the HDFS block size already. Unfortunately, it does not work yet with Amazon’s s3:// paths, only hdfs:// paths - and our pull request to add this functionality is incompleteConsolidator - a Hadoop file consolidation tool from the dfs-datastores library, written by Nathan Marz. There is scant documentation for this - we could only find one paragraph, in this email thread. It has fewer capabilities than filecrush, and could do with a CLI-like wrapper to invoke it (we started writing just such a wra…