Following psych0tik standard operating procedure, this was procrastinated and when I finally got around to doing it, we discovered magikarp had file system corruption and required a reboot and fsck.  With the magikarp splashing around ineffectively, we pointed irc.psych0tik.net to (a mostly unconfigured) Storm.  Shortly there after, we brought magikarp back online, hashed out the final issues with linking, and connected the two.

During magikarp’s reboot, our services database was corrupted and due to lack of backups (lazy.jpg), we’ve had to revert back to a version from about 6 months ago.  We apologize for this inconvenience.

With that all said, we do have a shiny new IRC node to connect to.  Now that Storm is back and functional, I’ll be working to restore some of the services that have been absent for the last few months.  This will include proxyElite and our GoogleSharing proxy.

Our recommended IRC domain, irc.psych0tik.net, has been re-pointed at magikarp and users who wish to use a US-based server (or otherwise desire to use Storm instead) should connect directly to storm.psych0tik.net.  As always, you’ll need to use SSL on port 6697 is required.

First I tried just rebasing it on master, but when that fell over in the same
way I decided to take a new approach. What I did is rougly analogous to a
rebase, with the exception that I didn’t have to lose any history (I’ll end up
with the original commits merged) but I also won’t create a ton of noise in my
logs.

I opened tig with the graph view of my branch and master tig origin/master features/tip_signing,
and then merged each commit by hand.

Having the diff that each commit had on the topic branch makes this easy, even
if the original commit was a PITA.

Once I finished merging, I was left with this monstrosity:


* 5194cc8 (HEAD, master) Merge commit '6e27da1' Richo Healey 8 seconds ago
|\
| * 6e27da1 (origin/features/tip_signing, features/tip_signing) Clean up the input form richo 2 weeks ago
* | 0fbf1f1 Merge commit 'c417108' Richo Healey 2 minutes ago
|\ \
| |/
| * c417108 Border around input bofixupx richo 2 weeks ago
* | d90c150 Merge commit 'a21bca6' Richo Healey 3 minutes ago
|\ \
| |/
| * a21bca6 Clean up styling richo 2 weeks ago
| * 2b4753d Give airship the option of a signing key richo 2 weeks ago
| * 1f0e7f3 Use good/bad colours from bootstrap richo 2 weeks ago
* | c1a0c50 Merge commit '1d4e710' Richo Healey 3 minutes ago
|\ \
| |/
| * 1d4e710 Send signature status to the client properly richo 2 weeks ago
| * 10db7fe Show signature status in the UI richo 2 weeks ago
* | 1d88e3a Merge commit 'f46ac6a' Richo Healey 3 minutes ago
|\ \
| |/
| * f46ac6a Squashed commit of the following: richo 2 weeks ago
* | 7e56994 Merge commit '2fa4d54' Richo Healey 4 minutes ago
|\ \
| |/
| * 2fa4d54 Script to sign grefs richo 2 weeks ago
| * 59d2004 Implement lookup of crypto adaptors richo 2 weeks ago
* | d2dede5 Merge commit '70bcce1' Richo Healey 5 minutes ago
|\ \
| |/
| * 70bcce1 Send signatures with responses in airship richo 3 weeks ago
* | 5de2154 Merge commit 'ce2327c' Richo Healey 7 minutes ago
|\ \
| |/
| * ce2327c Load crypto adaptor on station richo 3 weeks ago
* | aa627f0 Merge commit 'f76d976' Richo Healey 8 minutes ago
|\ \
| |/
| * f76d976 Nuke a ton of dead user code richo 3 weeks ago
| * 0e6e9fb Test deals with untrusted signatures richo 3 weeks ago
| * d4f2b35 Sanely marshall and store signatures richo 3 weeks ago
| * dee7b12 Tests for get signature richo 3 weeks ago
* | 2da7487 Merge commit 'ea46f19632003397a64650a4aa755416288dfcf9' Richo Healey 15 minutes ago
|\ \
| |/
| * ea46f19 Represent tips as (tip, signature) tuples richo 3 weeks ago
* | fb531e2 (origin/master, features/integration-testing/master) Test that we can send objects between nodes Richo
* | 11f6f12 (origin/features/integration-testing/master) Clean up after running tests Richo Healey 4 days ago
* | 2a5fb9a Simple test that stations can connect to one another Richo Healey 4 days ago
* | 73e0a7c Cleanup stream_client Richo Healey 4 days ago
* | ba7172a (features/socket_api) Test station init Richo Healey 2 weeks ago
* | 433ffc1 Bootstrap stations from environment richo 2 weeks ago
* | 2048da0 Merge pull request #23 from richo/bugs/ff-rendering Richo Healey 2 weeks ago


Once you’ve done this merge, you just need the name of the tree (Not the commit!) which you can get from git cat-file -p HEAD, which will emit something like:

Now it’s time to construct our “real” merge:

echo "Merge branch 'features/tip_signing'" | git commit-tree -p origin/master -p features/tip_signing 2efc6dee9eb621a1e95e88294228456d88d7790f

This will commit the final tree we arrived at, as a new commit that’s a child
of both master and our topic branch (a merge), but without the messy history.

As far as I know, the only live service depending on this machine is syncvid.info, which will probably either shutdown or be moved elsewhere, but no external disruption is planned. On the offchance you have some dependency we don’t know about- feel free to email staff@psych0tik.net

I decided to port groundstation to use bower for it’s frontend assets this evening. Technically, the component in question is airship, but that’s irrelevant. I need to pull in bootstrap, jquery, backbone and a few other bits and pieces and:

1. It’s a pain in the ass to stay up to date
2. Having all that code vendored screws with all my metrics

So off to bower land I go!

Cool, I can create a bowerrc with the target directory, and a component.json with some dependencies and run bower install and….

bower error 1 not found

There were errors, here's a summary of them:
- 1 not found

What the fuck is this. Oh, right, I can’t pass an array of dependencies in, it has to be a hash of dependency => version pairs. Cool. Pretty obvious from the error message right?

So I fix that, and run bower install and…

There were errors, here's a summary of them:
- jquery ENOENT, lstat '/Users/richo/.bower/cache/jquery/cf68c4c4e7507c8d20fee7b5f26709d9/.git/tags.3799'

At this point I’m fucking positive they’re just dicking with me. I run the same thing with –debug

There were errors, here's a summary of them:
- jquery ENOENT, lstat '/Users/richo/.bower/cache/jquery/cf68c4c4e7507c8d20fee7b5f26709d9/.git/tags.3799'

Obviously. Then you should see my jaw put a three foot divot in the floor when I notice that all of these commands exited 0.

Through trial and error, I manage to get a bit further to see:

There were errors, here's a summary of them:
- jquery ENOENT, lstat '/Users/richo/.bower/cache/jquery/cf68c4c4e7507c8d20fee7b5f26709d9/.git/tags.22124'
- underscore Didn't get expected byte count
expect: 667648
actual: 847834

and then again my jaw meets floor when I notice that It’s installing these assets anyway.

I’ve already opened a Pull request for the most trivial of these issues but seriously.. come on guys. This isn’t cool.

Chances are significant I’m going to roll my own ghetto make(1) based setup, because at least it does what it says on the tin.

I’ve been working on groundstation lately, which leverages git as it’s storage backend, but also assumes that all operations are threadsafe and concurrent. As such, I want to write testcases for it’s internal implementation that actually excercise these race conditions to assert their sanity.

Obviously, I can litter my code with optional callbacks, but that’s pretty much awful. I never thought I’d genuinely wish python had macros or a preprocessor. Do you know of a sane way to approach this?

This year I’ve decide to embark more on a quest to explore sleep, and the way I experience it personally. It’s something I’ve struggled with as long as I remember, and I figure that by gathering more data points I can at least start to unwind this puzzle.

A few days ago I was considering an experiment, and this post serves as a braindump to flesh out the idea, as well as hopefully a place to gather feedback and ideally hear from someone who’s tried something like this before.

The basic premise is that for a week I’d like to lose the circadian rhythm, sleep when I feel like it and until I wake up naturally, and only afterwards inspect the data.

This has some immediately obvious flaws, the first being that if I’m to lose all perception of time I’d have to black out my house and not go outside. I suspect that by not deliberately structuring my life anything specific this can be overcome.

In more detail, I plan to pull the clock off the devices I own and the interface for the software I’m likely to use in that time (my system taskbar, irssi and tmux all show my timestamps), and use my cli twitter client twat to tweet like a madman, giving my timestamped data points to look at afterwards.

There’s some planning to be done and some preliminary research to be done, but I’m hoping that at the very least it’ll be interesting, even if it serves only as a cautionary tale to others.

This evening, I had only my laptop with me: but “Not to worry,” I thought, “I’ll just light up my dev VM!”. At work we use vagrant to light up ondemand VM’s, bootstrap them with babushka and get on with it. We use some trickery in the vagrant-dns gem to make the VM addressible from the host, with vagrant taking care of NAT for us.

Which is where things got interesting. groundstation uses UDP broadcast to find it’s peers, which WILL penetrate most NAT configurations, but with the caveat that the source address will be rewritten- in this instance rewriting it to my external IP address, causing my daemon to attempt to connect to it’s “peer” and sync it’s objects- with itself.

Born of a discussion at work about how all existing solutions to issue tracking and project management pretty much suck, and my natural flair for finding the most awkward way imaginable to do something, I started building a framework for decentralised issue management, based on gossip protocols (full points to @wolfeidau for planting that seed in my head) and utilising git’s object store as a backend.

Right now all it can do is propogate git objects over the local network via broadcast discovery, but the TODO list is pretty significant. I’m planning to implement:

• Verification of changesets based on RSA/ECDSA cryptography (choosing those algorithms because of their ubiquity with developers)
• Implementation of an arbiter node making the broadcast discovery an effective local means, but not the only way to “sync” your events
• Better control over “channel” subscription to avoid the obvious DoS attacks that are trivial to exploit with the current implementation
• Better support for event based propagation, instead of polling as currently stands

… and if there’s time maybe a frontend or something so you can actually interact with it. Who can say.

You can try it out right now if you’re ok with not syncing any git objects other than blobs (the only primitive type that I’m planning to use in my tracker).

Clone the sources down on a few local machines on the same subnet,

pip install -r requirements.txt; python stationd

and watch whatever’s in

~/.groundstation

get synced up.

After playing with it briefly after lars sent it through, it looked like a much more flexible solution (and one that’d work on more platforms than just OSX, meaning I wasn’t entirely reliant on Mac like I had been).

I haven’t got much to report, beyond having implemented filebacked caching, resolution strategies, some privacy stuff (limit precision) and support for timeouts throughout.

Get doko 0.2.0 from it’s bitbucket repo, and again thanks to lars for spending the time on writing it!

At some point on the first night, it occurred to me that a ridiculous but potentially achievable feat might be to build a thing for every day I’m off work. I’ve already missed that goal, but keeping in spirit, I’ll be posting things as they come off the ranks. I’m starting late, and I’ve already gotten a few to a point where they’re worth talking about, so hopefully this will be enough filler to allow me to post daily until I go back to work. Which brings us to..

MapGit

A while ago I started geotagging my commits with a post-commit hook and a tool called whereami. I silently collected some (massively skewed- due to an oversight on my part) data for a while, and then remembered about it recently when lars whom I work with kindly offered to do some plotting magics in R.

He came up with this:

Which ultimately inspired me to start looking into this again and produce a more general solution.

I built http://mapgit.com in reasonably short order the next day. Right now it’s basically just a thin layer around redis which allows you to upload pairs of “commit location” pairs and have them transparently stored in redis.

Before the break is over I intend to built on the side of the github API to allow for selecting all the locations (both as a set and as a distribution) for a given rev-list or branch, fetched directly from github’s API or passed straight into mapgit, and export them in a format that R likes for easy plotting.

So far it’s quite barebones, but I’m happy with how it turned out for only a few hours invested.

Until tomorrow, Happy Haxmas all.