This server (the very one you are reading this post on), at the time of writing this post, runs OpenBSD 6.1-stable. It’s fully patched and updated and everything, so it’s a perfectly fine OS to run. But the VPS has limited memory and disk space, and the CPU isn’t very fast, so compiling large projects on it, especially Haskell ones, is impractical.
This post describes a way to build fully-functional, dynamically linked (so you get all those security updates, super important for a public-facing web service), native Haskell binaries with cabal-install, Stack and…OpenBSD’s (kind of) new native hypervisor, vmm.
I recently got an old Sun Ultra 5 working. It wasn’t too difficult, but I needed to dig up a few old serial cables…
It already had SunOS 5.8 installed, but I put OpenBSD 6.1 on it, since I need a modern OS to actually do anything with it.
Migrating mail servers is a tricky business, especially when one server doesn’t have IMAP set up. The easiest way is to download all the mail and reupload it to the new mail server.
This seems simple enough, but I ran into problems. After all, I was going from an IMAP server somewhere to a maildir (no IMAP sync tool supports mbox for some reason) to an mbox through procmail to a directory of mboxes. Not trivial.
There I was, a loyal user of http://mail.zoho.com, when I decided to download all of my emails. For archival purposes, you know.
So I fired up mbsync, set everything up, let ’er rip, but after only about 10,000 emails downloaded, I got this error:
IMAP command 'AUTHENTICATE PLAIN <authdata>' returned an error: NO [ALERT] Your account is currently not accessible via IMAP due to excessive usage. Kindly try after some time. *** IMAP ALERT *** Your account is currently not accessible via IMAP due to excessive usage. Kindly try after some time.
What sort of…anyway, this was unacceptable, so I decided to set up my web server as a mail server.
Before I get my Sun Ultra 5 working and can write something about that, I thought I’d go through all the hardware I’m using right now and the OSes I’m running on them. Spoilers: it’s all OpenBSD and Debian.
You may have heard of the Star Trek script search tool at http://scriptsearch.dxdy.name. I’m writing a similar thing for Stargate. The difference is, of course, is that my tool will be running on some crappy Amazon AWS t2.nano with no RAM.
The first prototype was written in Python, but the parsing code was always written in Haskell (Parsec is great). I decided to move everything into Haskell so there wouldn’t be this redundancy of parsing in Haskell then serializing to disk then reading it from Python…one codebase for everything would be much simpler.
I wrote up a quick Haskell version, but there was one small problem when I tried to use it:
$ ./stargate-search Setting phasers to stun... (port 5000) (ctrl-c to quit) Killed
That’s right, the OOM killer had to step in and put my app down like the sloppy wasteful piece of junk it was.
How could I fix this?
So I was setting up a laptop I had just picked up, when it came to deciding what OS to install on it. Obviously, I’d probably end up installing some Linux and maybe also OpenBSD (hard drives are huge nowadays, I could fit hundreds of OSs on there). While I had tried out FreeBSD and NetBSD, DragonflyBSD had never been on my radar.
It still isn’t, really, but I thought I’d try it out on an old laptop, just to see what it was like. It went pretty well, but there were a few oddities and one or two kind of weird design choices.
Today, I decided to install Gentoo on a spare machine I had lying around, since I was bored. Obviously, the first issue I ran into was that
emerge x11-base/xorg-server was taking a really long time to run since the Xorg server is a pretty bloated program. Then
firefox was taking forever too.
One solution (for Firefox anyway) was to use the provided binary packages, this meant
firefox-bin for Firefox. But this means I abandon all of the nice features (for me, that means USE flags) that Gentoo offers. If I’m going to download a load of binaries that I can’t customize, why not just install Debian?
So the solution is to speed up compilation. That means putting more CPU cores to work. But my poor old ThinkPad only has 2 cores! This is where
distcc comes in.
There are hundreds of blog posts about backing up your PGP keys around on the internet. Most of them just say something like: put a passphrase on it, keep it on a USB stick, a CD, a floppy disk, or something like that. These are all very useful ways to back important stuff up - in fact, I just restored a backup of my GPG keys from a CD after deleting them by accident. These mediums are, however, not going to survive for many decades like paper can. And if you store to a writeable medium like a rewritable CD, DVD, USB stick or floppy, there is still the danger of you trying to restore then accidentally deleting your backup. Or, more likely, you write over it without realising many months later that you wrote a Linux distro or movie to that DVD that had your only PGP key backup.
I wanted to make a list of the websites of the people on the website http://nixers.net, and I decided to solve it not by asking people to tell me what their sites were called, but by scraping the forum.
I didn’t scrape the whole forum, I just scraped one topic on the forum that I created a few years ago: https://nixers.net/showthread.php?tid=1547
Some people reading this might be thinking: “hey, it’s really easy to do this, why is he writing an article on this?”. You are partially right, this should be really easy, but there are some weird things that happened while I set this up that I feel should have been written down somewhere, so my fear that I was completely borking my system would have been assuaged.
What is frog?
Frog is a static website generator written in Racket. It does the same sort of thing as Jekyll, Hakyll and other software like that, some of which I’ve used in the past.
Before a few weeks ago, I was always one of those people who said that Lisp isn’t useful, it’s not type-safe, it’s not pure, Haskell is better etc etc ad nauseam. All of that may be true for writing some sorts of programs, but Lisp (well, Common Lisp anyway) provides something a lot more pervasive.
What does pervasive mean? Well, right now, I’m controlling my window
Is it portable?
First off, the information in this post definitely doesn’t apply to Linux (as it has a completely different way of doing things) and may or may not apply to other BSDs (I see that NetBSD and FreeBSD both have similar, maybe identical, kvm(3) interfaces). There certainly isn’t anything in POSIX to make this standard. The only real reason
I know that sometimes, I’ve bene stuck in an airport or in a coffee shop without internet. That’s annoying in and of itself, but it’s even more annoying when there’s a WiFi hotspot nearby, but it requires you to pay £4/hour or something crazy like that.
You can still connect to the network, it’s just that whatever URL you
You know something that really annoys me? When I’m writing some Racket, Clojure, or any other Lispy language, and my editor won’t cooperate. Emacs is far, far, better than most other editors for this sort of thing, mostly due to paredit-mode and SLIME (and geiser-mode, and clojure-mode, and evil-mode, and…), but there’s still one problem I hadn’t solved until recently.
There are a few Platform as a Service (PaaS) services out there, and the most famous is probably Heroku. I know I’ve had people come up to me and suggest using Heroku for the next big thing they’re planning. There is a problem with Heroku: it’s not free (libre). That didn’t stop me from at least trying it out to see what all the fuss was about.
Earlier, I was trying to find something I could talk about at my school’s maths society. It had to be something exciting, useful, or at least beautiful in some way. I really wanted to do something on quaternions and vectors, because it seemed fun. The problem came when I realised I had to do something more substantial than stand there and explain something that boring. Then I saw this quote:
The other day, I was trying to access http://sharelatex.com at school, and it didn’t really work, probably due to a combination of Internet Explorer and possibly an overzealous filter that could have been blocking something. That’s what I thought, anyway, until I tried it on Chrome and it still didn’t work. Odd. The best solution was obviously to set up my own ShareLaTeX instance on my server.
UPDATE: I now use Mercurial on the client side (i.e. everything I do locally involves
hg) and Git on the server-side. It just makes it easier to mirror to Gitorious, GitLab, etc. You can view all the repos at http://kaashif.co.uk/code.
Earlier in the year, I was getting curious about version control
Back in the day, I used to use Slackware. It was the best distro around and all the cool kids used it. Nowadays, it’s rather different: a much lower proportion of people use Slackware. Despite the efforts of Eric and Patrick (and whoever else), Slackware isn’t really all that popular. It’s still a solid distro, though. There is one problem
Earlier today, I was discussing operating systems and came onto the subject of ease of installation. Which OS had the easiest installer? The obvious answers would tend towards OSs with GUI installers, but is that really easy? Sure, it could be familiar, but there’s a lot more you have to do with GUI installers compared to, say, OpenBSD’s
I’ve heard a lot of things about Racket (well really, things about many different Lisp dialects), and most of them were good. Recently, I decided to try to decide between Haskell and a Lisp once and for all. I wanted to go for a Lisp–1, since they keep functions and values in the same namespace, which is how it should be. Eventually, after
I’ve seen many completely stupid articles where people furiously circlejerk over how Vim is the best and nothing will ever come close, but it’s rare that I see anyone write much about Emacs, probably because fewer people use it (or maybe Emacs attracts a different demographic). It’s rare I see an article like
Recently, I’ve been trying to understand the ins and outs of CVS in order to be able to contribute to OpenBSD without messing up anything. I have sent a few patches to ports@, but anything complex was beyond my abilities until recently.
Recently, I’ve been trying to get away from pre-packaged file sharing solutions (e.g. FreeNAS) and trying to set up the services they provide from scratch. While I obviously won’t be able to write a web GUI or create a whole distro, that simply isn’t necessary. What is necessary is setting up a file share and appropriate read/write permissions.
Earlier, I saw this article claiming to describe how to share a file “simply” by running Python’s web server module (with either Python 2 or 3). While that may be easy, it’s not simple, and certainly not fast.
The inspiration for this came from this blog post, where the author describes how he uses his computer. While he does use CRUX, a GNU/Linux distro and I use OpenBSD, our workflows are actually surprisingly similar, which can, in part, be attributed to the
I’m sure lots of people (dozens, perhaps) have installed OpenBSD on ThinkPad T61s of some description, but with the recent release of OpenBSD 5.5, lots of documentation has become (or already was, and now is even more so) obsolete, like this article
On all of my computers, I like being efficient. That means eliminating everything which uses all that precious CPU time and using applications which are very customisable and configurable. These sorts of applications tend to be text-based, which is, in my eyes, a good thing, since they’ll show you the information you need with a minimum of
There is always a lot of buzz around the idea of “learning to program”. While I think it’s very important that children learn logical thinking and problem solving, I also realise that the majority of children, and people in general, would probably not benefit from a very language specific, rote learning based, generally old-fashioned approach to
UPDATE: I now use Hakyll. See http://kaashif.co.uk/about for more. Also, 100% of the information in this blog post is now wrong or outdated.
When I decided I wanted to write a blog, I had to come up with some way of writing posts (in a markup format which isn’t HTML), and serving them somehow.
After a few months of running a website, there are a few things I have realised about how I ran my server when it was first delivered, and how I run it now. The changes have been, for the most part, for the better. Needless to say, when I first started out, I was clueless, overeager and far too ambitious with my plans for “the next Facebook” or something
UPDATE: The project died, it went nowhere.
On nixers.net, the IRC channel of which I spend a bit of time in, there has been a bit of a stir as the community tried to decide on a project to commit to. The idea of creating a distro of some OS came up. A few people wanted a BSD-based distro, but it was decided that the Linux kernel was
When I first started programming, I barely had any idea of what constituted a good text editor, or why I’d want to use some old, texty editor from the 90s which didn’t even have most of the features I took for granted in the IDE I was using at the time. Maybe this had something to do with one of my first languages being Java, which is widely considered an IDE language, but I went through the
Over the years, I’ve used a few different OSes and desktop environments, and the one I use currently is portable to many operating systems, mostly due to the efforts of the writers of i3 over at i3wm.org, but also the standards-compliance of POSIX, meaning that my shell scripts (which you can find here) work on all
Whenever you hear something about Haskell, chances are it sounds arcane and involves lots of complicated and intimidating mathematical language. Well, the truth is that all this talk of ‘endofunctors’ and ‘monoids’ is really unnecessary, if the concept of functors is explained using a simple analogy.
When running a website on a residential connection, a problem one might run into is the dynamic IP address usually assigned by one’s ISP. There are a few dynamic DNS services which basically let you have a subdomain (e.g. mydomain.example.com) and let you update it to point to your IP address whenever it changes. At one time, your IP might be 10.0.0.1, and your domain correctly
If you have spent time on programming or technology boards like /g/ or /r/programming, chances are you might have heard the word “monad” thrown around a lot. You may have even heard the oft misquoted phrase “A monad is a monoid in the class of endofunctors” intended to be a joke, or to scare programmers away from scary functional languages like Haskell. The truth is that monads aren’t
There are a few very important things that everyone involved in software (particularly free software) can do to help out. The most important is to file detailed and helpful bug reports, so the developers working on your favourite program can get the problem fixed. Since it is not very hard to write a bug report, and projects generally have their own bug report guidelines, I won’t
While I did have some old hardware lying around, I had never committed to actually getting that hardware usable. By that, I mean I had never tried to browse the web, read emails and that sort of day-to-day stuff on anything older than a few years. To see if it were really possible, I decided to buy an old ThinkPad (a 760EL from 1995) and see if I could get it working. Before I
stow is a cool little Perl script which basically just creates and deletes symlinks. That sounds pointless, but let me explain with an example. Let’s say you want to install a program using the usual
make install, which probably installs into /usr/local, which means it’s separated from the rest of your system, which is managed with a “real” package manager. Unless you’re using a
For a few years now, I have been using Vim to edit config files, program in C, Python, even Lisp (people apparently think that Vim isn’t the best for programming in Lisp). This isn’t because I took a side in the so-called “editor wars”, it’s just because it came preinstalled on the first GNU/Linux system I used, Debian. Over time,
This tutorial is designed for those who have programmed before, perhaps in a higher level language like Python or Ruby. It’s not too hard to understand for those who are completely inexperienced, but some knowledge of functions, data structures and pointers might help. Most of the low-level stuff will be new to high level programmers, however.
Often, people look at me oddly when I suggest that they email me something. “Why can’t I just send it to you on Facebook or Skype?” they say. Well, it doesn’t have to be those two media of communication, but it’s usually something like that. When I say often, I also mean that this has only happened on two occasions, so bear that in mind as I make things up about the types of people
This article is not only about disk wiping, it will hopefully teach you something about using some GNU command line tools . This tutorial was written on my ThinkPad, which runs Debian, so the output should be pretty similar to what you’d get on Ubuntu, Mint or any other Debian- or Ubuntu-based systems. Basically, if you’re using something
Installing packages from source
Recently, I had SSHed into one of Debian Stable virtual machines I was using as a file server. The main services I was running were FTP, an HTTP server with a directory listing and a Samba server, with shares set up for a few users on my
You probably have not heard of LaTeX before now. If you have, then it is likely that you have no idea what LaTeX is, save for a vague feeling that it relates to documents in some way. By the end of this short post, you will not only know what LaTeX is, but be able to understand why people use it, and what its advantages are. While you probably won’t switch to LaTeX immediately, you might
Imagine you’re a person on some sort of device, using the internet. You see all of these websites and what do you ask? “How can I set up a web server?”, of course. If you did not ask that question, then this guide is not for you. Anyway, down to business. You will need:
Why am I writing this?
I have looked up “how to use gpg” so many times, on so many websites, and have found every guide to be focused on something I don’t use or worded in such a way that I get confused and revoke all of my keys (that hasn’t actually happened…yet). I thought I’d whip up a quick guide