{ Will Richardson }

Me Blog RSS JSON Feed

Learn Enhancer Reaches Version 2.0

When I first started Learn Enhancer at the start of my degree - sometime in early 2014 - it was just a single JavaScript file that would redirect to a certain URL if it found a certain element. Eventually I coughed up the $5 required to submit it to the Chrome Webstore, and it started its slow creep on to students browsers.

This initial version didn’t catch every case, and so just over a year later I grew frustrated enough to dig right into the HTTP requests and alter them to ensure that the file could be previewed - which I outlined on this blog. Since then I branched out into re-styling every page on Learn with gentler colours (with design supervision from Sarang) - which had the delightful side-effect of letting me see whenever someone has Learn Enhancer installed. It is nice to see my work being used by people that have no clue who I am.

One of the features that I wanted to include but never quite worked out was auto-login, so that I didn’t have to two buttons before accessing content on Learn. This never quite worked out, until this morning I saw Patrick working on a script to do it. Thankfully he was receptive to the idea of incorporating this into Learn Enhancer, and soon he had a it working in the extension.

So here we go, Learn Enhancer has hit 2.0. Since this is my last year at UC, I have put Learn Enhancer on GitHub so that (in theory) current students can contribute fixes when Learn inevitably changes after I have graduated.

If you are a current UC student, download Learn Enhancer, you will almost definitely appreciate it.

Using tmux in the Real World

Every now and again I happen across a post outlining how to use tmux. Since I first happened upon tmux in 2015, my use of it has grown from “occasional”, to “frequent”, almost to “continual”. What I find frustating with these posts is that they don’t describe how to actually use tmux in the real world. The post in question that prompted this post tells you how to start a session, create new windows, then how to switch between and resize windows.

The thing that they fail to explain is that the default tmux commands and shortcuts are terrible. Common operations require far too many fiddley keystrokes to be done quickly. Moving between panes is by default Prefix followed by an arrow key, so to move to the right you would enter C-b → - but you can’t press the arrow key while you have control held down, because that will resize the window. So if you have four panes, you have to press C-b, release ctrl, press the arrow key, press C-b, release ctrl, press the arrow key, press C-b, release ctrl, press the arrow key once more and you’re there - unless of course you pressed the arrow key before the ctrl key was released, which means you will have a slightly resized pane instead 1.

This lack of usability is repeated - splits are created with Prefix % and Prefix " but which one does horizontal and which one is vertical? I have no idea, plus having keys that you have to use the shift key to get at just makes them harder to get at. The tmux command also leaves a lot to be desired - it’s not simple to connect back to an existing session given its name.

What irks me the most is that a beginner will read one of these posts and think that they have to remember all these arcane commands and be able to enter them at lighening speed. This is far from the reality of using tmux (or most other command-line tools) - everyone that I know that uses tmux has a config that makes tmux fit to the way they think of things. Each one of them had to learn the defaults and then find out if there was a better way - which is a significant barrier for most people.

The aspect of tmux that redeems these oddities is it’s extensive set of customisation options. Every command can be bound to a new shortcut, and shortcuts can be entered without needing to press the prefix key first. So what I’m going to do is build a set of reasonable defaults, so you can jump ahead and use tmux like a sane person.


This post isn’t a one-stop-shop for all your tmux needs, instead it’s just a quick walkthough of the basic ways that I make tmux more appropriate for daily use 2. All the snippets should be added to your tmux config file, which lines in ~/.tmux.conf by default.

The first thing that most tutorials tell you to do is remap the prefix to something other than C-b, because C-b is a bit too much of a stretch for most people. I use C-z, many people use C-a. Whatever you use is up to you. To remap the prefix, add this to your .tmux.conf:

unbind C-b
set -g prefix C-z
bind C-z send-prefix

This deactivates C-b, sets C-z as the prefix and makes a shortcut C-z C-z that will send C-z to the program inside tmux (so you can still use the shortcut). Replace C-z with another shortcut that tickles your fancy if you so desire. (I’ll use C-z when I’m talking about the prefix in examples, just remember to use yours if it is different).

The next thing is splitting panes. This will depend on how you visualise the panes, but I think of a horizontal split as two panes with a divider that is horizontal, and a vertical split has a vertical divider. This is the opposite to how tmux thinks of it, so depending on how you think, you may want to skip this.

Since tmux 1.9, new windows and panes open in the directory that tmux started in. I prefer the old method where they would open in the same directory as the previous window or pane. I frequently run some command, and if it takes a while I will open a split and continue working in the same location while waiting for the command to complete. I find this behaviour useful, and I think you will too. So:

# Open new windows with the same path (C-z c)
bind c new-window -c "#{pane_current_path}"
# Create a 'vertical' split (divide vertically) using C-z v
bind v split-window -h -c "#{pane_current_path}"
# And a horizontal split (divide horizontally) using C-z h
bind h split-window -v -c  "#{pane_current_path}"

Ok so on to the main event, the thing that makes tmux actually usable - faster pane switching. I use vim so I’m used to using h/j/k/l for left/down/up/right movement, you may prefer the arrow keys. Up to you. The key is to make these shortcuts not require the prefix before them, so you can smush some buttons repeatedly instead of repeating an exact sequence.

# For h/j/k/l movement
bind -n C-h select-pane -L
bind -n C-j select-pane -D
bind -n C-k select-pane -U
bind -n C-l select-pane -R
# For arrow key movement
bind -n C-Left select-pane -L
bind -n C-Down select-pane -D
bind -n C-Up select-pane -U
bind -n C-Right select-pane -R

These lowers the barrier to moving between your panes, which should hopefully encourage you to get crazy and open as many panes as you can fit on your screen. Wait, what if I don’t want to have everything in exact halves? Then you’ll have to resize a pane!

The post that inspired this one instructs you to resize panes by opening the command mode Prefix : and entering resize-pane -L, to move the split to the left. Now that is just super tedious. You can give it number of the amount you want to resize it, but that devolves into guesstimating pretty quickly. Instead I like to leverage the meta (alt/ option) key, so M-l (alt + L) will resize the pane to the left. Again you could make this M-Left if arrow keys are your forté.

# h/j/k/l
bind -n M-h resize-pane -L
bind -n M-j resize-pane -D
bind -n M-k resize-pane -U
bind -n M-l resize-pane -R
# Arrow keys
bind -n M-Left resize-pane -L
bind -n M-Down resize-pane -D
bind -n M-Up resize-pane -U
bind -n M-Right resize-pane -R

And boom, you can resize panes super quickly. One last shortcut that isn’t quite essential, but still useful is a quick window-switching shortcut, I like M-n and M-p to replace C-z n and C-z p. Especially if you’re flicking through a lot of windows.

bind -n M-n next-window
bind -n M-p previous-window

Two more useful things; set the default terminal to be 256 color so that your editor looks good, and set the starting index of windows to be 1 rather than 0 so it follows the order of the keyboard:

set -g base-index 1
set -g default-terminal "screen-256color"

So what to do about managing your sessions? Almost everyone I’ve talked to has made a little wrapper script that basically does this: if no arguments are given, list all the sessions. If an argument is given, connect to that session if it exists, otherwise create a session with that name. This avoids having unnamed sessions and means you don’t have to remember to run tmux ls every time. I’ve made a version of this with more bells and whistles but this is the basic idea:

mux() {
  local name="$1"
  if [ -z "$name" ]; then
    tmux ls
    return
  fi
  tmux attach -t "$name" || tmux new -s "$name"
}

Chuck that in your .bash_profile, .zshrc or whatever, then run mux to view your sessions, or mux my-session to create or connect to a session.

These are the changes that I have made to make tmux usable, but don’t forget that there are a whole load of things that I just do the default way. This post isn’t an exhaustive tutorial on using tmux, but rather an outline of how to make it more useful if you share my sensibilities.

  1. I know that if C-h/j/k/l were the default this would stop those keys being able to be used for other things, but I think the productivity gain is far greater than the loss of some keys (this is probably just because I don’t use anything that needs those shortcuts). 

  2. Other things that irk me are poor window indicators in the status bar - mine has more color to show the current window. The status bar also does a poor job of showing the status info - especially the current host. I change the color of part of the status bar depending on the host I’m on (mostly for aesthetics). And the default green highlight is super gross. 

Metaprogramming and Macros for Server-Side Swift

I have been a fan of the Swift programming language since it was first announced, and especially after it was open sourced. The place that I thought Swift could be the most interesting for me was for server applications - I’m not much of an iOS/ macOS developer. The progress of Swift-on-Linux is slow for someone that doesn’t like digging around Makefiles and linking to C libraries.

However, there are some things about web applications that aren’t currently served by the design of Swift. This can basically be boiled down to one thing - compile-time macros. Having a macro system allows for a lot of really cool syntactic sugar, as well as removing work that would otherwise need to be done on the first request, or at startup. Many of these are taken from my brief time learning Phoenix, a web framework written in Elixir - if I’ve misinterpreted something or ruled out some approach that is actually possible, let me know.

The main use of macros in your typical web framework is the routing configuration. Phoenix and Rails both support a DSL (implemented using the syntax of the language, Elixir or Ruby). Both of these look quite similar, basically allowing you to do this:

# In Phoenix
get "/", MyController, :index
# In Rails
get "/", to: 'my_controller#index'

The DSL gets more complicated when you include resourceful routes and other goodies. But at its core the purpose of the DSL is to allow the developer to use the same tools (i.e: the same editor and highlighting) to define their routes in a succinct manner. Phoenix can go one step further, because Elixir supports macros. The routes are checked when the project is compiled, and can be turned into arbitrary code that responds to web requests following the rules defined.

For example, the get macro can check that the path is valid, that it doesn’t clash with any other routes, and make helper functions for linking to that page (e.g. a my_controller_index_path() function). This is done at compile time, so when the code is run it is no different to running the “hand written” equivalent.

This is not the case in Ruby - because it is a dynamic language these methods can be created at runtime. There is basically no loss in performance because to support this level of metaprogramming (and because it is interpreted) Ruby is super slow compared to compiled languages.

When it comes to compiled languages without macros (like Swift, Go, Java, etc) you can’t pre-calculate information while the code is being compiled. Go lacks the features 1 to implement any kind of usable DSL. Revel (the #1 result when googling for “golang web framework”) has a separate routes file - written in a Revel-specific syntax that is parsed at runtime. This creates complexity in the packaging and distribution of the application - it no longer can be built as a single binary as it relies on this config file.

Swift does allow for creating concise DSLs. Vapor and Perfect are Swift web frameworks. Both of them offer routing DSLs that look something like:

app.get("/:page_id") { request ->
	return Response(.text, request.params["page_id"])
}

But this is processed at runtime, and doesn’t allow for creating helper methods for creating URLs, or grouping methods together into a class-based controller like Rails does. The latter could just be a necessary limitation of Swift, instead of making classes you could create a “controller factory” DSL, which you might use like:

controller("MyController") { app ->
	app.get("/stuff") { request ->
    // do something with stuff
  }
  // etc
}

Although this doesn’t get around the fact that much of your logic is defined in string literals that don’t get looked at until the application is running, or the fact that the routes must be generated when the application starts - if you wanted to make a super-efficient trie or other data structure for better processing requests, you sacrifice startup time in both development and production, even if the structure never changes until a new version is deployed.

Moving code-level information out of strings allows for static analysis to perform more useful checks when validating code. For example, regular expressions are often written as string literals (e.g: in Java) and so don’t get checked for validity until the program reaches them. Other languages have builtin regex literals (JavaScript, Ruby) to fix this problem. Elixir goes one step further thanks to (you guessed it) macros, specifically “sigils”. These are macros that wrap around a special “literal creation” syntax. This is used not only for regexes (written like ~r/abc\w{5}/) but for common “make a list of strings” helpers commonly found in other languages: ~w(foo bar) is equivalent to ["foo", "bar"]. So if you made a cool new type of regex that adds some awesome new feature, you can implement a macro that lets you write it easily and have all the same advantages as the builtin version.

View templates (think ERB, Liquid, Handlebars, etc) can also be parsed and optimised at compile time using macros - Phoenix does this so that when running the application all that needs to be done is string concatenation, no parsing needed.

So where does that leave us with Swift-based web development? It doesn’t seem any worse off than Go in terms of ability to dynamically create methods, etc - and Go appears to be used for web development a wee bit. The other option is code generation - but that’s always going to be a second-class way of doing this, as it relies on other tools and requires the other tool to parse the rest of the codebase to get the advantages you would from a macro system.

There might of course be a time when Swift gets a macro system, which will create a huge opportunity for new syntax and more concise, expressive code. However given the complexity of Swift and decisions so far, I would not hold my breath.

  1. lol no generics. 

Needless complexity: Generalising a Scheme for Aikido Training

It is perhaps a little known fact that I have practiced Aikido for about the last 13 years now.

I’m bad at writing introductions, so let’s jump straight to the problem. When training with more than one other person, you have to have some way of deciding who attacks who - you can’t just alternate. Normally when practicing tachi waza the most senior student goes first and does the technique four times to the uke before the roles are swapped. So what to do when someone else joins your pair?

You could just make a directed triangle - the person A is attacked by B, the B by C, and C by A, before the cycle repeats. This is easy to describe and can easily be extended to any number of people, but person A will never be attacked by person C - they miss out on any feedback that person C may have for them. You want a method that will allow everyone to train with everyone else, as well as allowing each member to do the technique enough times to improve.

At the moment, this is the recommended way of training:

A - B
A - C
B - C
B - A
C - A
C - B

Now this is fine. Apart from the fact that it only applies to exactly three people. The programmer in me wants a method that applies to any number of people. How about:

func train(members: [Person]) {
  for nage in members.sorted(by: .rank) {
    for uke in members.sorted(by: .rank) where uke != nage {
      uke.attack(nage)
    }
  }
}

Basically starting from the highest ranked member, each member should have a turn as nage. They should be attacked by each other member, in the order of their rank. This is how training in a pair works, and works just the same way if the whole class is training together.

This gets slightly more confusing when you doing weapons practice - there is a less clear distinction between the uke and the nage; the uke is often not thrown by the nage, and the uke still has to learn the attack as it not just a single strike or grab.

It’s common with weapons practice for a pair to train with one role, then swap and train before moving on to the next member of the group. This reduces the distraction of changing partners, letting you focus on the technique. This can be generalised in a similar way - this time each member of the group in descending rank order is the ‘key’ member, who practices both sides of the technique with each other member, then the ‘key’ member is changed to the next member in rank.

func train(members: [Person]) {
  for key in members.sorted(by: .rank) {
    for other in members.sorted(by: .rank) where key != member {
      other.attack(key)
      key.attack(other)
    }  
  }
}

Basically I think too much about the efficiency of how I am training, rather than focussing on the training itself. I guess that’s what happens when you spend all day learning about Software Engineering and stuff.

More Fun With Generics in Kotlin

Android now supporting Kotlin means more people playing around with it. Ben Trengrove is one of them - he has made a quite neat way of representing units in a type-safe wrapper. This disallows doing operations on two units of different measurements - for example speed cannot be added to a distance. Adding helper extensions to numeric types allows you to use it like this:

val distance = 21.kilometers

// This is OK because they are both distances
println(distance + 5.miles)

// This fails because you can't add distance to time
println(distance + 9.minutes)

You can have a look at Ben’s code here. Currently Ben’s code allows you to multiply one quantity by another. The result is a quantity with the same unit of the operands passed to the multiplication. This doesn’t follow the rules of dimensional analysis - if a distance is multiplied by another, the result not a distance, it’s a 2D area.

What this means is that you can do code like this:

val width = 10.meters
val height = 5.meters

// Area has unit Distance.Meter, not meters squared
val area = width * height

This piqued my interest - how could you implement units like speed and area, that are composed of multiple units? Of course you could just remove the .div and .times methods and replace them with extension functions that return a Quantity<Speed> or Quantity<Area> for each combination of units that you’re interested in.

But surely we can do better? This is what I set out to do, I wanted to be able to define the base units and derive every other unit from them. If you want to skip the rambling, you can check out the end result here.


The premise of this approach is to make two new subclasses of Unit1 that each have two generic constraints - each of which must also be a type of Unit. These represent a division type and a multiplication type, QuotientUnit and ProductUnit. All units have a suffix attribute that stores the standard identifier for that type (like “m” for meters, “s” for seconds, etc). QuotientUnit and ProductUnit create their suffix from the suffixes of their parts with either “/” or “.” inbetween.

The basic units are still defined in a similar way: 2

abstract class Unit(val suffix: String, val ratio: Double) {
  internal fun convertToBaseUnit(amount: Double) = amount * ratio
  internal fun convertFromBaseUnit(amount: Double) = amount / ratio

  override fun toString() = suffix
}

open class Distance(suffix: String, ratio: Double): Unit(suffix, ratio) {
  companion object {
    val Mile = Distance("mi", 1.60934 * 1000.0)
    val Kilometer = Distance("km", 1000.0)
    val Meter = Distance("m", 1.0)
    val Centimeter = Distance("cm", 0.1)
    val Millimeter = Distance("mm", 0.01)
  }
}

The implementation of the composite units looks like this: 3

class QuotientUnit<A: Unit, B: Unit>(a: A, b: B):
    Unit("$a/$b", a.ratio / b.ratio)
class ProductUnit<A: Unit, B: Unit>(a: A, b: B):
    Unit("$a.$b", a.ratio * b.ratio)

They are really just a placeholder to keep the type system in check. We can then use these to extend our .div and .times methods to return quantities with composite types. So inside the Quantity class we add:

operator fun <R: Unit> div(quantity: Quantity<R>): Quantity<QuotientUnit<T, R>> {
  return Quantity(amount / quantity.amount, QuotientUnit(unit, quantity.unit))
} 
operator fun <R: Unit> times(quantity: Quantity<R>): Quantity<ProductUnit<T, R>> {
  return Quantity(amount * quantity.amount, ProductUnit(unit, quantity.unit))
} 

So now when we divide a distance in kilometers by a time in hours, we get a QuotientUnit<Distance, Time> with a suffix of “km/h”:

val distance = 21.kilometers
val time = 1.5.hours
println("Speed is: ${distance / time}") // Speed is: 14 km/h

And we should be able to do conversions between composite units as well, because the ratio of a composite unit is calculated based on the original units.

val speed = 21.kilometers / 1.5.hours
val milesPerHour = speed.to(QuotientUnit(Mile, Hour))
println("Speed is $milesPerHour") // Speed is 8.7 mi/h

Now that’s quite useful. However typing QuotientUnit(Mile, Hour) is not very elegant. Perhaps we can use some helper function to make this a bit more readable?

We can actually do better than a helper function, instead we can make an extension operator that defines / and * on pairs of units. This lets us spell a composite unit like this: Mile / Hour, which is the same as QuotientUnit(Mile, Hour). You can do this like so:

operator fun <A: Unit, B: Unit> A.div(other: B) = QuotientUnit(this, other)
operator fun <A: Unit, B: Unit> A.times(other: B) = ProductUnit(this, other)

With all this, we can now do the unit conversion problems you get in physics class with almost no effort:

// James Bond is running along the roof of a train
// It takes him 1 minute to run the length of a 20-metre carriage
// The train is moving at 60 miles per hour
// How fast is James moving relative to the ground, in km/h?
val jamesSpeed = 20.meters / 1.minute
val trainSpeed = 60.miles / 1.hour

val totalSpeed = jamesSpeed + trainSpeed

val metricSpeed = totalSpeed.to(Kilometer / Hour)

println("Speed relative to ground is: $metricSpeed")
// Speed relative to ground is: 97.76 km/h

Easy!

The last thing I wanted to clear up was the need to repeat code for all the helper properties (6.minutes, 9.kilometers etc). These have to be repeated for every type of unit, and I wanted a way of creating units without this repitition. In reality you’d probably keep these to make it easier on yourself, but it’s nice to have an alternative.

How about just a simple infix function that operates on a number and a unit? Or how about if you multiply a number by a unit, it creates a quantity with that unit? What about invoking the unit with brackets - like a function call - to create a quantity in that unit? These are all quite straightforward: 4

// 5 into Minute makes a Quantity(5, Minute)
infix fun <T: Unit> Number.into(unit: T) = Quantity(this.toDouble(), unit)
// 79 * Kilometer makes a Quantity(79, Kilometer)
operator fun <T: Unit> Number.times(unit: T) = this into unit
// Second * 3 makes a Quantity(3, Second)
operator fun <T: Unit> T.times(value: Number) = value into this
// Hour(12) makes a Quantity(12, Hour)
operator fun <T: Unit> T.invoke(value: Number) = value into this

All of these make it super easy to create quantities. And of course, you can use them with composite units: 5 * (Kilometer / Hour), Second(8), 9.into(Metre * Metre) each create speed, duration, and area quantities.

Using these we could solve our physics problem from above like so:

// James' target is running twice as fast as him along the train
// How fast is the target moving relative to the ground, in m/s?
val jamesSpeed = 20 * (Meter / Minute)
val targetSpeed = 2 * jamesSpeed
val trainSpeed = 60 * (Mile / Hour)

val totalSpeed = targetSpeed + trainSpeed

val metricSpeed = totalSpeed.into(Meter / Second)

println("Speed relative to ground is: $metricSpeed")
// Speed relative to ground is: 27.5 m/s

Another helper that we can add is a division operator that operates on two quantities of the same type, producing a ratio of the two values rather than another quantity. You can use this to see “how many items of length X fit in space Y?”. This is done like so, inside the Quantity class:

operator fun div(other: Quantity<T>) = unit.convertToBaseUnit(amount) / other.unit.convertToBaseUnit(other.amount)

// To work out the ratio between two speeds:

println("Speed ratio: ${jamesSpeed / targetSpeed}")
// Speed ratio: 0.5

For completeness I also added operators for multiplying values by numbers with no units - letting you do things like “double this distance” with 2 * distance. Quantities are also comparable, so less than and greater than also work.

Hopefully this explanation illuminates some of the magic generics in the code - which you can view here. I’m sure there are operations and helpers that I’m missing, or ways that this code can be cleaned up and simplified. This would make for a kick-ass back-end for a unit conversion app!

  1. When implementing this, my first idea that turned out to be a dud was to make every different measure a Kotlin object. This did mean that the types could be part of generic constraints, so QuotientUnit<Kilometer, Second> is a valid type. This seemed like a good idea initially, but quickly ended when I realised that QuotientUnit<Mile, Second> is a different type to QuotientUnit<Kilometer, Minute> - even though they both represent distance/time

  2. In my examples I omit the prefix of the base type (eg Distance.) for readability. You can import them directly which allows you to use units with no prefix (eg Metre) or just import the class and access the companion variables (eg Distance.Metre). 

  3. I’ve changed my Unit class to have a .toString method that simply returns the suffix, differing slightly from Ben’s original version. 

  4. I decided to rename .to to .into so that it didn’t clash with the built-in .to extension in Kotlin that turns two objects into a Pair 

Templates, Code Generation, and Macros

Macros are a really cool feature that is includes in a few cool languages (Clojure/ Common Lisp/ other Lisps, Elixir, Rust, and Crystal) that allow you to reduce boilerplate code, extend the capabilities of the language, and process data at compile time. There is no shortage of tutorials on macros, but I am going to approach them from a direction that may be more familiar to some people.

It doesn’t take long while programming to come across some kind of template. Whether it’s str.format() in Python, or the moustache-templated bindings in $JS_FRAMEWORK you’ll end up writing something that will be used to generate something else. We can use these to separate the content from how it will be displayed.

This snippet is some Embedded Ruby, the <% and %> denote a start and end of Ruby code. Outside the Ruby is just HTML.

<ul>
  <% for item in items %>
    <li><%= item %></li>
  <% end %>
</ul>

Here we have items which is a list of strings, and we iterate over it creating an <li> with the content of each item. The server will run this code and generate some HTML, which is then sent to the browser. The browser doesn’t know that the HTML was part of a template, it looks exactly the same as hand-written HTML. What we’re doing is using Ruby to generate more code that could have been written by hand. Doing this for a formatting language makes sense - there is no way for HTML to show dynamic content. However because we’re just dealing with text, we can use a template to generate any type of file - even code.

function <%= func_name %>() {
  return "This is <%= func_name %>";
}

This is a silly example, but we’re using a template to create a Javascript function. All it does is returns “This is “ followed my the function name, but if we were making a library that interacted with a database, and we wanted to have an API that was something like <table_name>.get_by_<column_name>() without having to do metaprogramming (for example if the language doesn’t support metaprogramming, or you don’t want the runtime cost of doing reflection).

A common example of something that uses a “template” to generate code is a parser generator (like Cup, Yacc). These read a file that is in their own syntax for defining a grammar and how to store the AST, and produce a file of code in the target language that will parse something according to the specification.

Casey Liss recently wrote about Sourcery, a library for automaticaly generating boilerplate Swift code - it can do things like make a type Equatable by generating an == method that compares every property of the type. This is possible by generating a .swift file with the code needed to define this method.

Sourcery is quite cool - but it means that you have to have a special Sourcery file that defines what it should do, and remember to run the Sourcery script to generate the new files. Ideally you would put the definition for what boilerplate to generate inside the Swift file with all the rest of the code, and the Swift compiler would generate it automatically before running the program.

This is essentially what macros are. They are pieces of code that make more code, and are run when the program is compiled. For example if Swift supported this, it might look something like:

func ==(other: this.class.name) {
  for attr in this.storedVariables {
    quote {
      guard self.{{ attr }} == other.{{ attr }} else { return false }
    }
  }
  quote {
    return true
  }
}

I’m imagining that quote will turn whatever is inside it into code that will be generated (like other languages), and the double curly braces escape a variable - {{ attr }} would be expanded to the name of the attribute.

For macros to be super effective, the language should be represented in its own data structures (languages that are like this are called homoiconic). Clojure is one of these, and provides syntax for denoting which code is to be evaluated, and which code is to be used to generate more code. ` or ' start a “this is for code generation block” - everything after the that is like all the HTML outside the <% %> tags in Ruby. Code after ~ is equivalent to code inside the <% %> tags.

So we could make a macro that prints the code it will evaluate before it runs (like the -x option in Bash):

(defmacro debug [code]
  `(do
    (println '~code)
    ~code))

This can then be used just like a normal function call, but instead of calling the function at runtime, it gets replaced when the code is compiled.

(debug (println (+ 8 6 (* 5 7))))
; Will be replaced with
(do
  (println '(println (+ 8 6 (* 5 7))))
  (+ 8 6 (* 5 7)))

The new code will first print "(+ 8 6 (* 5 7))" then run the maths.

That’s a silly example, but a far more practical example is the Ecto library for Elixir. It is a DSL for running SQL queries, by using a macro it basically adds an SQL-like language right into Elixir, which can be checked for validity at compile time, rather than putting SQL in string literals where errors are only known when the code is run.

Running code at compile time also lets you do some cool tricks that don’t involve creating “new” syntax. For example, resource files can be loaded right into the program, so nothing has to be read from disk when the application is running. Phoenix (an Elixir web framework) loads all the views when the code compiles and turns it into a function that concatenates strings - so no parsing has to be done at runtime.

Of course, many smart compilers let you use lambdas and stuff to create “new syntax” that gets expanded at compile time, but macros allow the developer to have more control over what happens when code is compiled and truly add new contructs to the language.

Conditionals in SH

I’ve been spending more time than I would like writing shell scripts recently, as I spend more time configuring my setup than I do on ‘real’ projects. What I’ve found interesting is how simple the core of a shell is, and the tricks some commands do to build on this.

Most *nix users have probably had a moment were they were writing a shell script and forgotten the syntax for an if statement. I write shell scripts so infrequently I often have to look it up. However all the if statement does is run the condition command and check the exit status, if it is 0 it will run the main block, anything else and it runs the else block.

“But what about the square brackets?” I would think to myself. Well that’s just a command. You know, the [ command. man [ reveals that this is just a standard command with some flags to tell it what kind of thing to check.

Let’s take a simple conditional that checks that two numbers are equal:

if [ $num1 -eq $num2 ]; then
  echo "Equal!"
fi

If num1 is 4 and num2 is 5, the [ command will receive "4", "-eq", "5", and "]" (remember everything is a string in the shell). The command takes the arguments up to the closing square bracket and does the comparison, in this case -eq means integer comparison. As far as I can tell the closing bracket is just for readability - if you have a condition with with logical operators (|| or &&) then each part of the expression can be in separate brackets (or you can use the -o and -a options to keep them in the same set of brackets).

So this means that we can do things like this:

[ -f some/file/path ] && cool_function_on_file some/file/path

Making use of the && builtin, rather than writing a whole if;then;fi block. Or when we remember that the condition can be any command, we can be a bit smarter in scripts:

if git clone "$full_remote$user/$project.git" "$_path"; then
  echo_cd $_path
fi

This will only change to the cloned repos directory if it cloned successfully (indicated by the result of the git command).

for loops work in a similar way, except instead of the condition we have a command that produces an output with each element separated by the $IFS variable. The $IFS is basically just whitespace/ newlines so we can capture the output of ls and iterate through each filename:

for filename in $(ls); do
  echo "It's a thing: $filename"
done

So in short, I have learnt a bit about shell scripting and now think it’s kind of neat rather than getting frustrated at the seemingly nonsensicle syntax.

For reference [[ and == are builtins to BASH and other newer shells. == is no different to = (but it can’t be used to accidentally assign something). [[ works the same as [ apart form the fact that it can be used with < and > for comparison, as it can process them before they are interpreted as IO redirection as part of a normal command.

Tested: Apple Won't Make a Touch MacBook

Norm and Jeremy on the Tested.com podcast have frequently complained that Apple doesn’t make laptop with touchscreens. This past week they stated that it was almost an inevitability that there would be a touchscreen MacBook some time in the future. I think this is unlikely and most definitely not something that will be released anytime soon.

Before I go any further, just a note: I have not owned a laptop with a touchscreen (I do use a Pixel C like a laptop frequently, though). But when has having no experience in a subject stopped anyone from voicing their opinion on the internet?

Why am I so adamant that there will be no touch MacBooks? The answer is simple: macOS. MacOS/ OS X is not designed to accept imprecise inputs from a touchscreen - the touch targets are far too compact. The size of the window chrome on macOS has typically been smaller than the size of Window’s windows.

This was mentioned in an interesting post by the Chrome design team, which ran through the process of redesigning the Chrome UI across all platforms. The height of the chrome was significantly larger on Windows.

Most of the time I have spent using touchscreen laptops has been debugging group project code on team mates computers. This meant using IntelliJ - which has its fair share of menus and toolbar buttons - all designed for use with a mouse. Naturally because of the novelty of having a touchscreen (or the mediocre quality of the trackpads) I used it instead of the trackpad.

IntelliJ is basically unusable on the touchscreen, the menus and buttons are too small to hit. Navigating nested menus is not at all pleasant. Anyone that has used a Mac knows that most normal applications have all their actions in the menu, and common actions can be placed in the toolbar of the application. The minimum recommended size for a touch target on iOS is 44 by 44 points whereas the recommended size of toolbar items on macOS is “at least 19x19 points” the actual clickable area is slightly larger than this at 36 by 24 points. Menus are a similar story - they are only 30 points high.

For a touchscreen Mac to be a good user experience, macOS’s entire UI would have to be redesigned. This would mean a massive amount of work for third-party developers (maybe not so much for those that just use entirely system controls) and probably leave a sad collection of apps that look out of place in the new OS.

Of course Apple has not shied away from making massive changes that require significant work to support by developers (switching to Intel, introducing retina displays, Yosemite redesign, etc). However given fairly small changes to the Mac lineup, any major change seems unlikely.

This is coming from someone that uses the terminal to find files more often than Finder, and uses their Mac mostly for development. So just perhaps my useage is not quite the norm. Although almost everything that I do that is not development is done on my Pixel C.

I think Apple’s answer to people that want a touchscreen laptop is the iPad Pro. And no, they will not merge macOS and iOS.

Bluetooth is Great, Until it's Not

Did you hear that Apple removed the headphone jack from the latest iPhone? Oh you did, well that’s a relief. What do you think? Have you sworn to never use an Apple device ever again? Well I thought I would care - but I don’t any more.

Let’s rewind a bit. Most of my time listening to audio on my phone since 2013 has been podcasts. All the great shows. I had been using a beat-up pair of Apple Earpods, as they fit my ears better than any other in-ear headphones. However having to wiggle the cable every few minutes gets old fast, and I was on the lookout for a replacement.

I ended up looking at the Urbanears Plattan ADV, and the Marley Positive Vibration headphones. Both are fairly reasonably priced and look good. When I went to buy them, I found that because of a sale the Marley Rebel BT headphones were the price that I was expecting to pay for the OTHER MARLEY phones.

So I am now the proud owner of some bluetooth headphones, and the lack of a cable is liberating. I am no longer concerned about how my phone sits in my pocket, or if I leave it on my desk when I jump up to get something, or how the cable will tangle with the strap on my bag. I am a satisfied customer.

Of course it’s not all good. Bluetooth pairing is a scary business. Bluetooth devices don’t like sharing. Connecting headphones that are paired to my phone connect to my tablet is a recipe for disaster. If I did, then they would start auto-pairing to my tablet when I turned them on. Then I would have to venture into the settings each time I used my headphones. Having a cable for does make this easier - if I’m using my tablet or laptop then it’s unlikely that the cable will get in the way, so limiting the bluetooth to my phone is not a big deal.

What would be ideal would be a pair of headphones that could accept a few different inputs and either combine them all or select the most recent one - so you could be listening to a podcast on your phone, then play a video on your tablet, and the phone would be told to pause while the video plays. That would be ideal, although it would mean that the headphones would either have to be in constant pairing mode to connect to new devices as they come in range, or require some button-press to look for a new device.

Of course, all good things must come to a saddening end - the more technology you add to something, the more ways it can break. So when I turned my headphones back on in preparation for the skate back home, instead of making the comforting “boo-doop” to indicate they are on and paired, they went “beeeeeeeeeeeeeeeeeeeeeeeep booooooooooooooop zero zero zero zero zero zero zero zero zero zero zero zero zero zero zero zero zero zero…” (yes, they literally had a computer voice saying “zero” over and over. Plugged in they worked fine. But any sign of bluetooth working was gone. Back to the shop to claim that return policy!

So now I have a new pair and right now they are working fine (I’m listening to some Mutemath as I write this - playing via my phone while I write on my tablet). But they weren’t without their own issue - for a few days they decided that they would only connect to my phone if I manually told my phone to connect to them. They then completely out of the blue (heh) started to connect automatically. Great for me, but damn weird.

Overall I am happy with the fact that my “daily driver” headphones are wireless - I will still use my Sennheiser over-ears if I’m working on my laptop at home. The ability to leave my phone sitting on a table, to have it facing whichever way in my pocket, and to get rid of the possibility of the cable catching on things makes wearing headphones more seamless in almost all situations. However while bluetooth is great when it’s working, it has plenty of opportunities to stop working and become less convenient than just plugging in a cable. And of course this means another device that needs charging - meaning I now have six devices that require charging regularly.

Basics of Functional Programming

As someone who enjoys learning new programming languages, it was only a matter of time before I came across functional programming languages, higher order functions, and the like. Earlier this year I found out that Java 8 now supports some functional programming and have been writing less boilerplate code ever since - much to the horror of my team mates. So this is for you, so you can hopefully understand my spaghetti of lambdas.

Functional programming is based around the idea of passing code around just like you would any other object. If you’re into design patterns, it’s like you’re using a very loose version of the Strategy pattern or the Template pattern. You provide a set of instructions that will be inserted into an existing algorithm or operation.

Most languages that support higher-order functions (functions that take code as a parameter) have three ‘bread and butter’ functions built-in: map, filter, and reduce. These simplify common list operations by abstracting away the boilerplate.

Map

Let’s say that I have a list of countries, and I want to present them to a user in a certain format. This is a faily common example where I have a list and I want to do an operation on each of its elements to produce a new list. You could say that there will be a mapping from each element in the first list to the element in the second list. In first year you are told to do something like this:

countries = # Some list of country objects
country_names = []
for country in countries
    country_names.push(country.name)
end
# Do something with the list of countries

However a far more succinct way of doing this is to map the list:

countries = # some list of countries
country_names = countries.map { |country| country.name }

Both methods are doing the same thing, but (for someone who understands functional programming) the second is much clearer and reduces the amount of noise in the code. Of course the disadvantage is that it can hide potentially costly operations.

An important note with map is that the operation should affect the object that you are mapping. For example if you map the countries to get all their names, but also reset some attribute of the country - you’re asking for problems in the future. If someone later decides that they only want to get the names of the first ten countries and you were relying on the fact that some other action is performed on all of them - problems are inbound.

Filter

Filter treats your function like a sieve - everything that it accepts is let through, the rest is ignored. So in this case your lambda is taking an item and returning true if you want that item to make it through the sieve. Filter reduces even more boilerplate:

let numbers = [1, 2, 5, 6, 9]
var even_numbers = [Int]()
for number in numbers {
  if number % 2 == 0 {
    even_numbers.append(number)
  }
}
// Do something with the even ones
let numbers = [1, 2, 5, 6, 9]
let even_numbers = numbers.filter { number in number % 2 == 0 }
// Do something with the even ones

You can of course chain filter statements together, or include a few conditions - basically like an SQL WHERE clause. Filter is especially useful when you have a list of objects, and you want to get rid of the ones that are null.

Reduce

When you have a list of items and want to distill it down to one object that represents some aspect of the whole list, reduce is what you’re looking for. The lambda takes two arguments - the reduced list so far, and the item that you want to reduce ‘into’ this reduced form. Reduce also takes an intial value, which is what the reduced form should start off as. A great example is summing a list of numbers - the initial reduced form is 0, and each time you want to add the current number to that.

numbers = [1, 2, 3, 6, 7]
sum = numbers.reduce(0, { |so_far, number| so_far + number })

Reduce is hard to explain - mainly because I don’t end up using it very often. Most languages include helpers for the common reduce operations: join, sum, and product are great examples. Each take a list and give you back a single value that is the combination of every item in the list.

If you think about it, both map and filter can be implemented using reduce - making reduce the only list operation you really need. So really map and filter are just helpers the common cases of reduce.

Let’s make a lambda!

So with all this knowledge, how do you go about using it? Well…

In Ruby any method that accepts a block (Ruby has lots of names for it’s anonymous functions) can be followed by a code block, either with do ... end or { ... }

In Swift closures are a type (defined by their arguments and the type they return) and like ruby can either by inside the argument list, or after the function call if the argument is at the end.

Java doesn’t really support lambdas. They are instead an anonymous implementation of an interface that has just one method. So a lambda that turns a country into a string of the country name is actually a an implementation of the generic interface Function<T, R>, (ie it’s type is Function<Country, String>) and it has a method R apply(T t) that takes in a value of type T and returns a result of type R. The code in the lambda provides the implementation of this method.

All of the list operations are hidden in the stream() method on lists, as well as the Steam.of() method that can create a stream from an Array. To turn your stream back into a list, you’ll want the .collect(Collectors.toList()) method. So the country to coutry name would look something like:

List<Country> countries = // Some list from somewhere
List<String> names = countries
    .stream()
    .map(country -> country.getName())
    .collect(Collectors.toList());

(Of course Java manages to still make a one line function into four)

Method references

If you functionally program enough, there will be some boilerplate - like creating a lambda that just calls one method on an object. So you can often just refer to that method, rather than writing out the whole lambda declaration:

(item) -> item.method()
// Can be replaced with
Item::method
{ |item| item.method() }
# Can be replaced with
&:method

If you want to learn more functional programming, Haskell, Clojure (Or Common LISP), and Elixir are all interesting.